Aug 05 17:21:31.356661 ip-10-0-134-46 systemd[1]: Starting Kubernetes Kubelet...
Aug 05 17:21:31.818094 ip-10-0-134-46 kubenswrapper[2550]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 05 17:21:31.818094 ip-10-0-134-46 kubenswrapper[2550]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Aug 05 17:21:31.818094 ip-10-0-134-46 kubenswrapper[2550]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 05 17:21:31.818094 ip-10-0-134-46 kubenswrapper[2550]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Aug 05 17:21:31.818094 ip-10-0-134-46 kubenswrapper[2550]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 05 17:21:31.820197 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.820097    2550 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Aug 05 17:21:31.828515 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828485    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:21:31.828515 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828510    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:21:31.828515 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828514    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:21:31.828515 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828517    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:21:31.828515 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828521    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:21:31.828515 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828524    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:21:31.828515 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828526    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:21:31.828781 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828529    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:21:31.828781 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828532    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:21:31.828781 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828535    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:21:31.828781 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828537    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:21:31.828781 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828540    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:21:31.828781 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828543    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:21:31.828781 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828545    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:21:31.828781 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828548    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:21:31.828781 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828550    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:21:31.828781 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828553    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:21:31.828781 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828555    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:21:31.828781 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828558    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:21:31.828781 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828560    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:21:31.828781 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828563    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:21:31.828781 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828566    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:21:31.828781 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828569    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:21:31.828781 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828572    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:21:31.828781 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828574    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:21:31.828781 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828578    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:21:31.828781 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828581    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:21:31.829253 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828584    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:21:31.829253 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828586    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:21:31.829253 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828589    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:21:31.829253 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828593    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:21:31.829253 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828597    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:21:31.829253 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828600    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:21:31.829253 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828604    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:21:31.829253 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828607    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:21:31.829253 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828610    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:21:31.829253 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828613    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:21:31.829253 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828616    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:21:31.829253 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828619    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:21:31.829253 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828622    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:21:31.829253 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828625    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:21:31.829253 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828628    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:21:31.829253 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828630    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:21:31.829253 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828633    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:21:31.829253 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828636    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:21:31.829253 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828638    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:21:31.829786 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828641    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:21:31.829786 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828644    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:21:31.829786 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828646    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:21:31.829786 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828649    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:21:31.829786 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828652    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:21:31.829786 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828655    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:21:31.829786 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828658    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:21:31.829786 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828660    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:21:31.829786 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828663    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:21:31.829786 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828665    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:21:31.829786 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828668    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:21:31.829786 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828670    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:21:31.829786 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828675    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:21:31.829786 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828677    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:21:31.829786 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828680    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:21:31.829786 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828683    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:21:31.829786 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828685    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:21:31.829786 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828688    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:21:31.829786 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828690    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:21:31.829786 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828693    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:21:31.830298 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828696    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:21:31.830298 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828699    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:21:31.830298 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828702    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:21:31.830298 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828705    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:21:31.830298 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828707    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:21:31.830298 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828710    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:21:31.830298 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828713    2550 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:21:31.830298 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828715    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:21:31.830298 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828718    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:21:31.830298 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828721    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:21:31.830298 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828723    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:21:31.830298 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828726    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:21:31.830298 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828730    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:21:31.830298 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828735    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:21:31.830298 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828737    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:21:31.830298 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828740    2550 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:21:31.830298 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828743    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:21:31.830298 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828746    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:21:31.830298 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828749    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:21:31.830298 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828752    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:21:31.830822 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828754    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:21:31.830822 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828757    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:21:31.830822 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828760    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:21:31.830822 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828762    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:21:31.830822 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828765    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:21:31.830822 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828767    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:21:31.830822 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828770    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:21:31.830822 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828772    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:21:31.830822 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828775    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:21:31.830822 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828778    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:21:31.830822 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828780    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:21:31.830822 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828783    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:21:31.830822 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.828786    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:21:31.830822 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829565    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:21:31.830822 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829574    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:21:31.830822 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829577    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:21:31.830822 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829580    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:21:31.830822 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829584    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:21:31.830822 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829587    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:21:31.831321 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829590    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:21:31.831321 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829593    2550 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:21:31.831321 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829596    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:21:31.831321 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829599    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:21:31.831321 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829602    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:21:31.831321 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829605    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:21:31.831321 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829607    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:21:31.831321 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829610    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:21:31.831321 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829613    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:21:31.831321 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829616    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:21:31.831321 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829618    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:21:31.831321 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829622    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:21:31.831321 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829624    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:21:31.831321 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829627    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:21:31.831321 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829629    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:21:31.831321 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829632    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:21:31.831321 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829636    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:21:31.831321 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829640    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:21:31.831321 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829643    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:21:31.831807 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829646    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:21:31.831807 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829648    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:21:31.831807 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829651    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:21:31.831807 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829654    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:21:31.831807 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829657    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:21:31.831807 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829660    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:21:31.831807 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829662    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:21:31.831807 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829665    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:21:31.831807 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829668    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:21:31.831807 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829671    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:21:31.831807 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829673    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:21:31.831807 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829676    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:21:31.831807 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829679    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:21:31.831807 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829682    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:21:31.831807 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829685    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:21:31.831807 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829688    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:21:31.831807 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829690    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:21:31.831807 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829693    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:21:31.831807 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829695    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:21:31.832294 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829698    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:21:31.832294 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829700    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:21:31.832294 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829703    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:21:31.832294 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829706    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:21:31.832294 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829709    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:21:31.832294 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829711    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:21:31.832294 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829714    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:21:31.832294 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829717    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:21:31.832294 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829720    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:21:31.832294 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829722    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:21:31.832294 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829725    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:21:31.832294 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829727    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:21:31.832294 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829729    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:21:31.832294 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829732    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:21:31.832294 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829734    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:21:31.832294 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829737    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:21:31.832294 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829739    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:21:31.832294 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829742    2550 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:21:31.832294 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829744    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:21:31.832294 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829747    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:21:31.832787 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829750    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:21:31.832787 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829752    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:21:31.832787 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829755    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:21:31.832787 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829757    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:21:31.832787 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829761    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:21:31.832787 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829765    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:21:31.832787 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829768    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:21:31.832787 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829772    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:21:31.832787 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829776    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:21:31.832787 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829778    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:21:31.832787 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829783    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:21:31.832787 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829786    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:21:31.832787 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829789    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:21:31.832787 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829792    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:21:31.832787 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829795    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:21:31.832787 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829799    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:21:31.832787 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829801    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:21:31.832787 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829804    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:21:31.832787 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829807    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:21:31.833237 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829810    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:21:31.833237 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829813    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:21:31.833237 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829815    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:21:31.833237 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829818    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:21:31.833237 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829821    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:21:31.833237 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829823    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:21:31.833237 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829826    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:21:31.833237 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829828    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:21:31.833237 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829831    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:21:31.833237 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829833    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:21:31.833237 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829836    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:21:31.833237 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829839    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:21:31.833237 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829841    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:21:31.833237 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829844    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:21:31.833237 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829846    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:21:31.833237 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.829849    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:21:31.833237 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830243    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:21:31.833237 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830248    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:21:31.833237 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830251    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:21:31.833237 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830254    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:21:31.833751 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830257    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:21:31.833751 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830261    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:21:31.833751 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830265    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:21:31.833751 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830268    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:21:31.833751 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830271    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:21:31.833751 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830274    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:21:31.833751 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830277    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:21:31.833751 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830280    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:21:31.833751 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830283    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:21:31.833751 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830286    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:21:31.833751 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830289    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:21:31.833751 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830291    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:21:31.833751 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830295    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:21:31.833751 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830298    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:21:31.833751 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830301    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:21:31.833751 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830304    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:21:31.833751 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830306    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:21:31.833751 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830309    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:21:31.833751 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830312    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:21:31.834226 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830315    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:21:31.834226 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830318    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:21:31.834226 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830320    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:21:31.834226 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830323    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:21:31.834226 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830340    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:21:31.834226 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830343    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:21:31.834226 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830345    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:21:31.834226 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830348    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:21:31.834226 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830350    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:21:31.834226 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830353    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:21:31.834226 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830356    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:21:31.834226 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830359    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:21:31.834226 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830362    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:21:31.834226 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830364    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:21:31.834226 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830367    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:21:31.834226 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830369    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:21:31.834226 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830372    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:21:31.834226 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830375    2550 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:21:31.834226 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830377    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:21:31.834226 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830380    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:21:31.834714 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830383    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:21:31.834714 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830385    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:21:31.834714 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830388    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:21:31.834714 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830391    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:21:31.834714 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830393    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:21:31.834714 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830396    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:21:31.834714 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830399    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:21:31.834714 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830402    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:21:31.834714 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830404    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:21:31.834714 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830407    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:21:31.834714 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830412    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:21:31.834714 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830415    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:21:31.834714 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830418    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:21:31.834714 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830420    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:21:31.834714 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830423    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:21:31.834714 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830427    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:21:31.834714 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830430    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:21:31.834714 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830435    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:21:31.834714 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830438    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:21:31.835167 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830440    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:21:31.835167 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830443    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:21:31.835167 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830446    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:21:31.835167 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830449    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:21:31.835167 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830452    2550 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:21:31.835167 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830454    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:21:31.835167 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830457    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:21:31.835167 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830459    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:21:31.835167 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830462    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:21:31.835167 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830464    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:21:31.835167 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830467    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:21:31.835167 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830469    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:21:31.835167 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830472    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:21:31.835167 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830474    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:21:31.835167 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830477    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:21:31.835167 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830480    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:21:31.835167 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830482    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:21:31.835167 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830485    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:21:31.835167 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830488    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:21:31.835167 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830490    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:21:31.835671 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830493    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:21:31.835671 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830499    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:21:31.835671 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830502    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:21:31.835671 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830504    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:21:31.835671 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830507    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:21:31.835671 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830509    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:21:31.835671 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830511    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:21:31.835671 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830514    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:21:31.835671 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830517    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:21:31.835671 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830519    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:21:31.835671 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830522    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:21:31.835671 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830525    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:21:31.835671 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830527    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:21:31.835671 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830530    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:21:31.835671 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830532    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:21:31.835671 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830535    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:21:31.835671 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.830537    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:21:31.835671 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830616    2550 flags.go:64] FLAG: --address="0.0.0.0"
Aug 05 17:21:31.835671 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830629    2550 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Aug 05 17:21:31.835671 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830637    2550 flags.go:64] FLAG: --anonymous-auth="true"
Aug 05 17:21:31.836144 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830644    2550 flags.go:64] FLAG: --application-metrics-count-limit="100"
Aug 05 17:21:31.836144 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830650    2550 flags.go:64] FLAG: --authentication-token-webhook="false"
Aug 05 17:21:31.836144 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830655    2550 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Aug 05 17:21:31.836144 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830661    2550 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Aug 05 17:21:31.836144 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830666    2550 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Aug 05 17:21:31.836144 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830669    2550 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Aug 05 17:21:31.836144 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830672    2550 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Aug 05 17:21:31.836144 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830675    2550 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Aug 05 17:21:31.836144 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830680    2550 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Aug 05 17:21:31.836144 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830683    2550 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Aug 05 17:21:31.836144 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830686    2550 flags.go:64] FLAG: --cgroup-root=""
Aug 05 17:21:31.836144 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830688    2550 flags.go:64] FLAG: --cgroups-per-qos="true"
Aug 05 17:21:31.836144 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830692    2550 flags.go:64] FLAG: --client-ca-file=""
Aug 05 17:21:31.836144 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830702    2550 flags.go:64] FLAG: --cloud-provider="external"
Aug 05 17:21:31.836144 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830706    2550 flags.go:64] FLAG: --cluster-dns="[]"
Aug 05 17:21:31.836144 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830710    2550 flags.go:64] FLAG: --cluster-domain=""
Aug 05 17:21:31.836144 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830713    2550 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Aug 05 17:21:31.836144 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830716    2550 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Aug 05 17:21:31.836144 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830719    2550 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Aug 05 17:21:31.836144 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830723    2550 flags.go:64] FLAG: --container-log-max-files="5"
Aug 05 17:21:31.836144 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830727    2550 flags.go:64] FLAG: --container-log-max-size="10Mi"
Aug 05 17:21:31.836144 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830730    2550 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Aug 05 17:21:31.836144 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830733    2550 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Aug 05 17:21:31.836780 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830736    2550 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Aug 05 17:21:31.836780 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830740    2550 flags.go:64] FLAG: --contention-profiling="false"
Aug 05 17:21:31.836780 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830743    2550 flags.go:64] FLAG: --cpu-cfs-quota="true"
Aug 05 17:21:31.836780 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830746    2550 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Aug 05 17:21:31.836780 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830749    2550 flags.go:64] FLAG: --cpu-manager-policy="none"
Aug 05 17:21:31.836780 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830752    2550 flags.go:64] FLAG: --cpu-manager-policy-options=""
Aug 05 17:21:31.836780 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830756    2550 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Aug 05 17:21:31.836780 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830759    2550 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Aug 05 17:21:31.836780 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830762    2550 flags.go:64] FLAG: --enable-debugging-handlers="true"
Aug 05 17:21:31.836780 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830765    2550 flags.go:64] FLAG: --enable-load-reader="false"
Aug 05 17:21:31.836780 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830768    2550 flags.go:64] FLAG: --enable-server="true"
Aug 05 17:21:31.836780 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830771    2550 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Aug 05 17:21:31.836780 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830775    2550 flags.go:64] FLAG: --event-burst="100"
Aug 05 17:21:31.836780 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830778    2550 flags.go:64] FLAG: --event-qps="50"
Aug 05 17:21:31.836780 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830781    2550 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Aug 05 17:21:31.836780 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830784    2550 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Aug 05 17:21:31.836780 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830787    2550 flags.go:64] FLAG: --eviction-hard=""
Aug 05 17:21:31.836780 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830791    2550 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Aug 05 17:21:31.836780 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830794    2550 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Aug 05 17:21:31.836780 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830797    2550 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Aug 05 17:21:31.836780 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830800    2550 flags.go:64] FLAG: --eviction-soft=""
Aug 05 17:21:31.836780 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830803    2550 flags.go:64] FLAG: --eviction-soft-grace-period=""
Aug 05 17:21:31.836780 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830807    2550 flags.go:64] FLAG: --exit-on-lock-contention="false"
Aug 05 17:21:31.836780 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830811    2550 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Aug 05 17:21:31.836780 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830814    2550 flags.go:64] FLAG: --experimental-mounter-path=""
Aug 05 17:21:31.837446 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830817    2550 flags.go:64] FLAG: --fail-cgroupv1="false"
Aug 05 17:21:31.837446 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830820    2550 flags.go:64] FLAG: --fail-swap-on="true"
Aug 05 17:21:31.837446 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830824    2550 flags.go:64] FLAG: --feature-gates=""
Aug 05 17:21:31.837446 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830828    2550 flags.go:64] FLAG: --file-check-frequency="20s"
Aug 05 17:21:31.837446 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830831    2550 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Aug 05 17:21:31.837446 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830834    2550 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Aug 05 17:21:31.837446 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830839    2550 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Aug 05 17:21:31.837446 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830842    2550 flags.go:64] FLAG: --healthz-port="10248"
Aug 05 17:21:31.837446 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830845    2550 flags.go:64] FLAG: --help="false"
Aug 05 17:21:31.837446 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830848    2550 flags.go:64] FLAG: --hostname-override="ip-10-0-134-46.ec2.internal"
Aug 05 17:21:31.837446 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830851    2550 flags.go:64] FLAG: --housekeeping-interval="10s"
Aug 05 17:21:31.837446 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830854    2550 flags.go:64] FLAG: --http-check-frequency="20s"
Aug 05 17:21:31.837446 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830857    2550 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Aug 05 17:21:31.837446 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830860    2550 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Aug 05 17:21:31.837446 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830863    2550 flags.go:64] FLAG: --image-gc-high-threshold="85"
Aug 05 17:21:31.837446 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830866    2550 flags.go:64] FLAG: --image-gc-low-threshold="80"
Aug 05 17:21:31.837446 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830869    2550 flags.go:64] FLAG: --image-service-endpoint=""
Aug 05 17:21:31.837446 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830872    2550 flags.go:64] FLAG: --kernel-memcg-notification="false"
Aug 05 17:21:31.837446 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830875    2550 flags.go:64] FLAG: --kube-api-burst="100"
Aug 05 17:21:31.837446 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830877    2550 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Aug 05 17:21:31.837446 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830880    2550 flags.go:64] FLAG: --kube-api-qps="50"
Aug 05 17:21:31.837446 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830883    2550 flags.go:64] FLAG: --kube-reserved=""
Aug 05 17:21:31.837446 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830886    2550 flags.go:64] FLAG: --kube-reserved-cgroup=""
Aug 05 17:21:31.837446 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830889    2550 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Aug 05 17:21:31.838020 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830892    2550 flags.go:64] FLAG: --kubelet-cgroups=""
Aug 05 17:21:31.838020 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830894    2550 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Aug 05 17:21:31.838020 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830897    2550 flags.go:64] FLAG: --lock-file=""
Aug 05 17:21:31.838020 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830900    2550 flags.go:64] FLAG: --log-cadvisor-usage="false"
Aug 05 17:21:31.838020 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830903    2550 flags.go:64] FLAG: --log-flush-frequency="5s"
Aug 05 17:21:31.838020 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830907    2550 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Aug 05 17:21:31.838020 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830913    2550 flags.go:64] FLAG: --log-json-split-stream="false"
Aug 05 17:21:31.838020 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830916    2550 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Aug 05 17:21:31.838020 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830919    2550 flags.go:64] FLAG: --log-text-split-stream="false"
Aug 05 17:21:31.838020 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830922    2550 flags.go:64] FLAG: --logging-format="text"
Aug 05 17:21:31.838020 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830925    2550 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Aug 05 17:21:31.838020 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830928    2550 flags.go:64] FLAG: --make-iptables-util-chains="true"
Aug 05 17:21:31.838020 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830931    2550 flags.go:64] FLAG: --manifest-url=""
Aug 05 17:21:31.838020 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830934    2550 flags.go:64] FLAG: --manifest-url-header=""
Aug 05 17:21:31.838020 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830941    2550 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Aug 05 17:21:31.838020 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830945    2550 flags.go:64] FLAG: --max-open-files="1000000"
Aug 05 17:21:31.838020 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830949    2550 flags.go:64] FLAG: --max-pods="110"
Aug 05 17:21:31.838020 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830952    2550 flags.go:64] FLAG: --maximum-dead-containers="-1"
Aug 05 17:21:31.838020 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830955    2550 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Aug 05 17:21:31.838020 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830958    2550 flags.go:64] FLAG: --memory-manager-policy="None"
Aug 05 17:21:31.838020 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830961    2550 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Aug 05 17:21:31.838020 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830964    2550 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Aug 05 17:21:31.838020 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830967    2550 flags.go:64] FLAG: --node-ip="0.0.0.0"
Aug 05 17:21:31.838020 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830970    2550 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Aug 05 17:21:31.838020 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830978    2550 flags.go:64] FLAG: --node-status-max-images="50"
Aug 05 17:21:31.838640 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830981    2550 flags.go:64] FLAG: --node-status-update-frequency="10s"
Aug 05 17:21:31.838640 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830984    2550 flags.go:64] FLAG: --oom-score-adj="-999"
Aug 05 17:21:31.838640 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830987    2550 flags.go:64] FLAG: --pod-cidr=""
Aug 05 17:21:31.838640 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830990    2550 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8"
Aug 05 17:21:31.838640 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830995    2550 flags.go:64] FLAG: --pod-manifest-path=""
Aug 05 17:21:31.838640 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.830998    2550 flags.go:64] FLAG: --pod-max-pids="-1"
Aug 05 17:21:31.838640 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831001    2550 flags.go:64] FLAG: --pods-per-core="0"
Aug 05 17:21:31.838640 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831004    2550 flags.go:64] FLAG: --port="10250"
Aug 05 17:21:31.838640 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831007    2550 flags.go:64] FLAG: --protect-kernel-defaults="false"
Aug 05 17:21:31.838640 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831010    2550 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05d8e672b5a89984c"
Aug 05 17:21:31.838640 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831013    2550 flags.go:64] FLAG: --qos-reserved=""
Aug 05 17:21:31.838640 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831016    2550 flags.go:64] FLAG: --read-only-port="10255"
Aug 05 17:21:31.838640 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831021    2550 flags.go:64] FLAG: --register-node="true"
Aug 05 17:21:31.838640 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831023    2550 flags.go:64] FLAG: --register-with-taints=""
Aug 05 17:21:31.838640 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831027    2550 flags.go:64] FLAG: --registry-burst="10"
Aug 05 17:21:31.838640 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831031    2550 flags.go:64] FLAG: --registry-qps="5"
Aug 05 17:21:31.838640 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831034    2550 flags.go:64] FLAG: --reserved-cpus=""
Aug 05 17:21:31.838640 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831036    2550 flags.go:64] FLAG: --reserved-memory=""
Aug 05 17:21:31.838640 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831040    2550 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Aug 05 17:21:31.838640 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831043    2550 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Aug 05 17:21:31.838640 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831046    2550 flags.go:64] FLAG: --rotate-certificates="false"
Aug 05 17:21:31.838640 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831049    2550 flags.go:64] FLAG: --rotate-server-certificates="false"
Aug 05 17:21:31.838640 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831054    2550 flags.go:64] FLAG: --runonce="false"
Aug 05 17:21:31.838640 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831057    2550 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Aug 05 17:21:31.838640 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831060    2550 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Aug 05 17:21:31.839238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831063    2550 flags.go:64] FLAG: --seccomp-default="false"
Aug 05 17:21:31.839238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831066    2550 flags.go:64] FLAG: --serialize-image-pulls="true"
Aug 05 17:21:31.839238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831068    2550 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Aug 05 17:21:31.839238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831071    2550 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Aug 05 17:21:31.839238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831074    2550 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Aug 05 17:21:31.839238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831077    2550 flags.go:64] FLAG: --storage-driver-password="root"
Aug 05 17:21:31.839238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831080    2550 flags.go:64] FLAG: --storage-driver-secure="false"
Aug 05 17:21:31.839238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831083    2550 flags.go:64] FLAG: --storage-driver-table="stats"
Aug 05 17:21:31.839238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831086    2550 flags.go:64] FLAG: --storage-driver-user="root"
Aug 05 17:21:31.839238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831089    2550 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Aug 05 17:21:31.839238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831092    2550 flags.go:64] FLAG: --sync-frequency="1m0s"
Aug 05 17:21:31.839238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831095    2550 flags.go:64] FLAG: --system-cgroups=""
Aug 05 17:21:31.839238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831098    2550 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Aug 05 17:21:31.839238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831103    2550 flags.go:64] FLAG: --system-reserved-cgroup=""
Aug 05 17:21:31.839238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831106    2550 flags.go:64] FLAG: --tls-cert-file=""
Aug 05 17:21:31.839238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831108    2550 flags.go:64] FLAG: --tls-cipher-suites="[]"
Aug 05 17:21:31.839238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831113    2550 flags.go:64] FLAG: --tls-min-version=""
Aug 05 17:21:31.839238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831116    2550 flags.go:64] FLAG: --tls-private-key-file=""
Aug 05 17:21:31.839238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831118    2550 flags.go:64] FLAG: --topology-manager-policy="none"
Aug 05 17:21:31.839238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831123    2550 flags.go:64] FLAG: --topology-manager-policy-options=""
Aug 05 17:21:31.839238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831126    2550 flags.go:64] FLAG: --topology-manager-scope="container"
Aug 05 17:21:31.839238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831129    2550 flags.go:64] FLAG: --v="2"
Aug 05 17:21:31.839238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831133    2550 flags.go:64] FLAG: --version="false"
Aug 05 17:21:31.839238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831138    2550 flags.go:64] FLAG: --vmodule=""
Aug 05 17:21:31.839238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831141    2550 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Aug 05 17:21:31.839862 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.831145    2550 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Aug 05 17:21:31.839862 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831265    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:21:31.839862 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831269    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:21:31.839862 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831272    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:21:31.839862 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831276    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:21:31.839862 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831279    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:21:31.839862 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831281    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:21:31.839862 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831284    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:21:31.839862 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831287    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:21:31.839862 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831290    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:21:31.839862 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831292    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:21:31.839862 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831295    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:21:31.839862 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831298    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:21:31.839862 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831302    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:21:31.839862 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831305    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:21:31.839862 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831315    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:21:31.839862 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831319    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:21:31.839862 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831322    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:21:31.839862 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831340    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:21:31.839862 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831343    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:21:31.840363 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831346    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:21:31.840363 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831349    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:21:31.840363 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831352    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:21:31.840363 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831354    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:21:31.840363 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831357    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:21:31.840363 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831359    2550 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:21:31.840363 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831363    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:21:31.840363 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831365    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:21:31.840363 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831368    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:21:31.840363 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831371    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:21:31.840363 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831373    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:21:31.840363 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831377    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:21:31.840363 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831379    2550 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:21:31.840363 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831382    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:21:31.840363 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831385    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:21:31.840363 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831387    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:21:31.840363 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831391    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:21:31.840363 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831394    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:21:31.840363 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831396    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:21:31.840363 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831399    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:21:31.840855 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831402    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:21:31.840855 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831404    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:21:31.840855 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831407    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:21:31.840855 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831409    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:21:31.840855 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831412    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:21:31.840855 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831415    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:21:31.840855 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831417    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:21:31.840855 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831420    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:21:31.840855 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831422    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:21:31.840855 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831425    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:21:31.840855 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831428    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:21:31.840855 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831431    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:21:31.840855 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831433    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:21:31.840855 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831436    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:21:31.840855 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831439    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:21:31.840855 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831441    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:21:31.840855 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831444    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:21:31.840855 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831446    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:21:31.840855 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831449    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:21:31.840855 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831452    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:21:31.841388 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831454    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:21:31.841388 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831457    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:21:31.841388 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831460    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:21:31.841388 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831462    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:21:31.841388 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831465    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:21:31.841388 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831468    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:21:31.841388 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831470    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:21:31.841388 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831473    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:21:31.841388 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831476    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:21:31.841388 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831479    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:21:31.841388 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831482    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:21:31.841388 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831484    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:21:31.841388 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831487    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:21:31.841388 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831489    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:21:31.841388 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831492    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:21:31.841388 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831494    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:21:31.841388 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831497    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:21:31.841388 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831500    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:21:31.841388 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831502    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:21:31.841388 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831506    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:21:31.841858 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831510    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:21:31.841858 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831512    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:21:31.841858 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831515    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:21:31.841858 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831518    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:21:31.841858 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831520    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:21:31.841858 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831523    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:21:31.841858 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831525    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:21:31.841858 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831528    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:21:31.841858 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831530    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:21:31.841858 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831533    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:21:31.841858 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831536    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:21:31.841858 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831538    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:21:31.841858 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831541    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:21:31.841858 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831543    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:21:31.841858 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831546    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:21:31.841858 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831550    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:21:31.841858 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831553    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:21:31.841858 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831557    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:21:31.841858 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831560    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:21:31.842318 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.831562    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:21:31.842318 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.832314    2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 05 17:21:31.842318 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.839539    2550 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Aug 05 17:21:31.842318 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.839558    2550 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Aug 05 17:21:31.842318 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.840463    2550 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Aug 05 17:21:31.842318 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.840473    2550 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Aug 05 17:21:31.842318 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840534    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:21:31.842318 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840539    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:21:31.842318 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840542    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:21:31.842318 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840545    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:21:31.842318 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840549    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:21:31.842318 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840551    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:21:31.842318 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840554    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:21:31.842318 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840557    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:21:31.842681 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840560    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:21:31.842681 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840562    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:21:31.842681 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840565    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:21:31.842681 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840568    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:21:31.842681 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840570    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:21:31.842681 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840573    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:21:31.842681 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840576    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:21:31.842681 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840579    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:21:31.842681 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840581    2550 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:21:31.842681 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840584    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:21:31.842681 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840586    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:21:31.842681 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840589    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:21:31.842681 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840591    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:21:31.842681 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840594    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:21:31.842681 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840597    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:21:31.842681 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840600    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:21:31.842681 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840602    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:21:31.842681 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840605    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:21:31.842681 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840608    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:21:31.842681 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840610    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:21:31.843159 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840613    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:21:31.843159 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840617    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:21:31.843159 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840620    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:21:31.843159 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840623    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:21:31.843159 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840625    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:21:31.843159 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840628    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:21:31.843159 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840631    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:21:31.843159 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840633    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:21:31.843159 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840636    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:21:31.843159 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840638    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:21:31.843159 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840641    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:21:31.843159 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840643    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:21:31.843159 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840646    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:21:31.843159 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840648    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:21:31.843159 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840651    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:21:31.843159 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840653    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:21:31.843159 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840656    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:21:31.843159 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840658    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:21:31.843159 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840661    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:21:31.843159 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840663    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:21:31.843678 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840666    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:21:31.843678 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840668    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:21:31.843678 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840671    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:21:31.843678 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840673    2550 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:21:31.843678 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840676    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:21:31.843678 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840679    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:21:31.843678 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840682    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:21:31.843678 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840685    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:21:31.843678 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840687    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:21:31.843678 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840690    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:21:31.843678 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840693    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:21:31.843678 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840697    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:21:31.843678 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840700    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:21:31.843678 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840703    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:21:31.843678 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840706    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:21:31.843678 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840709    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:21:31.843678 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840712    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:21:31.843678 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840715    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:21:31.843678 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840718    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:21:31.843678 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840720    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:21:31.844169 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840723    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:21:31.844169 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840725    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:21:31.844169 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840728    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:21:31.844169 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840730    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:21:31.844169 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840733    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:21:31.844169 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840736    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:21:31.844169 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840738    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:21:31.844169 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840741    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:21:31.844169 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840743    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:21:31.844169 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840745    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:21:31.844169 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840749    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:21:31.844169 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840752    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:21:31.844169 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840754    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:21:31.844169 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840757    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:21:31.844169 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840760    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:21:31.844169 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840762    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:21:31.844169 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840766    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:21:31.844169 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840770    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:21:31.844169 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840773    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:21:31.844169 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840776    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:21:31.844674 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840779    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:21:31.844674 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840781    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:21:31.844674 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840784    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:21:31.844674 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840787    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:21:31.844674 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840789    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:21:31.844674 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840792    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:21:31.844674 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840796    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:21:31.844674 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840800    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:21:31.844674 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840803    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:21:31.844674 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840805    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:21:31.844674 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840808    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:21:31.844674 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.840813    2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 05 17:21:31.844674 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840929    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:21:31.844674 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840933    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:21:31.845016 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840936    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:21:31.845016 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840939    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:21:31.845016 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840942    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:21:31.845016 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840944    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:21:31.845016 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840947    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:21:31.845016 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840949    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:21:31.845016 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840952    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:21:31.845016 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840955    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:21:31.845016 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840957    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:21:31.845016 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840960    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:21:31.845016 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840963    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:21:31.845016 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840965    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:21:31.845016 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840968    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:21:31.845016 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840971    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:21:31.845016 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840973    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:21:31.845016 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840976    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:21:31.845016 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840979    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:21:31.845016 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840982    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:21:31.845016 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840984    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:21:31.845491 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840987    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:21:31.845491 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840989    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:21:31.845491 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840992    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:21:31.845491 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840994    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:21:31.845491 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840997    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:21:31.845491 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.840999    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:21:31.845491 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841002    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:21:31.845491 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841005    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:21:31.845491 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841007    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:21:31.845491 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841010    2550 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:21:31.845491 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841012    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:21:31.845491 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841015    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:21:31.845491 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841017    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:21:31.845491 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841020    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:21:31.845491 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841022    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:21:31.845491 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841025    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:21:31.845491 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841027    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:21:31.845491 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841030    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:21:31.845491 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841032    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:21:31.845491 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841035    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:21:31.845978 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841038    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:21:31.845978 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841040    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:21:31.845978 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841042    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:21:31.845978 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841045    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:21:31.845978 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841047    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:21:31.845978 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841050    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:21:31.845978 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841053    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:21:31.845978 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841056    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:21:31.845978 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841060    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:21:31.845978 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841064    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:21:31.845978 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841066    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:21:31.845978 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841069    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:21:31.845978 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841071    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:21:31.845978 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841074    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:21:31.845978 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841076    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:21:31.845978 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841079    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:21:31.845978 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841081    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:21:31.845978 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841084    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:21:31.845978 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841086    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:21:31.846497 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841089    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:21:31.846497 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841092    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:21:31.846497 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841094    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:21:31.846497 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841097    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:21:31.846497 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841099    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:21:31.846497 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841102    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:21:31.846497 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841104    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:21:31.846497 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841107    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:21:31.846497 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841109    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:21:31.846497 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841112    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:21:31.846497 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841114    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:21:31.846497 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841117    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:21:31.846497 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841119    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:21:31.846497 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841122    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:21:31.846497 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841124    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:21:31.846497 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841127    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:21:31.846497 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841130    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:21:31.846497 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841132    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:21:31.846497 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841134    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:21:31.846497 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841137    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:21:31.846974 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841139    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:21:31.846974 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841142    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:21:31.846974 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841145    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:21:31.846974 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841148    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:21:31.846974 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841151    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:21:31.846974 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841155    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:21:31.846974 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841159    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:21:31.846974 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841162    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:21:31.846974 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841165    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:21:31.846974 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841168    2550 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:21:31.846974 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841171    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:21:31.846974 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841174    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:21:31.846974 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841177    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:21:31.846974 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841180    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:21:31.846974 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841183    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:21:31.846974 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841185    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:21:31.846974 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841188    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:21:31.846974 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841193    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:21:31.846974 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:31.841196    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:21:31.847452 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.841200    2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 05 17:21:31.847452 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.841345    2550 server.go:962] "Client rotation is on, will bootstrap in background"
Aug 05 17:21:31.847452 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.843784    2550 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Aug 05 17:21:31.847452 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.845527    2550 server.go:1019] "Starting client certificate rotation"
Aug 05 17:21:31.847452 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.845624    2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 05 17:21:31.847452 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.845657    2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 05 17:21:31.848283 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.848268    2550 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 05 17:21:31.853030 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.853007    2550 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 05 17:21:31.865598 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.865570    2550 log.go:25] "Validated CRI v1 runtime API"
Aug 05 17:21:31.868190 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.868171    2550 log.go:25] "Validated CRI v1 image API"
Aug 05 17:21:31.868993 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.868975    2550 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Aug 05 17:21:31.872355 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.872308    2550 fs.go:135] Filesystem UUIDs: map[11c986ac-8df2-428d-9258-98a02e0042db:/dev/nvme0n1p3 17f75454-b09b-46ad-a63b-a3f58f1caa9c:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Aug 05 17:21:31.872420 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.872352    2550 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Aug 05 17:21:31.877337 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.877308    2550 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 05 17:21:31.878483 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.878367    2550 manager.go:216] Machine: {Timestamp:2026-08-05 17:21:31.876890546 +0000 UTC m=+0.381838753 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099826 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2351bb5eb57f18ff590e4eaf8c02ec SystemUUID:ec2351bb-5eb5-7f18-ff59-0e4eaf8c02ec BootID:d5714152-8212-4ae0-b67e-fb7ec3380845 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9f:48:6d:01:21 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9f:48:6d:01:21 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:48:2d:9a:fb:6d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Aug 05 17:21:31.878483 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.878478    2550 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Aug 05 17:21:31.878619 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.878608    2550 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Aug 05 17:21:31.882143 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.882110    2550 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Aug 05 17:21:31.882288 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.882145    2550 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-46.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Aug 05 17:21:31.882987 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.882976    2550 topology_manager.go:138] "Creating topology manager with none policy"
Aug 05 17:21:31.883024 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.882989    2550 container_manager_linux.go:309] "Creating device plugin manager"
Aug 05 17:21:31.883024 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.882997    2550 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 05 17:21:31.883024 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.883013    2550 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 05 17:21:31.884307 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.884296    2550 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Aug 05 17:21:31.889675 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.889661    2550 state_mem.go:36] "Initialized new in-memory state store"
Aug 05 17:21:31.889791 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.889781    2550 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Aug 05 17:21:31.889833 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.889812    2550 kubelet.go:486] "Attempting to sync node with API server"
Aug 05 17:21:31.889833 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.889821    2550 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Aug 05 17:21:31.889904 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.889835    2550 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Aug 05 17:21:31.889904 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.889846    2550 kubelet.go:398] "Adding apiserver pod source"
Aug 05 17:21:31.889904 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.889856    2550 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Aug 05 17:21:31.890246 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.890233    2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 05 17:21:31.890279 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.890255    2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 05 17:21:31.892664 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.892648    2550 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Aug 05 17:21:31.894562 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.894546    2550 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Aug 05 17:21:31.894627 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.894578    2550 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Aug 05 17:21:31.896158 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.896146    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Aug 05 17:21:31.896210 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.896164    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Aug 05 17:21:31.896210 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.896207    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Aug 05 17:21:31.896287 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.896215    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Aug 05 17:21:31.896287 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.896225    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Aug 05 17:21:31.896287 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.896232    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Aug 05 17:21:31.896287 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.896241    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Aug 05 17:21:31.896287 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.896246    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Aug 05 17:21:31.896287 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.896253    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Aug 05 17:21:31.896287 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.896258    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Aug 05 17:21:31.896287 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.896266    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Aug 05 17:21:31.896287 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.896271    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Aug 05 17:21:31.897165 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.897153    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Aug 05 17:21:31.897165 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.897164    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Aug 05 17:21:31.900361 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.900312    2550 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-46.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Aug 05 17:21:31.901236 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:31.901211    2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-46.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Aug 05 17:21:31.901537 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:31.901211    2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Aug 05 17:21:31.902182 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.902160    2550 server.go:1268] "Started kubelet"
Aug 05 17:21:31.903022 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.902988    2550 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Aug 05 17:21:31.903126 ip-10-0-134-46 systemd[1]: Started Kubernetes Kubelet.
Aug 05 17:21:31.903289 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.902967    2550 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Aug 05 17:21:31.903400 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.903322    2550 server_v1.go:49] "podresources" method="list" useActivePods=true
Aug 05 17:21:31.906082 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.906053    2550 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Aug 05 17:21:31.906183 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.906169    2550 server.go:310] "Adding debug handlers to kubelet server"
Aug 05 17:21:31.908245 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:31.908225    2550 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Aug 05 17:21:31.913840 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:31.912779    2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-46.ec2.internal.18c8f88290429a97  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-46.ec2.internal,UID:ip-10-0-134-46.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-46.ec2.internal,},FirstTimestamp:2026-08-05 17:21:31.902122647 +0000 UTC m=+0.407070857,LastTimestamp:2026-08-05 17:21:31.902122647 +0000 UTC m=+0.407070857,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-46.ec2.internal,}"
Aug 05 17:21:31.953017 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.952984    2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zwh9q"
Aug 05 17:21:31.960859 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:31.960833    2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zwh9q"
Aug 05 17:21:32.067982 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.067950    2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Aug 05 17:21:32.068077 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.067936    2550 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Aug 05 17:21:32.068202 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:32.068135    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-46.ec2.internal\" not found"
Aug 05 17:21:32.068878 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.068862    2550 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Aug 05 17:21:32.068878 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.068862    2550 volume_manager.go:311] "The desired_state_of_world populator starts"
Aug 05 17:21:32.069040 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.068887    2550 volume_manager.go:313] "Starting Kubelet Volume Manager"
Aug 05 17:21:32.069040 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.068931    2550 factory.go:55] Registering systemd factory
Aug 05 17:21:32.069040 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.068968    2550 factory.go:223] Registration of the systemd container factory successfully
Aug 05 17:21:32.069040 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.068997    2550 reconstruct.go:97] "Volume reconstruction finished"
Aug 05 17:21:32.069040 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.069005    2550 reconciler.go:29] "Reconciler: start to sync state"
Aug 05 17:21:32.069308 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.069232    2550 factory.go:153] Registering CRI-O factory
Aug 05 17:21:32.069308 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.069250    2550 factory.go:223] Registration of the crio container factory successfully
Aug 05 17:21:32.069308 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.069303    2550 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Aug 05 17:21:32.069447 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.069323    2550 factory.go:103] Registering Raw factory
Aug 05 17:21:32.069447 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.069351    2550 manager.go:1196] Started watching for new ooms in manager
Aug 05 17:21:32.070514 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.070498    2550 manager.go:349] Starting recovery of all containers
Aug 05 17:21:32.077790 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.077764    2550 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 05 17:21:32.080585 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.080565    2550 manager.go:354] Recovery completed
Aug 05 17:21:32.080705 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:32.080601    2550 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-134-46.ec2.internal\" not found" node="ip-10-0-134-46.ec2.internal"
Aug 05 17:21:32.084891 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.084872    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:21:32.087050 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.087028    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-46.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:21:32.087124 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.087067    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-46.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:21:32.087124 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.087081    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-46.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:21:32.087725 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.087713    2550 cpu_manager.go:222] "Starting CPU manager" policy="none"
Aug 05 17:21:32.087725 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.087724    2550 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Aug 05 17:21:32.087801 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.087741    2550 state_mem.go:36] "Initialized new in-memory state store"
Aug 05 17:21:32.087801 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.087777    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:21:32.088963 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.088945    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-46.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:21:32.089052 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.088971    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-46.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:21:32.089052 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.088982    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-46.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:21:32.089964 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.089950    2550 policy_none.go:49] "None policy: Start"
Aug 05 17:21:32.090028 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.089968    2550 memory_manager.go:187] "Starting memorymanager" policy="None"
Aug 05 17:21:32.090028 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.089981    2550 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Aug 05 17:21:32.092094 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.092080    2550 policy_none.go:47] "Start"
Aug 05 17:21:32.132158 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.131993    2550 manager.go:337] "Starting Device Plugin manager"
Aug 05 17:21:32.145705 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:32.132197    2550 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Aug 05 17:21:32.145705 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.132214    2550 server.go:87] "Starting device plugin registration server"
Aug 05 17:21:32.145705 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.132491    2550 eviction_manager.go:189] "Eviction manager: starting control loop"
Aug 05 17:21:32.145705 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.132502    2550 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Aug 05 17:21:32.145705 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.132610    2550 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Aug 05 17:21:32.145705 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.132698    2550 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Aug 05 17:21:32.145705 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.132707    2550 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Aug 05 17:21:32.145705 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:32.133173    2550 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Aug 05 17:21:32.145705 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:32.133225    2550 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-46.ec2.internal\" not found"
Aug 05 17:21:32.169207 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.169165    2550 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Aug 05 17:21:32.170401 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.170383    2550 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Aug 05 17:21:32.170502 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.170422    2550 status_manager.go:244] "Starting to sync pod status with apiserver"
Aug 05 17:21:32.170502 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.170447    2550 kubelet.go:2448] "Starting kubelet main sync loop"
Aug 05 17:21:32.170502 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:32.170484    2550 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Aug 05 17:21:32.172738 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.172718    2550 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 05 17:21:32.233520 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.233480    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:21:32.234465 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.234447    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-46.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:21:32.234575 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.234483    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-46.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:21:32.234575 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.234500    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-46.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:21:32.234575 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.234530    2550 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-46.ec2.internal"
Aug 05 17:21:32.249366 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.249343    2550 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-46.ec2.internal"
Aug 05 17:21:32.249428 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:32.249371    2550 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-46.ec2.internal\": node \"ip-10-0-134-46.ec2.internal\" not found"
Aug 05 17:21:32.260921 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:32.260894    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-46.ec2.internal\" not found"
Aug 05 17:21:32.271247 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.271220    2550 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-46.ec2.internal"]
Aug 05 17:21:32.271369 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.271298    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:21:32.272738 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.272715    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-46.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:21:32.272836 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.272749    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-46.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:21:32.272836 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.272760    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-46.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:21:32.275003 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.274991    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:21:32.275110 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.275096    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.ec2.internal"
Aug 05 17:21:32.275149 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.275126    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:21:32.275755 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.275740    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-46.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:21:32.275755 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.275747    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-46.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:21:32.275841 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.275769    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-46.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:21:32.275841 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.275771    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-46.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:21:32.275841 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.275782    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-46.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:21:32.275841 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.275789    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-46.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:21:32.278096 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.278082    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-46.ec2.internal"
Aug 05 17:21:32.278165 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.278108    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:21:32.278762 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.278749    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-46.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:21:32.278824 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.278775    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-46.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:21:32.278824 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.278784    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-46.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:21:32.306085 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:32.306055    2550 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-46.ec2.internal\" not found" node="ip-10-0-134-46.ec2.internal"
Aug 05 17:21:32.310773 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:32.310755    2550 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-46.ec2.internal\" not found" node="ip-10-0-134-46.ec2.internal"
Aug 05 17:21:32.361183 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:32.361100    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-46.ec2.internal\" not found"
Aug 05 17:21:32.370433 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.370408    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/af6ae06b6895f6ad9d2e2bf995b0fd40-config\") pod \"kube-apiserver-proxy-ip-10-0-134-46.ec2.internal\" (UID: \"af6ae06b6895f6ad9d2e2bf995b0fd40\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-46.ec2.internal"
Aug 05 17:21:32.370527 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.370442    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c95b41ba2403920dcca145af0d4b9bb1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-46.ec2.internal\" (UID: \"c95b41ba2403920dcca145af0d4b9bb1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.ec2.internal"
Aug 05 17:21:32.370527 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.370471    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c95b41ba2403920dcca145af0d4b9bb1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-46.ec2.internal\" (UID: \"c95b41ba2403920dcca145af0d4b9bb1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.ec2.internal"
Aug 05 17:21:32.461699 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:32.461667    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-46.ec2.internal\" not found"
Aug 05 17:21:32.471023 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.470987    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/af6ae06b6895f6ad9d2e2bf995b0fd40-config\") pod \"kube-apiserver-proxy-ip-10-0-134-46.ec2.internal\" (UID: \"af6ae06b6895f6ad9d2e2bf995b0fd40\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-46.ec2.internal"
Aug 05 17:21:32.471096 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.471029    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c95b41ba2403920dcca145af0d4b9bb1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-46.ec2.internal\" (UID: \"c95b41ba2403920dcca145af0d4b9bb1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.ec2.internal"
Aug 05 17:21:32.471096 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.471059    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c95b41ba2403920dcca145af0d4b9bb1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-46.ec2.internal\" (UID: \"c95b41ba2403920dcca145af0d4b9bb1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.ec2.internal"
Aug 05 17:21:32.471156 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.471095    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/af6ae06b6895f6ad9d2e2bf995b0fd40-config\") pod \"kube-apiserver-proxy-ip-10-0-134-46.ec2.internal\" (UID: \"af6ae06b6895f6ad9d2e2bf995b0fd40\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-46.ec2.internal"
Aug 05 17:21:32.471156 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.471103    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c95b41ba2403920dcca145af0d4b9bb1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-46.ec2.internal\" (UID: \"c95b41ba2403920dcca145af0d4b9bb1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.ec2.internal"
Aug 05 17:21:32.471156 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:32.471096    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c95b41ba2403920dcca145af0d4b9bb1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-46.ec2.internal\" (UID: \"c95b41ba2403920dcca145af0d4b9bb1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.ec2.internal"
Aug 05 17:21:34.085572 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:34.085542    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-46.ec2.internal\" not found"
Aug 05 17:21:34.132088 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.132051    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.ec2.internal"
Aug 05 17:21:34.137427 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.137271    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-46.ec2.internal"
Aug 05 17:21:34.186153 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:34.186122    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-46.ec2.internal\" not found"
Aug 05 17:21:34.286663 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:34.286628    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-46.ec2.internal\" not found"
Aug 05 17:21:34.329807 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.329784    2550 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 05 17:21:34.367727 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.367690    2550 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Aug 05 17:21:34.368150 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.367810    2550 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 05 17:21:34.368150 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.367854    2550 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 05 17:21:34.391744 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.391661    2550 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.ec2.internal"
Aug 05 17:21:34.405331 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.405308    2550 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 05 17:21:34.406616 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.406602    2550 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-46.ec2.internal"
Aug 05 17:21:34.413107 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.413087    2550 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 05 17:21:34.414168 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.414157    2550 apiserver.go:52] "Watching apiserver"
Aug 05 17:21:34.422101 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.422072    2550 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Aug 05 17:21:34.422439 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.422414    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-bpcrr","openshift-multus/multus-fbk5k","openshift-multus/network-metrics-daemon-rkln9","openshift-network-diagnostics/network-check-target-4jfwd","openshift-ovn-kubernetes/ovnkube-node-thnnz","kube-system/konnectivity-agent-g55gw","kube-system/kube-apiserver-proxy-ip-10-0-134-46.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4","openshift-cluster-node-tuning-operator/tuned-srf7f","openshift-dns/node-resolver-2f92r","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.ec2.internal","openshift-multus/multus-additional-cni-plugins-xhnch","openshift-network-operator/iptables-alerter-8gvbd","kube-system/global-pull-secret-syncer-h8vbl"]
Aug 05 17:21:34.425019 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.424997    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4"
Aug 05 17:21:34.427865 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.427840    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Aug 05 17:21:34.427865 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.427859    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Aug 05 17:21:34.427991 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.427859    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Aug 05 17:21:34.427991 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.427982    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-q4lw4\""
Aug 05 17:21:34.428560 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.428536    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.430676 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.430658    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:21:34.430777 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:34.430747    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkln9" podUID="0fca5e3d-e0e1-4b33-909c-46599a84cf4a"
Aug 05 17:21:34.431087 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.431073    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Aug 05 17:21:34.431403 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.431379    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Aug 05 17:21:34.431486 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.431454    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6xh9w\""
Aug 05 17:21:34.431486 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.431455    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Aug 05 17:21:34.431589 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.431522    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Aug 05 17:21:34.432867 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.432853    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xhnch"
Aug 05 17:21:34.434906 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.434891    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2pr4m\""
Aug 05 17:21:34.435000 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.434925    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Aug 05 17:21:34.435000 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.434968    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.435000 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.434986    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Aug 05 17:21:34.437123 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.437105    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Aug 05 17:21:34.437201 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.437131    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g55gw"
Aug 05 17:21:34.437201 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.437175    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6bmfv\""
Aug 05 17:21:34.437324 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.437309    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Aug 05 17:21:34.439268 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.439250    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Aug 05 17:21:34.439369 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.439292    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-p2tnh\""
Aug 05 17:21:34.439369 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.439302    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bpcrr"
Aug 05 17:21:34.439369 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.439302    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Aug 05 17:21:34.441270 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.441250    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Aug 05 17:21:34.441362 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.441301    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Aug 05 17:21:34.441423 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.441408    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Aug 05 17:21:34.441573 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.441556    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vq56f\""
Aug 05 17:21:34.441741 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.441724    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.443802 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.443785    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Aug 05 17:21:34.443895 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.443886    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Aug 05 17:21:34.443958 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.443930    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Aug 05 17:21:34.444238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.444226    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2f92r"
Aug 05 17:21:34.444836 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.444814    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-75mpn\""
Aug 05 17:21:34.444943 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.444925    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Aug 05 17:21:34.444943 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.444926    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Aug 05 17:21:34.445082 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.444928    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Aug 05 17:21:34.446470 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.446442    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Aug 05 17:21:34.446560 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.446498    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:21:34.446613 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.446573    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sfx75\""
Aug 05 17:21:34.446663 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:34.446624    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4jfwd" podUID="f8972708-830a-4ebc-b521-8a16ca1aee39"
Aug 05 17:21:34.446776 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.446656    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Aug 05 17:21:34.453072 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.452329    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8gvbd"
Aug 05 17:21:34.455238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.455218    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Aug 05 17:21:34.455335 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.455254    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Aug 05 17:21:34.455528 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.455502    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6m8rt\""
Aug 05 17:21:34.455528 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.455508    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Aug 05 17:21:34.455692 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.455640    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:21:34.455779 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:34.455717    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8vbl" podUID="bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f"
Aug 05 17:21:34.493176 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.493154    2550 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Aug 05 17:21:34.497547 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.497523    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bf34f30e-7b6a-49fc-a27f-e1955f248d7c-konnectivity-ca\") pod \"konnectivity-agent-g55gw\" (UID: \"bf34f30e-7b6a-49fc-a27f-e1955f248d7c\") " pod="kube-system/konnectivity-agent-g55gw"
Aug 05 17:21:34.497719 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.497558    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-kubelet-config\") pod \"global-pull-secret-syncer-h8vbl\" (UID: \"bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f\") " pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:21:34.497719 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.497579    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-multus-cni-dir\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.497719 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.497597    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee639fbc-354e-4612-bf13-47c4e4df965f-host-slash\") pod \"iptables-alerter-8gvbd\" (UID: \"ee639fbc-354e-4612-bf13-47c4e4df965f\") " pod="openshift-network-operator/iptables-alerter-8gvbd"
Aug 05 17:21:34.497719 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.497633    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7-socket-dir\") pod \"aws-ebs-csi-driver-node-9ftv4\" (UID: \"0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4"
Aug 05 17:21:34.497719 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.497657    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7-sys-fs\") pod \"aws-ebs-csi-driver-node-9ftv4\" (UID: \"0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4"
Aug 05 17:21:34.497719 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.497673    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-os-release\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.497719 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.497694    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-host-run-k8s-cni-cncf-io\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.497977 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.497731    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-hostroot\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.497977 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.497745    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/749fcabe-941f-454a-b97e-49d077abd98b-etc-tuned\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.497977 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.497760    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-host-run-netns\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.497977 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.497775    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-etc-openvswitch\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.497977 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.497793    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-ovnkube-script-lib\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.497977 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.497816    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-host-var-lib-kubelet\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.497977 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.497837    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-host-run-multus-certs\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.497977 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.497884    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5bd15c5f-5f78-4cba-b190-2e22851079f8-cni-binary-copy\") pod \"multus-additional-cni-plugins-xhnch\" (UID: \"5bd15c5f-5f78-4cba-b190-2e22851079f8\") " pod="openshift-multus/multus-additional-cni-plugins-xhnch"
Aug 05 17:21:34.497977 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.497902    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-log-socket\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.497977 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.497918    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-etc-systemd\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.497977 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.497951    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.498434 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.497988    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-env-overrides\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.498434 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498016    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-system-cni-dir\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.498434 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498040    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-etc-kubernetes\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.498434 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498064    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-etc-modprobe-d\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.498434 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498091    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4smm\" (UniqueName: \"kubernetes.io/projected/ee639fbc-354e-4612-bf13-47c4e4df965f-kube-api-access-n4smm\") pod \"iptables-alerter-8gvbd\" (UID: \"ee639fbc-354e-4612-bf13-47c4e4df965f\") " pod="openshift-network-operator/iptables-alerter-8gvbd"
Aug 05 17:21:34.498434 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498131    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/882b422e-055d-423f-b318-c8e66a901dd9-multus-daemon-config\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.498434 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498178    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdqpz\" (UniqueName: \"kubernetes.io/projected/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-kube-api-access-sdqpz\") pod \"network-metrics-daemon-rkln9\" (UID: \"0fca5e3d-e0e1-4b33-909c-46599a84cf4a\") " pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:21:34.498434 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498216    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5bd15c5f-5f78-4cba-b190-2e22851079f8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xhnch\" (UID: \"5bd15c5f-5f78-4cba-b190-2e22851079f8\") " pod="openshift-multus/multus-additional-cni-plugins-xhnch"
Aug 05 17:21:34.498434 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498250    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-etc-sysctl-conf\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.498434 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498269    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/610a012b-bd9a-4627-8976-b1f785945d61-host\") pod \"node-ca-bpcrr\" (UID: \"610a012b-bd9a-4627-8976-b1f785945d61\") " pod="openshift-image-registry/node-ca-bpcrr"
Aug 05 17:21:34.498434 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498284    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-host-var-lib-cni-multus\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.498434 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498316    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpdmv\" (UniqueName: \"kubernetes.io/projected/5bd15c5f-5f78-4cba-b190-2e22851079f8-kube-api-access-dpdmv\") pod \"multus-additional-cni-plugins-xhnch\" (UID: \"5bd15c5f-5f78-4cba-b190-2e22851079f8\") " pod="openshift-multus/multus-additional-cni-plugins-xhnch"
Aug 05 17:21:34.498434 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498352    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-host-kubelet\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.498434 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498400    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-host-run-ovn-kubernetes\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.498434 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498425    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-host-cni-netd\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.499106 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498448    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx2f2\" (UniqueName: \"kubernetes.io/projected/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-kube-api-access-hx2f2\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.499106 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498496    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hld45\" (UniqueName: \"kubernetes.io/projected/0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7-kube-api-access-hld45\") pod \"aws-ebs-csi-driver-node-9ftv4\" (UID: \"0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4"
Aug 05 17:21:34.499106 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498535    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-multus-socket-dir-parent\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.499106 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498562    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwpmc\" (UniqueName: \"kubernetes.io/projected/882b422e-055d-423f-b318-c8e66a901dd9-kube-api-access-rwpmc\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.499106 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498584    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5bd15c5f-5f78-4cba-b190-2e22851079f8-system-cni-dir\") pod \"multus-additional-cni-plugins-xhnch\" (UID: \"5bd15c5f-5f78-4cba-b190-2e22851079f8\") " pod="openshift-multus/multus-additional-cni-plugins-xhnch"
Aug 05 17:21:34.499106 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498606    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-original-pull-secret\") pod \"global-pull-secret-syncer-h8vbl\" (UID: \"bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f\") " pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:21:34.499106 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498628    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7-registration-dir\") pod \"aws-ebs-csi-driver-node-9ftv4\" (UID: \"0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4"
Aug 05 17:21:34.499106 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498651    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-lib-modules\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.499106 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498718    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-var-lib-kubelet\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.499106 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498767    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-etc-sysconfig\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.499106 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498796    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-etc-kubernetes\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.499106 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498820    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-etc-sysctl-d\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.499106 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498859    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/749fcabe-941f-454a-b97e-49d077abd98b-tmp\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.499106 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498886    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pb5jf\" (UniqueName: \"kubernetes.io/projected/749fcabe-941f-454a-b97e-49d077abd98b-kube-api-access-pb5jf\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.499106 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498902    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-systemd-units\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.499106 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498917    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b382a0b8-435d-430a-a8a2-faa3d0b3bdc1-hosts-file\") pod \"node-resolver-2f92r\" (UID: \"b382a0b8-435d-430a-a8a2-faa3d0b3bdc1\") " pod="openshift-dns/node-resolver-2f92r"
Aug 05 17:21:34.499782 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498937    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bf34f30e-7b6a-49fc-a27f-e1955f248d7c-agent-certs\") pod \"konnectivity-agent-g55gw\" (UID: \"bf34f30e-7b6a-49fc-a27f-e1955f248d7c\") " pod="kube-system/konnectivity-agent-g55gw"
Aug 05 17:21:34.499782 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498959    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlpgq\" (UniqueName: \"kubernetes.io/projected/610a012b-bd9a-4627-8976-b1f785945d61-kube-api-access-mlpgq\") pod \"node-ca-bpcrr\" (UID: \"610a012b-bd9a-4627-8976-b1f785945d61\") " pod="openshift-image-registry/node-ca-bpcrr"
Aug 05 17:21:34.499782 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498973    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs\") pod \"network-metrics-daemon-rkln9\" (UID: \"0fca5e3d-e0e1-4b33-909c-46599a84cf4a\") " pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:21:34.499782 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.498990    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5bd15c5f-5f78-4cba-b190-2e22851079f8-cnibin\") pod \"multus-additional-cni-plugins-xhnch\" (UID: \"5bd15c5f-5f78-4cba-b190-2e22851079f8\") " pod="openshift-multus/multus-additional-cni-plugins-xhnch"
Aug 05 17:21:34.499782 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499008    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-sys\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.499782 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499054    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-host-slash\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.499782 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499084    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7-etc-selinux\") pod \"aws-ebs-csi-driver-node-9ftv4\" (UID: \"0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4"
Aug 05 17:21:34.499782 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499107    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-cnibin\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.499782 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499130    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-host\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.499782 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499156    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-run\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.499782 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499179    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-ovnkube-config\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.499782 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499207    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-ovn-node-metrics-cert\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.499782 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499233    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/882b422e-055d-423f-b318-c8e66a901dd9-cni-binary-copy\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.499782 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499263    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-host-var-lib-cni-bin\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.499782 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499288    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5bd15c5f-5f78-4cba-b190-2e22851079f8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xhnch\" (UID: \"5bd15c5f-5f78-4cba-b190-2e22851079f8\") " pod="openshift-multus/multus-additional-cni-plugins-xhnch"
Aug 05 17:21:34.499782 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499312    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-var-lib-openvswitch\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.499782 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499334    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-run-ovn\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.500292 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499357    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-host-cni-bin\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.500292 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499381    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw8dm\" (UniqueName: \"kubernetes.io/projected/b382a0b8-435d-430a-a8a2-faa3d0b3bdc1-kube-api-access-mw8dm\") pod \"node-resolver-2f92r\" (UID: \"b382a0b8-435d-430a-a8a2-faa3d0b3bdc1\") " pod="openshift-dns/node-resolver-2f92r"
Aug 05 17:21:34.500292 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499433    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ee639fbc-354e-4612-bf13-47c4e4df965f-iptables-alerter-script\") pod \"iptables-alerter-8gvbd\" (UID: \"ee639fbc-354e-4612-bf13-47c4e4df965f\") " pod="openshift-network-operator/iptables-alerter-8gvbd"
Aug 05 17:21:34.500292 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499461    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7-device-dir\") pod \"aws-ebs-csi-driver-node-9ftv4\" (UID: \"0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4"
Aug 05 17:21:34.500292 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499485    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-multus-conf-dir\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.500292 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499507    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9ftv4\" (UID: \"0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4"
Aug 05 17:21:34.500292 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499538    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5bd15c5f-5f78-4cba-b190-2e22851079f8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xhnch\" (UID: \"5bd15c5f-5f78-4cba-b190-2e22851079f8\") " pod="openshift-multus/multus-additional-cni-plugins-xhnch"
Aug 05 17:21:34.500292 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499556    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-run-systemd\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.500292 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499570    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b382a0b8-435d-430a-a8a2-faa3d0b3bdc1-tmp-dir\") pod \"node-resolver-2f92r\" (UID: \"b382a0b8-435d-430a-a8a2-faa3d0b3bdc1\") " pod="openshift-dns/node-resolver-2f92r"
Aug 05 17:21:34.500292 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499587    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwfdc\" (UniqueName: \"kubernetes.io/projected/f8972708-830a-4ebc-b521-8a16ca1aee39-kube-api-access-kwfdc\") pod \"network-check-target-4jfwd\" (UID: \"f8972708-830a-4ebc-b521-8a16ca1aee39\") " pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:21:34.500292 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499613    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-dbus\") pod \"global-pull-secret-syncer-h8vbl\" (UID: \"bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f\") " pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:21:34.500292 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499638    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-host-run-netns\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.500292 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499653    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-node-log\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.500292 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499693    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-run-openvswitch\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.500292 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499736    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/610a012b-bd9a-4627-8976-b1f785945d61-serviceca\") pod \"node-ca-bpcrr\" (UID: \"610a012b-bd9a-4627-8976-b1f785945d61\") " pod="openshift-image-registry/node-ca-bpcrr"
Aug 05 17:21:34.500292 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.499776    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5bd15c5f-5f78-4cba-b190-2e22851079f8-os-release\") pod \"multus-additional-cni-plugins-xhnch\" (UID: \"5bd15c5f-5f78-4cba-b190-2e22851079f8\") " pod="openshift-multus/multus-additional-cni-plugins-xhnch"
Aug 05 17:21:34.502067 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.502051    2550 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 05 17:21:34.592008 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.591837    2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-04 17:16:33 +0000 UTC" deadline="2028-05-20 16:42:03.87192773 +0000 UTC"
Aug 05 17:21:34.592008 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.591892    2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15695h20m29.280039816s"
Aug 05 17:21:34.592008 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.591877    2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Aug 05 17:21:34.600084 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600055    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-host-run-multus-certs\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.600235 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600094    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5bd15c5f-5f78-4cba-b190-2e22851079f8-cni-binary-copy\") pod \"multus-additional-cni-plugins-xhnch\" (UID: \"5bd15c5f-5f78-4cba-b190-2e22851079f8\") " pod="openshift-multus/multus-additional-cni-plugins-xhnch"
Aug 05 17:21:34.600235 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600115    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-log-socket\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.600235 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600159    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-log-socket\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.600235 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600163    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-etc-systemd\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.600235 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600204    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.600235 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600218    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-etc-systemd\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.600235 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600232    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-env-overrides\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.600235 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600169    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-host-run-multus-certs\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.600647 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600255    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-system-cni-dir\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.600647 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600278    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-etc-kubernetes\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.600647 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600289    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.600647 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600341    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-system-cni-dir\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.600647 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600299    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-etc-modprobe-d\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.600647 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600378    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4smm\" (UniqueName: \"kubernetes.io/projected/ee639fbc-354e-4612-bf13-47c4e4df965f-kube-api-access-n4smm\") pod \"iptables-alerter-8gvbd\" (UID: \"ee639fbc-354e-4612-bf13-47c4e4df965f\") " pod="openshift-network-operator/iptables-alerter-8gvbd"
Aug 05 17:21:34.600647 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600387    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-etc-kubernetes\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.600647 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600406    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-etc-modprobe-d\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.600647 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600403    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/882b422e-055d-423f-b318-c8e66a901dd9-multus-daemon-config\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.600647 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600446    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdqpz\" (UniqueName: \"kubernetes.io/projected/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-kube-api-access-sdqpz\") pod \"network-metrics-daemon-rkln9\" (UID: \"0fca5e3d-e0e1-4b33-909c-46599a84cf4a\") " pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:21:34.601217 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600670    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-env-overrides\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.601217 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600673    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5bd15c5f-5f78-4cba-b190-2e22851079f8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xhnch\" (UID: \"5bd15c5f-5f78-4cba-b190-2e22851079f8\") " pod="openshift-multus/multus-additional-cni-plugins-xhnch"
Aug 05 17:21:34.601217 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600752    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-etc-sysctl-conf\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.601217 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600776    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/610a012b-bd9a-4627-8976-b1f785945d61-host\") pod \"node-ca-bpcrr\" (UID: \"610a012b-bd9a-4627-8976-b1f785945d61\") " pod="openshift-image-registry/node-ca-bpcrr"
Aug 05 17:21:34.601217 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600782    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5bd15c5f-5f78-4cba-b190-2e22851079f8-cni-binary-copy\") pod \"multus-additional-cni-plugins-xhnch\" (UID: \"5bd15c5f-5f78-4cba-b190-2e22851079f8\") " pod="openshift-multus/multus-additional-cni-plugins-xhnch"
Aug 05 17:21:34.601217 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600799    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-host-var-lib-cni-multus\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.601217 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600847    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-host-var-lib-cni-multus\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.601217 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600855    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/610a012b-bd9a-4627-8976-b1f785945d61-host\") pod \"node-ca-bpcrr\" (UID: \"610a012b-bd9a-4627-8976-b1f785945d61\") " pod="openshift-image-registry/node-ca-bpcrr"
Aug 05 17:21:34.601217 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600878    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpdmv\" (UniqueName: \"kubernetes.io/projected/5bd15c5f-5f78-4cba-b190-2e22851079f8-kube-api-access-dpdmv\") pod \"multus-additional-cni-plugins-xhnch\" (UID: \"5bd15c5f-5f78-4cba-b190-2e22851079f8\") " pod="openshift-multus/multus-additional-cni-plugins-xhnch"
Aug 05 17:21:34.601217 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600894    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/882b422e-055d-423f-b318-c8e66a901dd9-multus-daemon-config\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.601217 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600906    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-host-kubelet\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.601217 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600932    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-host-run-ovn-kubernetes\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.601217 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600908    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-etc-sysctl-conf\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.601217 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600955    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-host-cni-netd\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.601217 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601002    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-host-cni-netd\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.601217 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601008    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-host-run-ovn-kubernetes\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.601217 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.600947    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-host-kubelet\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.602098 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601031    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx2f2\" (UniqueName: \"kubernetes.io/projected/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-kube-api-access-hx2f2\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.602098 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601058    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hld45\" (UniqueName: \"kubernetes.io/projected/0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7-kube-api-access-hld45\") pod \"aws-ebs-csi-driver-node-9ftv4\" (UID: \"0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4"
Aug 05 17:21:34.602098 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601100    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-multus-socket-dir-parent\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.602098 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601104    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5bd15c5f-5f78-4cba-b190-2e22851079f8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xhnch\" (UID: \"5bd15c5f-5f78-4cba-b190-2e22851079f8\") " pod="openshift-multus/multus-additional-cni-plugins-xhnch"
Aug 05 17:21:34.602098 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601125    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwpmc\" (UniqueName: \"kubernetes.io/projected/882b422e-055d-423f-b318-c8e66a901dd9-kube-api-access-rwpmc\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.602098 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601148    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5bd15c5f-5f78-4cba-b190-2e22851079f8-system-cni-dir\") pod \"multus-additional-cni-plugins-xhnch\" (UID: \"5bd15c5f-5f78-4cba-b190-2e22851079f8\") " pod="openshift-multus/multus-additional-cni-plugins-xhnch"
Aug 05 17:21:34.602098 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601174    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-original-pull-secret\") pod \"global-pull-secret-syncer-h8vbl\" (UID: \"bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f\") " pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:21:34.602098 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601198    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7-registration-dir\") pod \"aws-ebs-csi-driver-node-9ftv4\" (UID: \"0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4"
Aug 05 17:21:34.602098 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601241    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-lib-modules\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.602098 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601265    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-var-lib-kubelet\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.602098 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601286    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-etc-sysconfig\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.602098 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601297    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-multus-socket-dir-parent\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.602098 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:34.601309    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:21:34.602098 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:34.601359    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-original-pull-secret podName:bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f nodeName:}" failed. No retries permitted until 2026-08-05 17:21:35.101343017 +0000 UTC m=+2.082920423 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-original-pull-secret") pod "global-pull-secret-syncer-h8vbl" (UID: "bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:21:34.602098 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601368    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-etc-sysconfig\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.602098 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601311    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-etc-kubernetes\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.602925 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601388    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5bd15c5f-5f78-4cba-b190-2e22851079f8-system-cni-dir\") pod \"multus-additional-cni-plugins-xhnch\" (UID: \"5bd15c5f-5f78-4cba-b190-2e22851079f8\") " pod="openshift-multus/multus-additional-cni-plugins-xhnch"
Aug 05 17:21:34.602925 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601413    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-etc-sysctl-d\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.602925 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601422    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7-registration-dir\") pod \"aws-ebs-csi-driver-node-9ftv4\" (UID: \"0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4"
Aug 05 17:21:34.602925 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601435    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-lib-modules\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.602925 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601445    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-etc-kubernetes\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.602925 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601438    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/749fcabe-941f-454a-b97e-49d077abd98b-tmp\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.602925 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601482    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb5jf\" (UniqueName: \"kubernetes.io/projected/749fcabe-941f-454a-b97e-49d077abd98b-kube-api-access-pb5jf\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.602925 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601497    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-var-lib-kubelet\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.602925 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601507    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-systemd-units\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.602925 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601527    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-etc-sysctl-d\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.602925 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601533    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b382a0b8-435d-430a-a8a2-faa3d0b3bdc1-hosts-file\") pod \"node-resolver-2f92r\" (UID: \"b382a0b8-435d-430a-a8a2-faa3d0b3bdc1\") " pod="openshift-dns/node-resolver-2f92r"
Aug 05 17:21:34.602925 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601559    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bf34f30e-7b6a-49fc-a27f-e1955f248d7c-agent-certs\") pod \"konnectivity-agent-g55gw\" (UID: \"bf34f30e-7b6a-49fc-a27f-e1955f248d7c\") " pod="kube-system/konnectivity-agent-g55gw"
Aug 05 17:21:34.602925 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601567    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-systemd-units\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.602925 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601582    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlpgq\" (UniqueName: \"kubernetes.io/projected/610a012b-bd9a-4627-8976-b1f785945d61-kube-api-access-mlpgq\") pod \"node-ca-bpcrr\" (UID: \"610a012b-bd9a-4627-8976-b1f785945d61\") " pod="openshift-image-registry/node-ca-bpcrr"
Aug 05 17:21:34.602925 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601608    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs\") pod \"network-metrics-daemon-rkln9\" (UID: \"0fca5e3d-e0e1-4b33-909c-46599a84cf4a\") " pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:21:34.602925 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601610    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b382a0b8-435d-430a-a8a2-faa3d0b3bdc1-hosts-file\") pod \"node-resolver-2f92r\" (UID: \"b382a0b8-435d-430a-a8a2-faa3d0b3bdc1\") " pod="openshift-dns/node-resolver-2f92r"
Aug 05 17:21:34.602925 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:34.601688    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:21:34.602925 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601757    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5bd15c5f-5f78-4cba-b190-2e22851079f8-cnibin\") pod \"multus-additional-cni-plugins-xhnch\" (UID: \"5bd15c5f-5f78-4cba-b190-2e22851079f8\") " pod="openshift-multus/multus-additional-cni-plugins-xhnch"
Aug 05 17:21:34.603830 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601809    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-sys\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.603830 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601811    2550 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Aug 05 17:21:34.603830 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:34.601837    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs podName:0fca5e3d-e0e1-4b33-909c-46599a84cf4a nodeName:}" failed. No retries permitted until 2026-08-05 17:21:35.101819565 +0000 UTC m=+2.083396999 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs") pod "network-metrics-daemon-rkln9" (UID: "0fca5e3d-e0e1-4b33-909c-46599a84cf4a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:21:34.603830 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601852    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5bd15c5f-5f78-4cba-b190-2e22851079f8-cnibin\") pod \"multus-additional-cni-plugins-xhnch\" (UID: \"5bd15c5f-5f78-4cba-b190-2e22851079f8\") " pod="openshift-multus/multus-additional-cni-plugins-xhnch"
Aug 05 17:21:34.603830 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601859    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-host-slash\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.603830 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601876    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-sys\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.603830 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601889    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7-etc-selinux\") pod \"aws-ebs-csi-driver-node-9ftv4\" (UID: \"0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4"
Aug 05 17:21:34.603830 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601912    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-cnibin\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.603830 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601930    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-host-slash\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.603830 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601939    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-host\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.603830 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601963    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-run\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.603830 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.601989    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-ovnkube-config\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.603830 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.602018    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-ovn-node-metrics-cert\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.603830 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.602019    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7-etc-selinux\") pod \"aws-ebs-csi-driver-node-9ftv4\" (UID: \"0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4"
Aug 05 17:21:34.603830 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.602054    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/882b422e-055d-423f-b318-c8e66a901dd9-cni-binary-copy\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.603830 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.602109    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-run\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.603830 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.602155    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/749fcabe-941f-454a-b97e-49d077abd98b-host\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.604670 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.602197    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-cnibin\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.604670 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.602233    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-host-var-lib-cni-bin\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.604670 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.602271    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5bd15c5f-5f78-4cba-b190-2e22851079f8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xhnch\" (UID: \"5bd15c5f-5f78-4cba-b190-2e22851079f8\") " pod="openshift-multus/multus-additional-cni-plugins-xhnch"
Aug 05 17:21:34.604670 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.602316    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-host-var-lib-cni-bin\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.604670 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.602577    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/882b422e-055d-423f-b318-c8e66a901dd9-cni-binary-copy\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.604670 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.602632    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-ovnkube-config\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.604670 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.602714    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-var-lib-openvswitch\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.604670 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.602747    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-run-ovn\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.604670 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.602792    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-host-cni-bin\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.604670 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.602802    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5bd15c5f-5f78-4cba-b190-2e22851079f8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xhnch\" (UID: \"5bd15c5f-5f78-4cba-b190-2e22851079f8\") " pod="openshift-multus/multus-additional-cni-plugins-xhnch"
Aug 05 17:21:34.604670 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.602837    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw8dm\" (UniqueName: \"kubernetes.io/projected/b382a0b8-435d-430a-a8a2-faa3d0b3bdc1-kube-api-access-mw8dm\") pod \"node-resolver-2f92r\" (UID: \"b382a0b8-435d-430a-a8a2-faa3d0b3bdc1\") " pod="openshift-dns/node-resolver-2f92r"
Aug 05 17:21:34.604670 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.602864    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-var-lib-openvswitch\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.604670 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.602871    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ee639fbc-354e-4612-bf13-47c4e4df965f-iptables-alerter-script\") pod \"iptables-alerter-8gvbd\" (UID: \"ee639fbc-354e-4612-bf13-47c4e4df965f\") " pod="openshift-network-operator/iptables-alerter-8gvbd"
Aug 05 17:21:34.604670 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.602911    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-host-cni-bin\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.604670 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.602953    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7-device-dir\") pod \"aws-ebs-csi-driver-node-9ftv4\" (UID: \"0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4"
Aug 05 17:21:34.604670 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.602997    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-multus-conf-dir\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.604670 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603000    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7-device-dir\") pod \"aws-ebs-csi-driver-node-9ftv4\" (UID: \"0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4"
Aug 05 17:21:34.605581 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603029    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9ftv4\" (UID: \"0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4"
Aug 05 17:21:34.605581 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603053    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5bd15c5f-5f78-4cba-b190-2e22851079f8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xhnch\" (UID: \"5bd15c5f-5f78-4cba-b190-2e22851079f8\") " pod="openshift-multus/multus-additional-cni-plugins-xhnch"
Aug 05 17:21:34.605581 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603084    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-run-systemd\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.605581 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603106    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b382a0b8-435d-430a-a8a2-faa3d0b3bdc1-tmp-dir\") pod \"node-resolver-2f92r\" (UID: \"b382a0b8-435d-430a-a8a2-faa3d0b3bdc1\") " pod="openshift-dns/node-resolver-2f92r"
Aug 05 17:21:34.605581 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603240    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5bd15c5f-5f78-4cba-b190-2e22851079f8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xhnch\" (UID: \"5bd15c5f-5f78-4cba-b190-2e22851079f8\") " pod="openshift-multus/multus-additional-cni-plugins-xhnch"
Aug 05 17:21:34.605581 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603260    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-run-ovn\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.605581 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603278    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-multus-conf-dir\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.605581 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603302    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9ftv4\" (UID: \"0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4"
Aug 05 17:21:34.605581 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603311    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-run-systemd\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.605581 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603352    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwfdc\" (UniqueName: \"kubernetes.io/projected/f8972708-830a-4ebc-b521-8a16ca1aee39-kube-api-access-kwfdc\") pod \"network-check-target-4jfwd\" (UID: \"f8972708-830a-4ebc-b521-8a16ca1aee39\") " pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:21:34.605581 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603391    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-dbus\") pod \"global-pull-secret-syncer-h8vbl\" (UID: \"bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f\") " pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:21:34.605581 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603413    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-host-run-netns\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.605581 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603447    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-node-log\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.605581 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603475    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-run-openvswitch\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.605581 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603510    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/610a012b-bd9a-4627-8976-b1f785945d61-serviceca\") pod \"node-ca-bpcrr\" (UID: \"610a012b-bd9a-4627-8976-b1f785945d61\") " pod="openshift-image-registry/node-ca-bpcrr"
Aug 05 17:21:34.605581 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603535    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5bd15c5f-5f78-4cba-b190-2e22851079f8-os-release\") pod \"multus-additional-cni-plugins-xhnch\" (UID: \"5bd15c5f-5f78-4cba-b190-2e22851079f8\") " pod="openshift-multus/multus-additional-cni-plugins-xhnch"
Aug 05 17:21:34.605581 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603598    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-node-log\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.605581 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603598    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b382a0b8-435d-430a-a8a2-faa3d0b3bdc1-tmp-dir\") pod \"node-resolver-2f92r\" (UID: \"b382a0b8-435d-430a-a8a2-faa3d0b3bdc1\") " pod="openshift-dns/node-resolver-2f92r"
Aug 05 17:21:34.606466 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603627    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-host-run-netns\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.606466 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603689    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ee639fbc-354e-4612-bf13-47c4e4df965f-iptables-alerter-script\") pod \"iptables-alerter-8gvbd\" (UID: \"ee639fbc-354e-4612-bf13-47c4e4df965f\") " pod="openshift-network-operator/iptables-alerter-8gvbd"
Aug 05 17:21:34.606466 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603673    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5bd15c5f-5f78-4cba-b190-2e22851079f8-os-release\") pod \"multus-additional-cni-plugins-xhnch\" (UID: \"5bd15c5f-5f78-4cba-b190-2e22851079f8\") " pod="openshift-multus/multus-additional-cni-plugins-xhnch"
Aug 05 17:21:34.606466 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603797    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bf34f30e-7b6a-49fc-a27f-e1955f248d7c-konnectivity-ca\") pod \"konnectivity-agent-g55gw\" (UID: \"bf34f30e-7b6a-49fc-a27f-e1955f248d7c\") " pod="kube-system/konnectivity-agent-g55gw"
Aug 05 17:21:34.606466 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603839    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-kubelet-config\") pod \"global-pull-secret-syncer-h8vbl\" (UID: \"bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f\") " pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:21:34.606466 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603857    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-dbus\") pod \"global-pull-secret-syncer-h8vbl\" (UID: \"bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f\") " pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:21:34.606466 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603896    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-multus-cni-dir\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.606466 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603944    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee639fbc-354e-4612-bf13-47c4e4df965f-host-slash\") pod \"iptables-alerter-8gvbd\" (UID: \"ee639fbc-354e-4612-bf13-47c4e4df965f\") " pod="openshift-network-operator/iptables-alerter-8gvbd"
Aug 05 17:21:34.606466 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603955    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-run-openvswitch\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.606466 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603966    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7-socket-dir\") pod \"aws-ebs-csi-driver-node-9ftv4\" (UID: \"0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4"
Aug 05 17:21:34.606466 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.603991    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7-sys-fs\") pod \"aws-ebs-csi-driver-node-9ftv4\" (UID: \"0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4"
Aug 05 17:21:34.606466 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.604001    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-kubelet-config\") pod \"global-pull-secret-syncer-h8vbl\" (UID: \"bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f\") " pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:21:34.606466 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.604014    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-os-release\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.606466 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.604038    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-host-run-k8s-cni-cncf-io\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.606466 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.604047    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee639fbc-354e-4612-bf13-47c4e4df965f-host-slash\") pod \"iptables-alerter-8gvbd\" (UID: \"ee639fbc-354e-4612-bf13-47c4e4df965f\") " pod="openshift-network-operator/iptables-alerter-8gvbd"
Aug 05 17:21:34.606466 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.604069    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/610a012b-bd9a-4627-8976-b1f785945d61-serviceca\") pod \"node-ca-bpcrr\" (UID: \"610a012b-bd9a-4627-8976-b1f785945d61\") " pod="openshift-image-registry/node-ca-bpcrr"
Aug 05 17:21:34.606466 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.604082    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-hostroot\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.606466 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.604106    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-multus-cni-dir\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.607420 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.604122    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/749fcabe-941f-454a-b97e-49d077abd98b-etc-tuned\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.607420 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.604140    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-os-release\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.607420 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.604196    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7-sys-fs\") pod \"aws-ebs-csi-driver-node-9ftv4\" (UID: \"0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4"
Aug 05 17:21:34.607420 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.604215    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7-socket-dir\") pod \"aws-ebs-csi-driver-node-9ftv4\" (UID: \"0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4"
Aug 05 17:21:34.607420 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.604264    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-host-run-netns\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.607420 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.604294    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-hostroot\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.607420 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.604314    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-etc-openvswitch\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.607420 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.604341    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-ovnkube-script-lib\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.607420 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.604364    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-host-var-lib-kubelet\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.607420 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.604424    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-host-var-lib-kubelet\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.607420 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.604459    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-host-run-netns\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.607420 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.604466    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bf34f30e-7b6a-49fc-a27f-e1955f248d7c-konnectivity-ca\") pod \"konnectivity-agent-g55gw\" (UID: \"bf34f30e-7b6a-49fc-a27f-e1955f248d7c\") " pod="kube-system/konnectivity-agent-g55gw"
Aug 05 17:21:34.607420 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.604497    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-etc-openvswitch\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.607420 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.604341    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/882b422e-055d-423f-b318-c8e66a901dd9-host-run-k8s-cni-cncf-io\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.607420 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.604622    2550 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 05 17:21:34.607420 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.605085    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-ovnkube-script-lib\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.607420 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.605850    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/749fcabe-941f-454a-b97e-49d077abd98b-tmp\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.607420 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.605915    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-ovn-node-metrics-cert\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.608285 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.606236    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/749fcabe-941f-454a-b97e-49d077abd98b-etc-tuned\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.608285 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.606437    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bf34f30e-7b6a-49fc-a27f-e1955f248d7c-agent-certs\") pod \"konnectivity-agent-g55gw\" (UID: \"bf34f30e-7b6a-49fc-a27f-e1955f248d7c\") " pod="kube-system/konnectivity-agent-g55gw"
Aug 05 17:21:34.610365 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.610337    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hld45\" (UniqueName: \"kubernetes.io/projected/0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7-kube-api-access-hld45\") pod \"aws-ebs-csi-driver-node-9ftv4\" (UID: \"0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4"
Aug 05 17:21:34.610471 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.610425    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4smm\" (UniqueName: \"kubernetes.io/projected/ee639fbc-354e-4612-bf13-47c4e4df965f-kube-api-access-n4smm\") pod \"iptables-alerter-8gvbd\" (UID: \"ee639fbc-354e-4612-bf13-47c4e4df965f\") " pod="openshift-network-operator/iptables-alerter-8gvbd"
Aug 05 17:21:34.610539 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.610497    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdqpz\" (UniqueName: \"kubernetes.io/projected/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-kube-api-access-sdqpz\") pod \"network-metrics-daemon-rkln9\" (UID: \"0fca5e3d-e0e1-4b33-909c-46599a84cf4a\") " pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:21:34.611072 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.611040    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx2f2\" (UniqueName: \"kubernetes.io/projected/1e69a0d5-f55f-4ff7-bd66-82fcc68f8674-kube-api-access-hx2f2\") pod \"ovnkube-node-thnnz\" (UID: \"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674\") " pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.611834 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.611813    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpdmv\" (UniqueName: \"kubernetes.io/projected/5bd15c5f-5f78-4cba-b190-2e22851079f8-kube-api-access-dpdmv\") pod \"multus-additional-cni-plugins-xhnch\" (UID: \"5bd15c5f-5f78-4cba-b190-2e22851079f8\") " pod="openshift-multus/multus-additional-cni-plugins-xhnch"
Aug 05 17:21:34.611914 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:34.611845    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:21:34.611914 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:34.611865    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:21:34.611914 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:34.611887    2550 projected.go:196] Error preparing data for projected volume kube-api-access-kwfdc for pod openshift-network-diagnostics/network-check-target-4jfwd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:21:34.612099 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:34.612057    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8972708-830a-4ebc-b521-8a16ca1aee39-kube-api-access-kwfdc podName:f8972708-830a-4ebc-b521-8a16ca1aee39 nodeName:}" failed. No retries permitted until 2026-08-05 17:21:35.112036023 +0000 UTC m=+2.093613447 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kwfdc" (UniqueName: "kubernetes.io/projected/f8972708-830a-4ebc-b521-8a16ca1aee39-kube-api-access-kwfdc") pod "network-check-target-4jfwd" (UID: "f8972708-830a-4ebc-b521-8a16ca1aee39") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:21:34.612228 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.612206    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw8dm\" (UniqueName: \"kubernetes.io/projected/b382a0b8-435d-430a-a8a2-faa3d0b3bdc1-kube-api-access-mw8dm\") pod \"node-resolver-2f92r\" (UID: \"b382a0b8-435d-430a-a8a2-faa3d0b3bdc1\") " pod="openshift-dns/node-resolver-2f92r"
Aug 05 17:21:34.612757 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.612735    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwpmc\" (UniqueName: \"kubernetes.io/projected/882b422e-055d-423f-b318-c8e66a901dd9-kube-api-access-rwpmc\") pod \"multus-fbk5k\" (UID: \"882b422e-055d-423f-b318-c8e66a901dd9\") " pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.612854 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.612841    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pb5jf\" (UniqueName: \"kubernetes.io/projected/749fcabe-941f-454a-b97e-49d077abd98b-kube-api-access-pb5jf\") pod \"tuned-srf7f\" (UID: \"749fcabe-941f-454a-b97e-49d077abd98b\") " pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.613611 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.613596    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlpgq\" (UniqueName: \"kubernetes.io/projected/610a012b-bd9a-4627-8976-b1f785945d61-kube-api-access-mlpgq\") pod \"node-ca-bpcrr\" (UID: \"610a012b-bd9a-4627-8976-b1f785945d61\") " pod="openshift-image-registry/node-ca-bpcrr"
Aug 05 17:21:34.630035 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.630006    2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hj8cr"
Aug 05 17:21:34.638162 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.638134    2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hj8cr"
Aug 05 17:21:34.706767 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:34.706732    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc95b41ba2403920dcca145af0d4b9bb1.slice/crio-73703aa53c2aeddb67e273fcd411715333bac3a7d58a40c9e5971ab18684c002 WatchSource:0}: Error finding container 73703aa53c2aeddb67e273fcd411715333bac3a7d58a40c9e5971ab18684c002: Status 404 returned error can't find the container with id 73703aa53c2aeddb67e273fcd411715333bac3a7d58a40c9e5971ab18684c002
Aug 05 17:21:34.711808 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.711792    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 17:21:34.758064 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.758017    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4"
Aug 05 17:21:34.767855 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:34.767833    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0805ff4f_43b2_46d4_b3bb_fcfbeb5b36b7.slice/crio-dc8091e59c341a91995f2cbd8d1f8bb669211cf34e1e145628d9cd80db2a9ebe WatchSource:0}: Error finding container dc8091e59c341a91995f2cbd8d1f8bb669211cf34e1e145628d9cd80db2a9ebe: Status 404 returned error can't find the container with id dc8091e59c341a91995f2cbd8d1f8bb669211cf34e1e145628d9cd80db2a9ebe
Aug 05 17:21:34.773006 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.772981    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fbk5k"
Aug 05 17:21:34.780842 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:34.780820    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod882b422e_055d_423f_b318_c8e66a901dd9.slice/crio-64a06fff57614075ba830a6cbd0edd9d97b4f8d65535fd06187bd51f47960106 WatchSource:0}: Error finding container 64a06fff57614075ba830a6cbd0edd9d97b4f8d65535fd06187bd51f47960106: Status 404 returned error can't find the container with id 64a06fff57614075ba830a6cbd0edd9d97b4f8d65535fd06187bd51f47960106
Aug 05 17:21:34.798399 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.798376    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xhnch"
Aug 05 17:21:34.802172 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.802156    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-srf7f"
Aug 05 17:21:34.807772 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:34.807745    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bd15c5f_5f78_4cba_b190_2e22851079f8.slice/crio-70752665d3f2fa1c303430ee058a315f345f4ddc39bee9bf8a23001a468e78ac WatchSource:0}: Error finding container 70752665d3f2fa1c303430ee058a315f345f4ddc39bee9bf8a23001a468e78ac: Status 404 returned error can't find the container with id 70752665d3f2fa1c303430ee058a315f345f4ddc39bee9bf8a23001a468e78ac
Aug 05 17:21:34.810290 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:34.810271    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod749fcabe_941f_454a_b97e_49d077abd98b.slice/crio-30c0c0315e3e92a9f67ab5535a790646e3cd8bbfba179dde03ffa9f88480bcbc WatchSource:0}: Error finding container 30c0c0315e3e92a9f67ab5535a790646e3cd8bbfba179dde03ffa9f88480bcbc: Status 404 returned error can't find the container with id 30c0c0315e3e92a9f67ab5535a790646e3cd8bbfba179dde03ffa9f88480bcbc
Aug 05 17:21:34.833870 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:34.833841    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf6ae06b6895f6ad9d2e2bf995b0fd40.slice/crio-ee9a5bc5b0953fdad79f03bb4d178a755512282d471442f7d37229d299b4835d WatchSource:0}: Error finding container ee9a5bc5b0953fdad79f03bb4d178a755512282d471442f7d37229d299b4835d: Status 404 returned error can't find the container with id ee9a5bc5b0953fdad79f03bb4d178a755512282d471442f7d37229d299b4835d
Aug 05 17:21:34.842642 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.842620    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g55gw"
Aug 05 17:21:34.850669 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:34.850643    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf34f30e_7b6a_49fc_a27f_e1955f248d7c.slice/crio-05625d9f7d8462370df7ef6ae2c3c66169243a9b71cdfd5103407dc8932b5577 WatchSource:0}: Error finding container 05625d9f7d8462370df7ef6ae2c3c66169243a9b71cdfd5103407dc8932b5577: Status 404 returned error can't find the container with id 05625d9f7d8462370df7ef6ae2c3c66169243a9b71cdfd5103407dc8932b5577
Aug 05 17:21:34.856847 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.856824    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bpcrr"
Aug 05 17:21:34.864867 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:34.864842    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod610a012b_bd9a_4627_8976_b1f785945d61.slice/crio-92cff7ca1fb8b2e684457ab64315f4a7a4d66189ccc9e181c6d8a1c8090efc42 WatchSource:0}: Error finding container 92cff7ca1fb8b2e684457ab64315f4a7a4d66189ccc9e181c6d8a1c8090efc42: Status 404 returned error can't find the container with id 92cff7ca1fb8b2e684457ab64315f4a7a4d66189ccc9e181c6d8a1c8090efc42
Aug 05 17:21:34.876058 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.876034    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:34.884040 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:34.884014    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e69a0d5_f55f_4ff7_bd66_82fcc68f8674.slice/crio-2c03128ac88a715d4b9b050bf94650a32f8db16053e8d63815753ac2eb9f1f20 WatchSource:0}: Error finding container 2c03128ac88a715d4b9b050bf94650a32f8db16053e8d63815753ac2eb9f1f20: Status 404 returned error can't find the container with id 2c03128ac88a715d4b9b050bf94650a32f8db16053e8d63815753ac2eb9f1f20
Aug 05 17:21:34.890049 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.890027    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2f92r"
Aug 05 17:21:34.898838 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:34.898821    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8gvbd"
Aug 05 17:21:34.899140 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:34.899119    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb382a0b8_435d_430a_a8a2_faa3d0b3bdc1.slice/crio-01dccf8780778c7c3b5ca52884df297691e4eca795e639f5fb2e637b5cd9030b WatchSource:0}: Error finding container 01dccf8780778c7c3b5ca52884df297691e4eca795e639f5fb2e637b5cd9030b: Status 404 returned error can't find the container with id 01dccf8780778c7c3b5ca52884df297691e4eca795e639f5fb2e637b5cd9030b
Aug 05 17:21:34.907911 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:21:34.907876    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee639fbc_354e_4612_bf13_47c4e4df965f.slice/crio-504d5fa00a1e89c373b417714bf9d8fc80beccbef0eaa67a4bc71b7d3c5d19f3 WatchSource:0}: Error finding container 504d5fa00a1e89c373b417714bf9d8fc80beccbef0eaa67a4bc71b7d3c5d19f3: Status 404 returned error can't find the container with id 504d5fa00a1e89c373b417714bf9d8fc80beccbef0eaa67a4bc71b7d3c5d19f3
Aug 05 17:21:34.910619 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:34.910597    2550 kuberuntime_manager.go:1459] "Unhandled Error" err="container iptables-alerter start failed in pod iptables-alerter-8gvbd_openshift-network-operator(ee639fbc-354e-4612-bf13-47c4e4df965f): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Aug 05 17:21:34.910692 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:34.910636    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"iptables-alerter\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-network-operator/iptables-alerter-8gvbd" podUID="ee639fbc-354e-4612-bf13-47c4e4df965f"
Aug 05 17:21:35.109419 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:35.109329    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-original-pull-secret\") pod \"global-pull-secret-syncer-h8vbl\" (UID: \"bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f\") " pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:21:35.109419 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:35.109372    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs\") pod \"network-metrics-daemon-rkln9\" (UID: \"0fca5e3d-e0e1-4b33-909c-46599a84cf4a\") " pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:21:35.109621 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:35.109470    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:21:35.109621 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:35.109494    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:21:35.109621 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:35.109528    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-original-pull-secret podName:bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f nodeName:}" failed. No retries permitted until 2026-08-05 17:21:36.109513193 +0000 UTC m=+3.091090601 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-original-pull-secret") pod "global-pull-secret-syncer-h8vbl" (UID: "bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:21:35.109621 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:35.109543    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs podName:0fca5e3d-e0e1-4b33-909c-46599a84cf4a nodeName:}" failed. No retries permitted until 2026-08-05 17:21:36.109535622 +0000 UTC m=+3.091113029 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs") pod "network-metrics-daemon-rkln9" (UID: "0fca5e3d-e0e1-4b33-909c-46599a84cf4a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:21:35.210105 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:35.210075    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwfdc\" (UniqueName: \"kubernetes.io/projected/f8972708-830a-4ebc-b521-8a16ca1aee39-kube-api-access-kwfdc\") pod \"network-check-target-4jfwd\" (UID: \"f8972708-830a-4ebc-b521-8a16ca1aee39\") " pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:21:35.210281 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:35.210221    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:21:35.210281 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:35.210248    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:21:35.210281 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:35.210262    2550 projected.go:196] Error preparing data for projected volume kube-api-access-kwfdc for pod openshift-network-diagnostics/network-check-target-4jfwd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:21:35.210459 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:35.210325    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8972708-830a-4ebc-b521-8a16ca1aee39-kube-api-access-kwfdc podName:f8972708-830a-4ebc-b521-8a16ca1aee39 nodeName:}" failed. No retries permitted until 2026-08-05 17:21:36.210306462 +0000 UTC m=+3.191883870 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwfdc" (UniqueName: "kubernetes.io/projected/f8972708-830a-4ebc-b521-8a16ca1aee39-kube-api-access-kwfdc") pod "network-check-target-4jfwd" (UID: "f8972708-830a-4ebc-b521-8a16ca1aee39") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:21:35.391879 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:35.391774    2550 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 05 17:21:35.550777 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:35.550529    2550 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 05 17:21:35.640940 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:35.640842    2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-04 17:16:34 +0000 UTC" deadline="2028-04-08 13:18:50.791759246 +0000 UTC"
Aug 05 17:21:35.640940 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:35.640880    2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14683h57m15.150885085s"
Aug 05 17:21:35.698288 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:35.697113    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:21:35.698288 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:35.697289    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4jfwd" podUID="f8972708-830a-4ebc-b521-8a16ca1aee39"
Aug 05 17:21:35.698288 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:35.697718    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:21:35.698288 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:35.697869    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8vbl" podUID="bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f"
Aug 05 17:21:35.698288 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:35.698014    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:21:35.698288 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:35.698098    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkln9" podUID="0fca5e3d-e0e1-4b33-909c-46599a84cf4a"
Aug 05 17:21:35.736724 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:35.736651    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fbk5k" event={"ID":"882b422e-055d-423f-b318-c8e66a901dd9","Type":"ContainerStarted","Data":"64a06fff57614075ba830a6cbd0edd9d97b4f8d65535fd06187bd51f47960106"}
Aug 05 17:21:35.745210 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:35.745173    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8gvbd" event={"ID":"ee639fbc-354e-4612-bf13-47c4e4df965f","Type":"ContainerStarted","Data":"504d5fa00a1e89c373b417714bf9d8fc80beccbef0eaa67a4bc71b7d3c5d19f3"}
Aug 05 17:21:35.751158 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:35.751120    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"iptables-alerter\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:74b8d9cff26ab06c9d7bbeb115f3d2d4319dafb5e9f367b231d40f610b152391\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-network-operator/iptables-alerter-8gvbd" podUID="ee639fbc-354e-4612-bf13-47c4e4df965f"
Aug 05 17:21:35.761074 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:35.761009    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2f92r" event={"ID":"b382a0b8-435d-430a-a8a2-faa3d0b3bdc1","Type":"ContainerStarted","Data":"01dccf8780778c7c3b5ca52884df297691e4eca795e639f5fb2e637b5cd9030b"}
Aug 05 17:21:35.763881 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:35.763847    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-thnnz" event={"ID":"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674","Type":"ContainerStarted","Data":"2c03128ac88a715d4b9b050bf94650a32f8db16053e8d63815753ac2eb9f1f20"}
Aug 05 17:21:35.773500 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:35.773424    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-46.ec2.internal" event={"ID":"af6ae06b6895f6ad9d2e2bf995b0fd40","Type":"ContainerStarted","Data":"ee9a5bc5b0953fdad79f03bb4d178a755512282d471442f7d37229d299b4835d"}
Aug 05 17:21:35.789068 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:35.788972    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-srf7f" event={"ID":"749fcabe-941f-454a-b97e-49d077abd98b","Type":"ContainerStarted","Data":"30c0c0315e3e92a9f67ab5535a790646e3cd8bbfba179dde03ffa9f88480bcbc"}
Aug 05 17:21:35.805404 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:35.805314    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xhnch" event={"ID":"5bd15c5f-5f78-4cba-b190-2e22851079f8","Type":"ContainerStarted","Data":"70752665d3f2fa1c303430ee058a315f345f4ddc39bee9bf8a23001a468e78ac"}
Aug 05 17:21:35.823536 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:35.823440    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4" event={"ID":"0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7","Type":"ContainerStarted","Data":"dc8091e59c341a91995f2cbd8d1f8bb669211cf34e1e145628d9cd80db2a9ebe"}
Aug 05 17:21:35.833711 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:35.833629    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.ec2.internal" event={"ID":"c95b41ba2403920dcca145af0d4b9bb1","Type":"ContainerStarted","Data":"73703aa53c2aeddb67e273fcd411715333bac3a7d58a40c9e5971ab18684c002"}
Aug 05 17:21:35.860408 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:35.860367    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bpcrr" event={"ID":"610a012b-bd9a-4627-8976-b1f785945d61","Type":"ContainerStarted","Data":"92cff7ca1fb8b2e684457ab64315f4a7a4d66189ccc9e181c6d8a1c8090efc42"}
Aug 05 17:21:35.871802 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:35.871764    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g55gw" event={"ID":"bf34f30e-7b6a-49fc-a27f-e1955f248d7c","Type":"ContainerStarted","Data":"05625d9f7d8462370df7ef6ae2c3c66169243a9b71cdfd5103407dc8932b5577"}
Aug 05 17:21:36.119360 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:36.118479    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-original-pull-secret\") pod \"global-pull-secret-syncer-h8vbl\" (UID: \"bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f\") " pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:21:36.119360 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:36.118536    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs\") pod \"network-metrics-daemon-rkln9\" (UID: \"0fca5e3d-e0e1-4b33-909c-46599a84cf4a\") " pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:21:36.119360 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:36.118660    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:21:36.119360 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:36.118748    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs podName:0fca5e3d-e0e1-4b33-909c-46599a84cf4a nodeName:}" failed. No retries permitted until 2026-08-05 17:21:38.118726956 +0000 UTC m=+5.100304367 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs") pod "network-metrics-daemon-rkln9" (UID: "0fca5e3d-e0e1-4b33-909c-46599a84cf4a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:21:36.119360 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:36.119174    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:21:36.119360 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:36.119227    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-original-pull-secret podName:bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f nodeName:}" failed. No retries permitted until 2026-08-05 17:21:38.119209929 +0000 UTC m=+5.100787360 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-original-pull-secret") pod "global-pull-secret-syncer-h8vbl" (UID: "bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:21:36.219728 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:36.219661    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwfdc\" (UniqueName: \"kubernetes.io/projected/f8972708-830a-4ebc-b521-8a16ca1aee39-kube-api-access-kwfdc\") pod \"network-check-target-4jfwd\" (UID: \"f8972708-830a-4ebc-b521-8a16ca1aee39\") " pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:21:36.219894 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:36.219855    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:21:36.219894 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:36.219873    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:21:36.219894 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:36.219888    2550 projected.go:196] Error preparing data for projected volume kube-api-access-kwfdc for pod openshift-network-diagnostics/network-check-target-4jfwd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:21:36.220063 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:36.219945    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8972708-830a-4ebc-b521-8a16ca1aee39-kube-api-access-kwfdc podName:f8972708-830a-4ebc-b521-8a16ca1aee39 nodeName:}" failed. No retries permitted until 2026-08-05 17:21:38.21992708 +0000 UTC m=+5.201504504 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwfdc" (UniqueName: "kubernetes.io/projected/f8972708-830a-4ebc-b521-8a16ca1aee39-kube-api-access-kwfdc") pod "network-check-target-4jfwd" (UID: "f8972708-830a-4ebc-b521-8a16ca1aee39") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:21:36.641819 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:36.641771    2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-04 17:16:34 +0000 UTC" deadline="2028-05-12 14:08:01.198875572 +0000 UTC"
Aug 05 17:21:36.641819 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:36.641816    2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15500h46m24.557064592s"
Aug 05 17:21:37.697755 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:37.696839    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:21:37.697755 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:37.696964    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4jfwd" podUID="f8972708-830a-4ebc-b521-8a16ca1aee39"
Aug 05 17:21:37.697755 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:37.697371    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:21:37.697755 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:37.697461    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8vbl" podUID="bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f"
Aug 05 17:21:37.697755 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:37.697516    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:21:37.697755 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:37.697595    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkln9" podUID="0fca5e3d-e0e1-4b33-909c-46599a84cf4a"
Aug 05 17:21:38.136395 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:38.135559    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-original-pull-secret\") pod \"global-pull-secret-syncer-h8vbl\" (UID: \"bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f\") " pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:21:38.136395 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:38.135621    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs\") pod \"network-metrics-daemon-rkln9\" (UID: \"0fca5e3d-e0e1-4b33-909c-46599a84cf4a\") " pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:21:38.136395 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:38.135798    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:21:38.136395 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:38.135862    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs podName:0fca5e3d-e0e1-4b33-909c-46599a84cf4a nodeName:}" failed. No retries permitted until 2026-08-05 17:21:42.135844379 +0000 UTC m=+9.117421788 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs") pod "network-metrics-daemon-rkln9" (UID: "0fca5e3d-e0e1-4b33-909c-46599a84cf4a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:21:38.136395 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:38.136308    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:21:38.136395 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:38.136360    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-original-pull-secret podName:bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f nodeName:}" failed. No retries permitted until 2026-08-05 17:21:42.136344405 +0000 UTC m=+9.117921828 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-original-pull-secret") pod "global-pull-secret-syncer-h8vbl" (UID: "bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:21:38.237106 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:38.237068    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwfdc\" (UniqueName: \"kubernetes.io/projected/f8972708-830a-4ebc-b521-8a16ca1aee39-kube-api-access-kwfdc\") pod \"network-check-target-4jfwd\" (UID: \"f8972708-830a-4ebc-b521-8a16ca1aee39\") " pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:21:38.237305 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:38.237245    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:21:38.237305 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:38.237272    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:21:38.237305 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:38.237286    2550 projected.go:196] Error preparing data for projected volume kube-api-access-kwfdc for pod openshift-network-diagnostics/network-check-target-4jfwd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:21:38.237475 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:38.237348    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8972708-830a-4ebc-b521-8a16ca1aee39-kube-api-access-kwfdc podName:f8972708-830a-4ebc-b521-8a16ca1aee39 nodeName:}" failed. No retries permitted until 2026-08-05 17:21:42.237330665 +0000 UTC m=+9.218908075 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwfdc" (UniqueName: "kubernetes.io/projected/f8972708-830a-4ebc-b521-8a16ca1aee39-kube-api-access-kwfdc") pod "network-check-target-4jfwd" (UID: "f8972708-830a-4ebc-b521-8a16ca1aee39") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:21:39.695226 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:39.694878    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:21:39.695226 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:39.695008    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4jfwd" podUID="f8972708-830a-4ebc-b521-8a16ca1aee39"
Aug 05 17:21:39.695226 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:39.695107    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:21:39.695226 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:39.695217    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8vbl" podUID="bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f"
Aug 05 17:21:39.695874 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:39.695248    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:21:39.695874 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:39.695342    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkln9" podUID="0fca5e3d-e0e1-4b33-909c-46599a84cf4a"
Aug 05 17:21:41.695642 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:41.694802    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:21:41.695642 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:41.694926    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4jfwd" podUID="f8972708-830a-4ebc-b521-8a16ca1aee39"
Aug 05 17:21:41.695642 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:41.695349    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:21:41.695642 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:41.695444    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8vbl" podUID="bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f"
Aug 05 17:21:41.695642 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:41.695502    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:21:41.695642 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:41.695585    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkln9" podUID="0fca5e3d-e0e1-4b33-909c-46599a84cf4a"
Aug 05 17:21:42.173006 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:42.172968    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-original-pull-secret\") pod \"global-pull-secret-syncer-h8vbl\" (UID: \"bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f\") " pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:21:42.173198 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:42.173029    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs\") pod \"network-metrics-daemon-rkln9\" (UID: \"0fca5e3d-e0e1-4b33-909c-46599a84cf4a\") " pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:21:42.173198 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:42.173131    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:21:42.173198 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:42.173198    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-original-pull-secret podName:bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f nodeName:}" failed. No retries permitted until 2026-08-05 17:21:50.173178514 +0000 UTC m=+17.154755937 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-original-pull-secret") pod "global-pull-secret-syncer-h8vbl" (UID: "bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:21:42.173367 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:42.173137    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:21:42.173367 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:42.173311    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs podName:0fca5e3d-e0e1-4b33-909c-46599a84cf4a nodeName:}" failed. No retries permitted until 2026-08-05 17:21:50.173286101 +0000 UTC m=+17.154863510 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs") pod "network-metrics-daemon-rkln9" (UID: "0fca5e3d-e0e1-4b33-909c-46599a84cf4a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:21:42.274883 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:42.274176    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwfdc\" (UniqueName: \"kubernetes.io/projected/f8972708-830a-4ebc-b521-8a16ca1aee39-kube-api-access-kwfdc\") pod \"network-check-target-4jfwd\" (UID: \"f8972708-830a-4ebc-b521-8a16ca1aee39\") " pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:21:42.274883 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:42.274391    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:21:42.274883 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:42.274409    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:21:42.274883 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:42.274421    2550 projected.go:196] Error preparing data for projected volume kube-api-access-kwfdc for pod openshift-network-diagnostics/network-check-target-4jfwd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:21:42.274883 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:42.274481    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8972708-830a-4ebc-b521-8a16ca1aee39-kube-api-access-kwfdc podName:f8972708-830a-4ebc-b521-8a16ca1aee39 nodeName:}" failed. No retries permitted until 2026-08-05 17:21:50.274462115 +0000 UTC m=+17.256039524 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwfdc" (UniqueName: "kubernetes.io/projected/f8972708-830a-4ebc-b521-8a16ca1aee39-kube-api-access-kwfdc") pod "network-check-target-4jfwd" (UID: "f8972708-830a-4ebc-b521-8a16ca1aee39") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:21:43.695091 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:43.695054    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:21:43.695533 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:43.695181    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkln9" podUID="0fca5e3d-e0e1-4b33-909c-46599a84cf4a"
Aug 05 17:21:43.695533 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:43.695330    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:21:43.695533 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:43.695338    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:21:43.695533 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:43.695443    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4jfwd" podUID="f8972708-830a-4ebc-b521-8a16ca1aee39"
Aug 05 17:21:43.695774 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:43.695559    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8vbl" podUID="bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f"
Aug 05 17:21:45.694954 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:45.694923    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:21:45.694954 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:45.694951    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:21:45.695449 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:45.695039    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkln9" podUID="0fca5e3d-e0e1-4b33-909c-46599a84cf4a"
Aug 05 17:21:45.695449 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:45.694921    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:21:45.695449 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:45.695120    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8vbl" podUID="bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f"
Aug 05 17:21:45.695449 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:45.695222    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4jfwd" podUID="f8972708-830a-4ebc-b521-8a16ca1aee39"
Aug 05 17:21:47.694832 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:47.694793    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:21:47.695287 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:47.694800    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:21:47.695287 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:47.694938    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8vbl" podUID="bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f"
Aug 05 17:21:47.695287 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:47.694802    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:21:47.695287 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:47.694990    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4jfwd" podUID="f8972708-830a-4ebc-b521-8a16ca1aee39"
Aug 05 17:21:47.695627 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:47.695373    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkln9" podUID="0fca5e3d-e0e1-4b33-909c-46599a84cf4a"
Aug 05 17:21:49.697631 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:49.697604    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:21:49.698096 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:49.697633    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:21:49.698096 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:49.697633    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:21:49.698096 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:49.697730    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8vbl" podUID="bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f"
Aug 05 17:21:49.698096 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:49.697811    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4jfwd" podUID="f8972708-830a-4ebc-b521-8a16ca1aee39"
Aug 05 17:21:49.698096 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:49.697890    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkln9" podUID="0fca5e3d-e0e1-4b33-909c-46599a84cf4a"
Aug 05 17:21:50.229149 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:50.229109    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs\") pod \"network-metrics-daemon-rkln9\" (UID: \"0fca5e3d-e0e1-4b33-909c-46599a84cf4a\") " pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:21:50.229329 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:50.229201    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-original-pull-secret\") pod \"global-pull-secret-syncer-h8vbl\" (UID: \"bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f\") " pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:21:50.229329 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:50.229269    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:21:50.229329 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:50.229296    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:21:50.229482 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:50.229347    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs podName:0fca5e3d-e0e1-4b33-909c-46599a84cf4a nodeName:}" failed. No retries permitted until 2026-08-05 17:22:06.229325466 +0000 UTC m=+33.210902890 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs") pod "network-metrics-daemon-rkln9" (UID: "0fca5e3d-e0e1-4b33-909c-46599a84cf4a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:21:50.229482 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:50.229367    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-original-pull-secret podName:bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f nodeName:}" failed. No retries permitted until 2026-08-05 17:22:06.229356773 +0000 UTC m=+33.210934183 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-original-pull-secret") pod "global-pull-secret-syncer-h8vbl" (UID: "bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:21:50.330073 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:50.329978    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwfdc\" (UniqueName: \"kubernetes.io/projected/f8972708-830a-4ebc-b521-8a16ca1aee39-kube-api-access-kwfdc\") pod \"network-check-target-4jfwd\" (UID: \"f8972708-830a-4ebc-b521-8a16ca1aee39\") " pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:21:50.330253 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:50.330165    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:21:50.330253 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:50.330189    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:21:50.330253 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:50.330204    2550 projected.go:196] Error preparing data for projected volume kube-api-access-kwfdc for pod openshift-network-diagnostics/network-check-target-4jfwd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:21:50.330404 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:50.330276    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8972708-830a-4ebc-b521-8a16ca1aee39-kube-api-access-kwfdc podName:f8972708-830a-4ebc-b521-8a16ca1aee39 nodeName:}" failed. No retries permitted until 2026-08-05 17:22:06.330255526 +0000 UTC m=+33.311832935 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwfdc" (UniqueName: "kubernetes.io/projected/f8972708-830a-4ebc-b521-8a16ca1aee39-kube-api-access-kwfdc") pod "network-check-target-4jfwd" (UID: "f8972708-830a-4ebc-b521-8a16ca1aee39") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:21:51.696826 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:51.696796    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:21:51.697270 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:51.696888    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:21:51.697270 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:51.696895    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4jfwd" podUID="f8972708-830a-4ebc-b521-8a16ca1aee39"
Aug 05 17:21:51.697270 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:51.696915    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:21:51.697270 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:51.696990    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8vbl" podUID="bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f"
Aug 05 17:21:51.697270 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:51.697065    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkln9" podUID="0fca5e3d-e0e1-4b33-909c-46599a84cf4a"
Aug 05 17:21:53.695537 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:53.695333    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:21:53.695968 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:53.695391    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:21:53.695968 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:53.695603    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4jfwd" podUID="f8972708-830a-4ebc-b521-8a16ca1aee39"
Aug 05 17:21:53.695968 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:53.695433    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:21:53.695968 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:53.695690    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8vbl" podUID="bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f"
Aug 05 17:21:53.695968 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:53.695764    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkln9" podUID="0fca5e3d-e0e1-4b33-909c-46599a84cf4a"
Aug 05 17:21:54.910880 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:54.910582    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g55gw" event={"ID":"bf34f30e-7b6a-49fc-a27f-e1955f248d7c","Type":"ContainerStarted","Data":"818aa7afd7999b4484ddf6ce817657260895074cc5b8b42c54db7ced851ccb68"}
Aug 05 17:21:54.912313 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:54.912285    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fbk5k" event={"ID":"882b422e-055d-423f-b318-c8e66a901dd9","Type":"ContainerStarted","Data":"926916c75dc6e11a25b8c0cd7d272a48f24c9fd14cf3783ae6af1bceed6e50c0"}
Aug 05 17:21:54.913835 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:54.913801    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2f92r" event={"ID":"b382a0b8-435d-430a-a8a2-faa3d0b3bdc1","Type":"ContainerStarted","Data":"4f9cd337156c5993190804d608885196a8151b1f6934f9b1561782446ecb5f82"}
Aug 05 17:21:54.916217 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:54.916200    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 17:21:54.916515 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:54.916494    2550 generic.go:356] "Generic (PLEG): container finished" podID="1e69a0d5-f55f-4ff7-bd66-82fcc68f8674" containerID="81b32be2685bee66f66e40580f16ce66dd3f496768ca2c613ad3de9e8dd9bf68" exitCode=1
Aug 05 17:21:54.916602 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:54.916555    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-thnnz" event={"ID":"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674","Type":"ContainerStarted","Data":"0e072313b6cedcadbc2dc3e3d61ed3e6e31d28a0fba40c5604da228aabbdfb48"}
Aug 05 17:21:54.916645 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:54.916615    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-thnnz" event={"ID":"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674","Type":"ContainerStarted","Data":"74a846b0c4d255d26660c45bb0aa83e946588690cfda659447a1b8f13d8d7e3f"}
Aug 05 17:21:54.916758 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:54.916647    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-thnnz" event={"ID":"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674","Type":"ContainerDied","Data":"81b32be2685bee66f66e40580f16ce66dd3f496768ca2c613ad3de9e8dd9bf68"}
Aug 05 17:21:54.916758 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:54.916680    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-thnnz" event={"ID":"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674","Type":"ContainerStarted","Data":"2380a87d8f028ed1a1a16c261575dd7da94928edc799f074c1640734b60bf56c"}
Aug 05 17:21:54.918211 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:54.918164    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-46.ec2.internal" event={"ID":"af6ae06b6895f6ad9d2e2bf995b0fd40","Type":"ContainerStarted","Data":"bc99b92ee29873a89567a176505d7c0d749b683bc3165d18e7b9e305ec895e6f"}
Aug 05 17:21:54.919816 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:54.919791    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-srf7f" event={"ID":"749fcabe-941f-454a-b97e-49d077abd98b","Type":"ContainerStarted","Data":"abdceca109d9aa432ad9dd597979443223af11e4230ea1a8d129d057d1d2c81c"}
Aug 05 17:21:54.921381 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:54.921360    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xhnch" event={"ID":"5bd15c5f-5f78-4cba-b190-2e22851079f8","Type":"ContainerStarted","Data":"a16b9adead3feec93b7a821fb5eafffd76210a78538c651ae4bf126cbca15c23"}
Aug 05 17:21:54.922971 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:54.922940    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4" event={"ID":"0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7","Type":"ContainerStarted","Data":"a856ec41a425780e85793a4175f3a7f18a5d59a97ab13a5619b755e055d143ed"}
Aug 05 17:21:54.924434 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:54.924413    2550 generic.go:356] "Generic (PLEG): container finished" podID="c95b41ba2403920dcca145af0d4b9bb1" containerID="7f3fb9b336826617a6d3383ee83544d7f4b454af2088f6f5115bbf4511ab0f08" exitCode=0
Aug 05 17:21:54.924509 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:54.924488    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.ec2.internal" event={"ID":"c95b41ba2403920dcca145af0d4b9bb1","Type":"ContainerDied","Data":"7f3fb9b336826617a6d3383ee83544d7f4b454af2088f6f5115bbf4511ab0f08"}
Aug 05 17:21:54.924772 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:54.924681    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-g55gw" podStartSLOduration=2.495248502 podStartE2EDuration="21.924670377s" totalImagesPullingTime="19.429421875s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:21:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:21:54.924093097 +0000 UTC m=+21.905670528" watchObservedRunningTime="2026-08-05 17:21:54.924670377 +0000 UTC m=+21.906247806"
Aug 05 17:21:54.926077 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:54.926058    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bpcrr" event={"ID":"610a012b-bd9a-4627-8976-b1f785945d61","Type":"ContainerStarted","Data":"eaad3d50c49f5d97fbbbddde17fb7a17c35d8336f5a725bb916c6f28aeb9db08"}
Aug 05 17:21:54.939585 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:54.939553    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-46.ec2.internal" podStartSLOduration=20.93954258 podStartE2EDuration="20.93954258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:21:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:21:54.939249726 +0000 UTC m=+21.920827154" watchObservedRunningTime="2026-08-05 17:21:54.93954258 +0000 UTC m=+21.921120009"
Aug 05 17:21:54.953772 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:54.953738    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-srf7f" podStartSLOduration=2.481650429 podStartE2EDuration="21.953727555s" totalImagesPullingTime="19.472077126s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:21:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:21:54.953127504 +0000 UTC m=+21.934704945" watchObservedRunningTime="2026-08-05 17:21:54.953727555 +0000 UTC m=+21.935304984"
Aug 05 17:21:54.969929 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:54.969767    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fbk5k" podStartSLOduration=2.229297511 podStartE2EDuration="21.969755547s" totalImagesPullingTime="19.740458036s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:21:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:21:54.96886899 +0000 UTC m=+21.950446420" watchObservedRunningTime="2026-08-05 17:21:54.969755547 +0000 UTC m=+21.951332975"
Aug 05 17:21:55.016766 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:55.016732    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2f92r" podStartSLOduration=1.6654987509999999 podStartE2EDuration="21.016719709s" totalImagesPullingTime="19.351220958s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:21:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:21:55.016650603 +0000 UTC m=+21.998228033" watchObservedRunningTime="2026-08-05 17:21:55.016719709 +0000 UTC m=+21.998297137"
Aug 05 17:21:55.029681 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:55.029648    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bpcrr" podStartSLOduration=2.614278257 podStartE2EDuration="22.029625456s" totalImagesPullingTime="19.415347199s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:21:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:21:55.029554465 +0000 UTC m=+22.011131895" watchObservedRunningTime="2026-08-05 17:21:55.029625456 +0000 UTC m=+22.011202885"
Aug 05 17:21:55.694335 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:55.694306    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:21:55.694487 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:55.694316    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:21:55.694487 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:55.694437    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:21:55.694487 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:55.694432    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4jfwd" podUID="f8972708-830a-4ebc-b521-8a16ca1aee39"
Aug 05 17:21:55.694646 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:55.694539    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8vbl" podUID="bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f"
Aug 05 17:21:55.694729 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:55.694634    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkln9" podUID="0fca5e3d-e0e1-4b33-909c-46599a84cf4a"
Aug 05 17:21:55.929864 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:55.929825    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8gvbd" event={"ID":"ee639fbc-354e-4612-bf13-47c4e4df965f","Type":"ContainerStarted","Data":"6336e7b3b9974078de586a1936970e1e98dba5735e611a7220fcd0f992546f2e"}
Aug 05 17:21:55.933091 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:55.933073    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 17:21:55.933513 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:55.933490    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-thnnz" event={"ID":"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674","Type":"ContainerStarted","Data":"7c5db0b21d38e0f9d55b2efbcd2b8601345b5fcb2200e007d42736c9153b807d"}
Aug 05 17:21:55.933606 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:55.933570    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-thnnz" event={"ID":"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674","Type":"ContainerStarted","Data":"5d7dd4296f8781f6181ee9d0779f555119abf4e7dbe57b1a06ee65fe176b7ea5"}
Aug 05 17:21:55.934933 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:55.934909    2550 generic.go:356] "Generic (PLEG): container finished" podID="5bd15c5f-5f78-4cba-b190-2e22851079f8" containerID="a16b9adead3feec93b7a821fb5eafffd76210a78538c651ae4bf126cbca15c23" exitCode=0
Aug 05 17:21:55.935045 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:55.934987    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xhnch" event={"ID":"5bd15c5f-5f78-4cba-b190-2e22851079f8","Type":"ContainerDied","Data":"a16b9adead3feec93b7a821fb5eafffd76210a78538c651ae4bf126cbca15c23"}
Aug 05 17:21:55.936793 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:55.936770    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.ec2.internal" event={"ID":"c95b41ba2403920dcca145af0d4b9bb1","Type":"ContainerStarted","Data":"f4679fea9aa4e51d02b700934222f7fd2b611eecd03ef2916f1d703e2511ee31"}
Aug 05 17:21:55.970351 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:55.970272    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8gvbd" podStartSLOduration=21.970259681 podStartE2EDuration="21.970259681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:21:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=1 observedRunningTime="2026-08-05 17:21:55.969849442 +0000 UTC m=+22.951426872" watchObservedRunningTime="2026-08-05 17:21:55.970259681 +0000 UTC m=+22.951837102"
Aug 05 17:21:55.987011 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:55.986974    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.ec2.internal" podStartSLOduration=21.986961977 podStartE2EDuration="21.986961977s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:21:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:21:55.986837165 +0000 UTC m=+22.968414622" watchObservedRunningTime="2026-08-05 17:21:55.986961977 +0000 UTC m=+22.968539406"
Aug 05 17:21:56.286168 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:56.286136    2550 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Aug 05 17:21:56.671917 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:56.671716    2550 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-05T17:21:56.286165404Z","UUID":"c60fd354-6f60-4885-85c6-333bc9712fa0","Handler":null,"Name":"","Endpoint":""}
Aug 05 17:21:56.673489 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:56.673458    2550 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Aug 05 17:21:56.673613 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:56.673505    2550 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Aug 05 17:21:56.863273 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:56.863188    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-g55gw"
Aug 05 17:21:56.863949 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:56.863924    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-g55gw"
Aug 05 17:21:56.940035 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:56.939956    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4" event={"ID":"0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7","Type":"ContainerStarted","Data":"3a0e41c67f06c1b81f9c4e9aa526cc4f661f9c97d8ea0fe9ad8b6ae00fad0e23"}
Aug 05 17:21:56.940402 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:56.940291    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-g55gw"
Aug 05 17:21:56.940651 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:56.940637    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-g55gw"
Aug 05 17:21:57.694123 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:57.694083    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:21:57.694123 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:57.694110    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:21:57.694447 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:57.694144    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:21:57.694447 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:57.694246    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkln9" podUID="0fca5e3d-e0e1-4b33-909c-46599a84cf4a"
Aug 05 17:21:57.694447 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:57.694302    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4jfwd" podUID="f8972708-830a-4ebc-b521-8a16ca1aee39"
Aug 05 17:21:57.694447 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:57.694370    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8vbl" podUID="bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f"
Aug 05 17:21:57.945376 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:57.945306    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 17:21:57.945867 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:57.945774    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-thnnz" event={"ID":"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674","Type":"ContainerStarted","Data":"5f67899323ce89878d154cafd02a5eeade7c6cd4689261a398d6dc4b5babcd2b"}
Aug 05 17:21:59.694310 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:59.694140    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:21:59.694842 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:59.694141    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:21:59.694842 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:59.694394    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkln9" podUID="0fca5e3d-e0e1-4b33-909c-46599a84cf4a"
Aug 05 17:21:59.694842 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:59.694189    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:21:59.694842 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:59.694479    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8vbl" podUID="bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f"
Aug 05 17:21:59.694842 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:21:59.694568    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4jfwd" podUID="f8972708-830a-4ebc-b521-8a16ca1aee39"
Aug 05 17:21:59.953090 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:59.953022    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 17:21:59.953387 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:59.953363    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-thnnz" event={"ID":"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674","Type":"ContainerStarted","Data":"4a2aa36cdbe3b88c688ce13be40141c94d3e62a74b31b36b0efee5bdcd512ec2"}
Aug 05 17:21:59.953668 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:59.953643    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:59.953793 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:59.953734    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:21:59.953883 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:59.953868    2550 scope.go:117] "RemoveContainer" containerID="81b32be2685bee66f66e40580f16ce66dd3f496768ca2c613ad3de9e8dd9bf68"
Aug 05 17:21:59.955346 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:59.955314    2550 generic.go:356] "Generic (PLEG): container finished" podID="5bd15c5f-5f78-4cba-b190-2e22851079f8" containerID="bcdf1ee78bb9c150cf825c39c3ba66ec7b8fa1131604ff1bacb807231b3095b1" exitCode=0
Aug 05 17:21:59.955425 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:59.955374    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xhnch" event={"ID":"5bd15c5f-5f78-4cba-b190-2e22851079f8","Type":"ContainerDied","Data":"bcdf1ee78bb9c150cf825c39c3ba66ec7b8fa1131604ff1bacb807231b3095b1"}
Aug 05 17:21:59.957718 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:59.957683    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4" event={"ID":"0805ff4f-43b2-46d4-b3bb-fcfbeb5b36b7","Type":"ContainerStarted","Data":"e8539022703fc26d9cd1d3f1f408b1eec28e8855d6fca7acb70bc6b9af2e060e"}
Aug 05 17:21:59.972693 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:21:59.972675    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:22:00.964717 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:00.964672    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 17:22:00.965517 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:00.965173    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-thnnz" event={"ID":"1e69a0d5-f55f-4ff7-bd66-82fcc68f8674","Type":"ContainerStarted","Data":"61c5967d4810a955d2640e9f1ef84ef7b68fe6dec9dce951fd3e148239400df8"}
Aug 05 17:22:00.965811 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:00.965726    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:22:00.982387 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:00.982364    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:22:00.994048 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:00.994009    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-thnnz" podStartSLOduration=7.552281082 podStartE2EDuration="26.993994035s" totalImagesPullingTime="19.441712953s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:21:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:22:00.993117217 +0000 UTC m=+27.974694671" watchObservedRunningTime="2026-08-05 17:22:00.993994035 +0000 UTC m=+27.975571464"
Aug 05 17:22:00.994220 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:00.994201    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ftv4" podStartSLOduration=4.463270473 podStartE2EDuration="27.994196226s" totalImagesPullingTime="23.530925753s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:21:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:22:00.051961497 +0000 UTC m=+27.033538925" watchObservedRunningTime="2026-08-05 17:22:00.994196226 +0000 UTC m=+27.975773646"
Aug 05 17:22:01.379039 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:01.379008    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rkln9"]
Aug 05 17:22:01.379202 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:01.379191    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:22:01.379346 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:01.379317    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkln9" podUID="0fca5e3d-e0e1-4b33-909c-46599a84cf4a"
Aug 05 17:22:01.381608 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:01.381584    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h8vbl"]
Aug 05 17:22:01.381843 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:01.381711    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:22:01.381843 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:01.381812    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8vbl" podUID="bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f"
Aug 05 17:22:01.391176 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:01.391155    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4jfwd"]
Aug 05 17:22:01.391285 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:01.391257    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:22:01.391370 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:01.391348    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4jfwd" podUID="f8972708-830a-4ebc-b521-8a16ca1aee39"
Aug 05 17:22:01.969112 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:01.969078    2550 generic.go:356] "Generic (PLEG): container finished" podID="5bd15c5f-5f78-4cba-b190-2e22851079f8" containerID="3fc2579379f15bb1628990454788022233b158b6c82acf3db6f50d9084d8b3bc" exitCode=0
Aug 05 17:22:01.969455 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:01.969163    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xhnch" event={"ID":"5bd15c5f-5f78-4cba-b190-2e22851079f8","Type":"ContainerDied","Data":"3fc2579379f15bb1628990454788022233b158b6c82acf3db6f50d9084d8b3bc"}
Aug 05 17:22:02.694325 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:02.694286    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:22:02.694520 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:02.694286    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:22:02.694520 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:02.694432    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkln9" podUID="0fca5e3d-e0e1-4b33-909c-46599a84cf4a"
Aug 05 17:22:02.694520 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:02.694463    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8vbl" podUID="bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f"
Aug 05 17:22:03.694938 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:03.694683    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:22:03.695282 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:03.694934    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4jfwd" podUID="f8972708-830a-4ebc-b521-8a16ca1aee39"
Aug 05 17:22:03.974893 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:03.974860    2550 generic.go:356] "Generic (PLEG): container finished" podID="5bd15c5f-5f78-4cba-b190-2e22851079f8" containerID="48dd321e99855aed245679b35bd8993afb7656aaf5dd9e71aca8b423d44aada7" exitCode=0
Aug 05 17:22:03.975025 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:03.974899    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xhnch" event={"ID":"5bd15c5f-5f78-4cba-b190-2e22851079f8","Type":"ContainerDied","Data":"48dd321e99855aed245679b35bd8993afb7656aaf5dd9e71aca8b423d44aada7"}
Aug 05 17:22:04.694520 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:04.694490    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:22:04.694733 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:04.694608    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkln9" podUID="0fca5e3d-e0e1-4b33-909c-46599a84cf4a"
Aug 05 17:22:04.694733 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:04.694658    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:22:04.694858 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:04.694759    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8vbl" podUID="bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f"
Aug 05 17:22:05.694717 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:05.694674    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:22:05.695228 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:05.694818    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4jfwd" podUID="f8972708-830a-4ebc-b521-8a16ca1aee39"
Aug 05 17:22:06.256317 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:06.256284    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-original-pull-secret\") pod \"global-pull-secret-syncer-h8vbl\" (UID: \"bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f\") " pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:22:06.256317 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:06.256326    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs\") pod \"network-metrics-daemon-rkln9\" (UID: \"0fca5e3d-e0e1-4b33-909c-46599a84cf4a\") " pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:22:06.256749 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:06.256459    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:22:06.256749 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:06.256460    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:22:06.256749 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:06.256515    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-original-pull-secret podName:bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f nodeName:}" failed. No retries permitted until 2026-08-05 17:22:38.256500155 +0000 UTC m=+65.238077566 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-original-pull-secret") pod "global-pull-secret-syncer-h8vbl" (UID: "bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:22:06.256749 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:06.256531    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs podName:0fca5e3d-e0e1-4b33-909c-46599a84cf4a nodeName:}" failed. No retries permitted until 2026-08-05 17:22:38.256523706 +0000 UTC m=+65.238101112 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs") pod "network-metrics-daemon-rkln9" (UID: "0fca5e3d-e0e1-4b33-909c-46599a84cf4a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:22:06.357662 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:06.357625    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwfdc\" (UniqueName: \"kubernetes.io/projected/f8972708-830a-4ebc-b521-8a16ca1aee39-kube-api-access-kwfdc\") pod \"network-check-target-4jfwd\" (UID: \"f8972708-830a-4ebc-b521-8a16ca1aee39\") " pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:22:06.357827 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:06.357811    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:22:06.357871 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:06.357834    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:22:06.357871 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:06.357845    2550 projected.go:196] Error preparing data for projected volume kube-api-access-kwfdc for pod openshift-network-diagnostics/network-check-target-4jfwd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:22:06.357932 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:06.357894    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8972708-830a-4ebc-b521-8a16ca1aee39-kube-api-access-kwfdc podName:f8972708-830a-4ebc-b521-8a16ca1aee39 nodeName:}" failed. No retries permitted until 2026-08-05 17:22:38.357880023 +0000 UTC m=+65.339457429 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwfdc" (UniqueName: "kubernetes.io/projected/f8972708-830a-4ebc-b521-8a16ca1aee39-kube-api-access-kwfdc") pod "network-check-target-4jfwd" (UID: "f8972708-830a-4ebc-b521-8a16ca1aee39") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:22:06.694435 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:06.694404    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:22:06.694625 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:06.694406    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:22:06.694625 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:06.694544    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkln9" podUID="0fca5e3d-e0e1-4b33-909c-46599a84cf4a"
Aug 05 17:22:06.694625 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:06.694613    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8vbl" podUID="bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f"
Aug 05 17:22:07.330816 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.330737    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-46.ec2.internal" event="NodeReady"
Aug 05 17:22:07.330960 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.330894    2550 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Aug 05 17:22:07.366781 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.366746    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-d8d4888f6-kkxqq"]
Aug 05 17:22:07.371862 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.371839    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96949699d-rvdxv"]
Aug 05 17:22:07.372018 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.371961    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:07.374777 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.374755    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb"]
Aug 05 17:22:07.374912 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.374897    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96949699d-rvdxv"
Aug 05 17:22:07.375822 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.375801    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Aug 05 17:22:07.376224 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.376195    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Aug 05 17:22:07.376323 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.376239    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Aug 05 17:22:07.376323 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.376195    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4cfrs\""
Aug 05 17:22:07.377510 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.377444    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-576bd4884b-5qmch"]
Aug 05 17:22:07.377607 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.377516    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Aug 05 17:22:07.377807 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.377787    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb"
Aug 05 17:22:07.378669 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.378354    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-srvnh\""
Aug 05 17:22:07.378669 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.378445    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Aug 05 17:22:07.378861 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.378690    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Aug 05 17:22:07.379211 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.379051    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Aug 05 17:22:07.381145 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.381124    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Aug 05 17:22:07.381226 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.381129    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-576bd4884b-5qmch"
Aug 05 17:22:07.381287 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.381230    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Aug 05 17:22:07.381629 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.381607    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Aug 05 17:22:07.381737 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.381665    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Aug 05 17:22:07.384691 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.384261    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Aug 05 17:22:07.386314 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.386287    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96949699d-rvdxv"]
Aug 05 17:22:07.386440 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.386383    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Aug 05 17:22:07.386751 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.386681    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d8d4888f6-kkxqq"]
Aug 05 17:22:07.387572 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.387528    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-576bd4884b-5qmch"]
Aug 05 17:22:07.389134 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.389115    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6xcf5"]
Aug 05 17:22:07.396028 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.395375    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb"]
Aug 05 17:22:07.396028 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.395497    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6xcf5"
Aug 05 17:22:07.396941 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.396920    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gxsh5"]
Aug 05 17:22:07.398065 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.398047    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Aug 05 17:22:07.398356 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.398067    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hv8kx\""
Aug 05 17:22:07.398520 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.398116    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Aug 05 17:22:07.400074 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.400057    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gxsh5"
Aug 05 17:22:07.402467 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.402273    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Aug 05 17:22:07.402467 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.402342    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Aug 05 17:22:07.402467 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.402356    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bkmmp\""
Aug 05 17:22:07.402467 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.402370    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Aug 05 17:22:07.405320 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.405299    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6xcf5"]
Aug 05 17:22:07.407315 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.407297    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gxsh5"]
Aug 05 17:22:07.466293 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.466260    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7dd286cb-de80-463c-8036-217b255c3518-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-96949699d-rvdxv\" (UID: \"7dd286cb-de80-463c-8036-217b255c3518\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96949699d-rvdxv"
Aug 05 17:22:07.466496 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.466473    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f396771-0bb3-4221-a787-c8cc1718032c-tmp\") pod \"klusterlet-addon-workmgr-576bd4884b-5qmch\" (UID: \"0f396771-0bb3-4221-a787-c8cc1718032c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-576bd4884b-5qmch"
Aug 05 17:22:07.466605 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.466510    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xckp4\" (UniqueName: \"kubernetes.io/projected/0cbcfe8b-7d26-444e-9338-c944b590ed0f-kube-api-access-xckp4\") pod \"cluster-proxy-proxy-agent-8655fc97b8-gkddb\" (UID: \"0cbcfe8b-7d26-444e-9338-c944b590ed0f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb"
Aug 05 17:22:07.466605 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.466544    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:07.466605 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.466567    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srcts\" (UniqueName: \"kubernetes.io/projected/0f396771-0bb3-4221-a787-c8cc1718032c-kube-api-access-srcts\") pod \"klusterlet-addon-workmgr-576bd4884b-5qmch\" (UID: \"0f396771-0bb3-4221-a787-c8cc1718032c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-576bd4884b-5qmch"
Aug 05 17:22:07.466605 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.466598    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0cbcfe8b-7d26-444e-9338-c944b590ed0f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-8655fc97b8-gkddb\" (UID: \"0cbcfe8b-7d26-444e-9338-c944b590ed0f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb"
Aug 05 17:22:07.466920 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.466652    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-certificates\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:07.466920 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.466694    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/0cbcfe8b-7d26-444e-9338-c944b590ed0f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-8655fc97b8-gkddb\" (UID: \"0cbcfe8b-7d26-444e-9338-c944b590ed0f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb"
Aug 05 17:22:07.466920 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.466759    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/034231f3-7707-44c2-ad8b-63307a9c4e81-ca-trust-extracted\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:07.466920 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.466798    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0f396771-0bb3-4221-a787-c8cc1718032c-klusterlet-config\") pod \"klusterlet-addon-workmgr-576bd4884b-5qmch\" (UID: \"0f396771-0bb3-4221-a787-c8cc1718032c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-576bd4884b-5qmch"
Aug 05 17:22:07.466920 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.466875    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-bound-sa-token\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:07.466920 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.466902    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86c9z\" (UniqueName: \"kubernetes.io/projected/7dd286cb-de80-463c-8036-217b255c3518-kube-api-access-86c9z\") pod \"managed-serviceaccount-addon-agent-96949699d-rvdxv\" (UID: \"7dd286cb-de80-463c-8036-217b255c3518\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96949699d-rvdxv"
Aug 05 17:22:07.467222 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.466925    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/034231f3-7707-44c2-ad8b-63307a9c4e81-trusted-ca\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:07.467222 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.466963    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/034231f3-7707-44c2-ad8b-63307a9c4e81-installation-pull-secrets\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:07.467222 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.466985    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/0cbcfe8b-7d26-444e-9338-c944b590ed0f-hub\") pod \"cluster-proxy-proxy-agent-8655fc97b8-gkddb\" (UID: \"0cbcfe8b-7d26-444e-9338-c944b590ed0f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb"
Aug 05 17:22:07.467222 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.467007    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x24g9\" (UniqueName: \"kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-kube-api-access-x24g9\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:07.467222 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.467061    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cbcfe8b-7d26-444e-9338-c944b590ed0f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-8655fc97b8-gkddb\" (UID: \"0cbcfe8b-7d26-444e-9338-c944b590ed0f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb"
Aug 05 17:22:07.467222 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.467133    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/034231f3-7707-44c2-ad8b-63307a9c4e81-image-registry-private-configuration\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:07.467222 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.467159    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/0cbcfe8b-7d26-444e-9338-c944b590ed0f-ca\") pod \"cluster-proxy-proxy-agent-8655fc97b8-gkddb\" (UID: \"0cbcfe8b-7d26-444e-9338-c944b590ed0f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb"
Aug 05 17:22:07.568214 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.568179    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/976b5f92-171d-43bc-9117-b2f42f830596-config-volume\") pod \"dns-default-6xcf5\" (UID: \"976b5f92-171d-43bc-9117-b2f42f830596\") " pod="openshift-dns/dns-default-6xcf5"
Aug 05 17:22:07.568388 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.568225    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:07.568388 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.568251    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srcts\" (UniqueName: \"kubernetes.io/projected/0f396771-0bb3-4221-a787-c8cc1718032c-kube-api-access-srcts\") pod \"klusterlet-addon-workmgr-576bd4884b-5qmch\" (UID: \"0f396771-0bb3-4221-a787-c8cc1718032c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-576bd4884b-5qmch"
Aug 05 17:22:07.568388 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:07.568378    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 05 17:22:07.568531 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:07.568401    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d8d4888f6-kkxqq: secret "image-registry-tls" not found
Aug 05 17:22:07.568531 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:07.568477    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls podName:034231f3-7707-44c2-ad8b-63307a9c4e81 nodeName:}" failed. No retries permitted until 2026-08-05 17:22:08.068454557 +0000 UTC m=+35.050031969 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls") pod "image-registry-d8d4888f6-kkxqq" (UID: "034231f3-7707-44c2-ad8b-63307a9c4e81") : secret "image-registry-tls" not found
Aug 05 17:22:07.568531 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.568377    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0cbcfe8b-7d26-444e-9338-c944b590ed0f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-8655fc97b8-gkddb\" (UID: \"0cbcfe8b-7d26-444e-9338-c944b590ed0f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb"
Aug 05 17:22:07.568691 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.568559    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-certificates\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:07.568691 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.568590    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/0cbcfe8b-7d26-444e-9338-c944b590ed0f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-8655fc97b8-gkddb\" (UID: \"0cbcfe8b-7d26-444e-9338-c944b590ed0f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb"
Aug 05 17:22:07.568691 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.568626    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/034231f3-7707-44c2-ad8b-63307a9c4e81-ca-trust-extracted\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:07.568691 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.568651    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0f396771-0bb3-4221-a787-c8cc1718032c-klusterlet-config\") pod \"klusterlet-addon-workmgr-576bd4884b-5qmch\" (UID: \"0f396771-0bb3-4221-a787-c8cc1718032c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-576bd4884b-5qmch"
Aug 05 17:22:07.568691 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.568681    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wbxb\" (UniqueName: \"kubernetes.io/projected/8bdd9d59-2279-49b8-882d-210c0d146301-kube-api-access-7wbxb\") pod \"ingress-canary-gxsh5\" (UID: \"8bdd9d59-2279-49b8-882d-210c0d146301\") " pod="openshift-ingress-canary/ingress-canary-gxsh5"
Aug 05 17:22:07.568949 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.568780    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-bound-sa-token\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:07.568949 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.568814    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86c9z\" (UniqueName: \"kubernetes.io/projected/7dd286cb-de80-463c-8036-217b255c3518-kube-api-access-86c9z\") pod \"managed-serviceaccount-addon-agent-96949699d-rvdxv\" (UID: \"7dd286cb-de80-463c-8036-217b255c3518\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96949699d-rvdxv"
Aug 05 17:22:07.568949 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.568838    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/034231f3-7707-44c2-ad8b-63307a9c4e81-trusted-ca\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:07.568949 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.568862    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/034231f3-7707-44c2-ad8b-63307a9c4e81-installation-pull-secrets\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:07.568949 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.568889    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/0cbcfe8b-7d26-444e-9338-c944b590ed0f-hub\") pod \"cluster-proxy-proxy-agent-8655fc97b8-gkddb\" (UID: \"0cbcfe8b-7d26-444e-9338-c944b590ed0f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb"
Aug 05 17:22:07.568949 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.568917    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x24g9\" (UniqueName: \"kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-kube-api-access-x24g9\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:07.568949 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.568943    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cbcfe8b-7d26-444e-9338-c944b590ed0f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-8655fc97b8-gkddb\" (UID: \"0cbcfe8b-7d26-444e-9338-c944b590ed0f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb"
Aug 05 17:22:07.569269 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.568970    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/976b5f92-171d-43bc-9117-b2f42f830596-metrics-tls\") pod \"dns-default-6xcf5\" (UID: \"976b5f92-171d-43bc-9117-b2f42f830596\") " pod="openshift-dns/dns-default-6xcf5"
Aug 05 17:22:07.569269 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.569006    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/034231f3-7707-44c2-ad8b-63307a9c4e81-ca-trust-extracted\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:07.569269 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.569017    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8bdd9d59-2279-49b8-882d-210c0d146301-cert\") pod \"ingress-canary-gxsh5\" (UID: \"8bdd9d59-2279-49b8-882d-210c0d146301\") " pod="openshift-ingress-canary/ingress-canary-gxsh5"
Aug 05 17:22:07.569269 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.569059    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/034231f3-7707-44c2-ad8b-63307a9c4e81-image-registry-private-configuration\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:07.569269 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.569083    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/0cbcfe8b-7d26-444e-9338-c944b590ed0f-ca\") pod \"cluster-proxy-proxy-agent-8655fc97b8-gkddb\" (UID: \"0cbcfe8b-7d26-444e-9338-c944b590ed0f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb"
Aug 05 17:22:07.569269 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.569106    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/976b5f92-171d-43bc-9117-b2f42f830596-tmp-dir\") pod \"dns-default-6xcf5\" (UID: \"976b5f92-171d-43bc-9117-b2f42f830596\") " pod="openshift-dns/dns-default-6xcf5"
Aug 05 17:22:07.569269 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.569136    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7dd286cb-de80-463c-8036-217b255c3518-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-96949699d-rvdxv\" (UID: \"7dd286cb-de80-463c-8036-217b255c3518\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96949699d-rvdxv"
Aug 05 17:22:07.569269 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.569155    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-certificates\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:07.569269 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.569163    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f396771-0bb3-4221-a787-c8cc1718032c-tmp\") pod \"klusterlet-addon-workmgr-576bd4884b-5qmch\" (UID: \"0f396771-0bb3-4221-a787-c8cc1718032c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-576bd4884b-5qmch"
Aug 05 17:22:07.569670 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.569293    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xckp4\" (UniqueName: \"kubernetes.io/projected/0cbcfe8b-7d26-444e-9338-c944b590ed0f-kube-api-access-xckp4\") pod \"cluster-proxy-proxy-agent-8655fc97b8-gkddb\" (UID: \"0cbcfe8b-7d26-444e-9338-c944b590ed0f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb"
Aug 05 17:22:07.569670 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.569327    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnv6v\" (UniqueName: \"kubernetes.io/projected/976b5f92-171d-43bc-9117-b2f42f830596-kube-api-access-mnv6v\") pod \"dns-default-6xcf5\" (UID: \"976b5f92-171d-43bc-9117-b2f42f830596\") " pod="openshift-dns/dns-default-6xcf5"
Aug 05 17:22:07.569670 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.569524    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f396771-0bb3-4221-a787-c8cc1718032c-tmp\") pod \"klusterlet-addon-workmgr-576bd4884b-5qmch\" (UID: \"0f396771-0bb3-4221-a787-c8cc1718032c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-576bd4884b-5qmch"
Aug 05 17:22:07.571255 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.569882    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/034231f3-7707-44c2-ad8b-63307a9c4e81-trusted-ca\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:07.571255 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.570237    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cbcfe8b-7d26-444e-9338-c944b590ed0f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-8655fc97b8-gkddb\" (UID: \"0cbcfe8b-7d26-444e-9338-c944b590ed0f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb"
Aug 05 17:22:07.573914 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.573831    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/034231f3-7707-44c2-ad8b-63307a9c4e81-installation-pull-secrets\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:07.573914 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.573852    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/0cbcfe8b-7d26-444e-9338-c944b590ed0f-ca\") pod \"cluster-proxy-proxy-agent-8655fc97b8-gkddb\" (UID: \"0cbcfe8b-7d26-444e-9338-c944b590ed0f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb"
Aug 05 17:22:07.574069 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.573920    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/0cbcfe8b-7d26-444e-9338-c944b590ed0f-hub\") pod \"cluster-proxy-proxy-agent-8655fc97b8-gkddb\" (UID: \"0cbcfe8b-7d26-444e-9338-c944b590ed0f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb"
Aug 05 17:22:07.574261 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.574218    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/0cbcfe8b-7d26-444e-9338-c944b590ed0f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-8655fc97b8-gkddb\" (UID: \"0cbcfe8b-7d26-444e-9338-c944b590ed0f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb"
Aug 05 17:22:07.574432 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.574407    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0cbcfe8b-7d26-444e-9338-c944b590ed0f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-8655fc97b8-gkddb\" (UID: \"0cbcfe8b-7d26-444e-9338-c944b590ed0f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb"
Aug 05 17:22:07.574537 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.574479    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/034231f3-7707-44c2-ad8b-63307a9c4e81-image-registry-private-configuration\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:07.574537 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.574503    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7dd286cb-de80-463c-8036-217b255c3518-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-96949699d-rvdxv\" (UID: \"7dd286cb-de80-463c-8036-217b255c3518\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96949699d-rvdxv"
Aug 05 17:22:07.574900 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.574884    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0f396771-0bb3-4221-a787-c8cc1718032c-klusterlet-config\") pod \"klusterlet-addon-workmgr-576bd4884b-5qmch\" (UID: \"0f396771-0bb3-4221-a787-c8cc1718032c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-576bd4884b-5qmch"
Aug 05 17:22:07.577553 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.577524    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86c9z\" (UniqueName: \"kubernetes.io/projected/7dd286cb-de80-463c-8036-217b255c3518-kube-api-access-86c9z\") pod \"managed-serviceaccount-addon-agent-96949699d-rvdxv\" (UID: \"7dd286cb-de80-463c-8036-217b255c3518\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96949699d-rvdxv"
Aug 05 17:22:07.577951 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.577928    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srcts\" (UniqueName: \"kubernetes.io/projected/0f396771-0bb3-4221-a787-c8cc1718032c-kube-api-access-srcts\") pod \"klusterlet-addon-workmgr-576bd4884b-5qmch\" (UID: \"0f396771-0bb3-4221-a787-c8cc1718032c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-576bd4884b-5qmch"
Aug 05 17:22:07.585956 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.585895    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x24g9\" (UniqueName: \"kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-kube-api-access-x24g9\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:07.585956 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.585911    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-bound-sa-token\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:07.586712 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.586671    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xckp4\" (UniqueName: \"kubernetes.io/projected/0cbcfe8b-7d26-444e-9338-c944b590ed0f-kube-api-access-xckp4\") pod \"cluster-proxy-proxy-agent-8655fc97b8-gkddb\" (UID: \"0cbcfe8b-7d26-444e-9338-c944b590ed0f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb"
Aug 05 17:22:07.590971 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.590944    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:07.669985 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.669945    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wbxb\" (UniqueName: \"kubernetes.io/projected/8bdd9d59-2279-49b8-882d-210c0d146301-kube-api-access-7wbxb\") pod \"ingress-canary-gxsh5\" (UID: \"8bdd9d59-2279-49b8-882d-210c0d146301\") " pod="openshift-ingress-canary/ingress-canary-gxsh5"
Aug 05 17:22:07.670224 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.670019    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/976b5f92-171d-43bc-9117-b2f42f830596-metrics-tls\") pod \"dns-default-6xcf5\" (UID: \"976b5f92-171d-43bc-9117-b2f42f830596\") " pod="openshift-dns/dns-default-6xcf5"
Aug 05 17:22:07.670224 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.670046    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8bdd9d59-2279-49b8-882d-210c0d146301-cert\") pod \"ingress-canary-gxsh5\" (UID: \"8bdd9d59-2279-49b8-882d-210c0d146301\") " pod="openshift-ingress-canary/ingress-canary-gxsh5"
Aug 05 17:22:07.670224 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.670072    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/976b5f92-171d-43bc-9117-b2f42f830596-tmp-dir\") pod \"dns-default-6xcf5\" (UID: \"976b5f92-171d-43bc-9117-b2f42f830596\") " pod="openshift-dns/dns-default-6xcf5"
Aug 05 17:22:07.670224 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:07.670167    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 05 17:22:07.670224 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.670213    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnv6v\" (UniqueName: \"kubernetes.io/projected/976b5f92-171d-43bc-9117-b2f42f830596-kube-api-access-mnv6v\") pod \"dns-default-6xcf5\" (UID: \"976b5f92-171d-43bc-9117-b2f42f830596\") " pod="openshift-dns/dns-default-6xcf5"
Aug 05 17:22:07.670468 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:07.670230    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/976b5f92-171d-43bc-9117-b2f42f830596-metrics-tls podName:976b5f92-171d-43bc-9117-b2f42f830596 nodeName:}" failed. No retries permitted until 2026-08-05 17:22:08.170209871 +0000 UTC m=+35.151787284 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/976b5f92-171d-43bc-9117-b2f42f830596-metrics-tls") pod "dns-default-6xcf5" (UID: "976b5f92-171d-43bc-9117-b2f42f830596") : secret "dns-default-metrics-tls" not found
Aug 05 17:22:07.670468 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.670263    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/976b5f92-171d-43bc-9117-b2f42f830596-config-volume\") pod \"dns-default-6xcf5\" (UID: \"976b5f92-171d-43bc-9117-b2f42f830596\") " pod="openshift-dns/dns-default-6xcf5"
Aug 05 17:22:07.670468 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.670406    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/976b5f92-171d-43bc-9117-b2f42f830596-tmp-dir\") pod \"dns-default-6xcf5\" (UID: \"976b5f92-171d-43bc-9117-b2f42f830596\") " pod="openshift-dns/dns-default-6xcf5"
Aug 05 17:22:07.670834 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.670816    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/976b5f92-171d-43bc-9117-b2f42f830596-config-volume\") pod \"dns-default-6xcf5\" (UID: \"976b5f92-171d-43bc-9117-b2f42f830596\") " pod="openshift-dns/dns-default-6xcf5"
Aug 05 17:22:07.672810 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.672789    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8bdd9d59-2279-49b8-882d-210c0d146301-cert\") pod \"ingress-canary-gxsh5\" (UID: \"8bdd9d59-2279-49b8-882d-210c0d146301\") " pod="openshift-ingress-canary/ingress-canary-gxsh5"
Aug 05 17:22:07.679044 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.679015    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:07.679559 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.679531    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnv6v\" (UniqueName: \"kubernetes.io/projected/976b5f92-171d-43bc-9117-b2f42f830596-kube-api-access-mnv6v\") pod \"dns-default-6xcf5\" (UID: \"976b5f92-171d-43bc-9117-b2f42f830596\") " pod="openshift-dns/dns-default-6xcf5"
Aug 05 17:22:07.679777 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.679760    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wbxb\" (UniqueName: \"kubernetes.io/projected/8bdd9d59-2279-49b8-882d-210c0d146301-kube-api-access-7wbxb\") pod \"ingress-canary-gxsh5\" (UID: \"8bdd9d59-2279-49b8-882d-210c0d146301\") " pod="openshift-ingress-canary/ingress-canary-gxsh5"
Aug 05 17:22:07.694772 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.694727    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:22:07.697516 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.697491    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 05 17:22:07.697656 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.697522    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 05 17:22:07.697656 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.697522    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8p6dg\""
Aug 05 17:22:07.711170 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.711145    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96949699d-rvdxv"
Aug 05 17:22:07.718325 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.718303    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb"
Aug 05 17:22:07.739008 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.738979    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-576bd4884b-5qmch"
Aug 05 17:22:07.753397 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:07.753372    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gxsh5"
Aug 05 17:22:08.074576 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:08.074535    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:08.074804 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:08.074711    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 05 17:22:08.074804 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:08.074740    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d8d4888f6-kkxqq: secret "image-registry-tls" not found
Aug 05 17:22:08.074926 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:08.074811    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls podName:034231f3-7707-44c2-ad8b-63307a9c4e81 nodeName:}" failed. No retries permitted until 2026-08-05 17:22:09.074788289 +0000 UTC m=+36.056365701 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls") pod "image-registry-d8d4888f6-kkxqq" (UID: "034231f3-7707-44c2-ad8b-63307a9c4e81") : secret "image-registry-tls" not found
Aug 05 17:22:08.174993 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:08.174949    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/976b5f92-171d-43bc-9117-b2f42f830596-metrics-tls\") pod \"dns-default-6xcf5\" (UID: \"976b5f92-171d-43bc-9117-b2f42f830596\") " pod="openshift-dns/dns-default-6xcf5"
Aug 05 17:22:08.175162 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:08.175123    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 05 17:22:08.175218 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:08.175196    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/976b5f92-171d-43bc-9117-b2f42f830596-metrics-tls podName:976b5f92-171d-43bc-9117-b2f42f830596 nodeName:}" failed. No retries permitted until 2026-08-05 17:22:09.175175334 +0000 UTC m=+36.156752745 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/976b5f92-171d-43bc-9117-b2f42f830596-metrics-tls") pod "dns-default-6xcf5" (UID: "976b5f92-171d-43bc-9117-b2f42f830596") : secret "dns-default-metrics-tls" not found
Aug 05 17:22:08.696938 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:08.694800    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:22:08.696938 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:08.695461    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:22:08.700292 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:08.699102    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 05 17:22:08.700292 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:08.699304    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 05 17:22:08.700292 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:08.700193    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l8xxn\""
Aug 05 17:22:08.743715 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:08.743669    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gxsh5"]
Aug 05 17:22:08.745213 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:08.745185    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96949699d-rvdxv"]
Aug 05 17:22:08.745988 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:08.745968    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb"]
Aug 05 17:22:08.763205 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:08.763177    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-576bd4884b-5qmch"]
Aug 05 17:22:08.777453 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:22:08.777384    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bdd9d59_2279_49b8_882d_210c0d146301.slice/crio-9c1b9871c1cac4d4710891bdc91df9316b4ec299c7d369e6ca087297bb632c34 WatchSource:0}: Error finding container 9c1b9871c1cac4d4710891bdc91df9316b4ec299c7d369e6ca087297bb632c34: Status 404 returned error can't find the container with id 9c1b9871c1cac4d4710891bdc91df9316b4ec299c7d369e6ca087297bb632c34
Aug 05 17:22:08.778053 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:22:08.778021    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0cbcfe8b_7d26_444e_9338_c944b590ed0f.slice/crio-75530dfbe22aad7793aa28c5d3114ac4564b4ae7750847a70f765162982ed940 WatchSource:0}: Error finding container 75530dfbe22aad7793aa28c5d3114ac4564b4ae7750847a70f765162982ed940: Status 404 returned error can't find the container with id 75530dfbe22aad7793aa28c5d3114ac4564b4ae7750847a70f765162982ed940
Aug 05 17:22:08.778587 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:22:08.778565    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7dd286cb_de80_463c_8036_217b255c3518.slice/crio-f7e5cc0ccb36b8377a89292837dc09cc07e34afe6d13ae0ffa6b6af2e773d1ce WatchSource:0}: Error finding container f7e5cc0ccb36b8377a89292837dc09cc07e34afe6d13ae0ffa6b6af2e773d1ce: Status 404 returned error can't find the container with id f7e5cc0ccb36b8377a89292837dc09cc07e34afe6d13ae0ffa6b6af2e773d1ce
Aug 05 17:22:08.779609 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:22:08.779585    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f396771_0bb3_4221_a787_c8cc1718032c.slice/crio-b4ccb16450eb1cbadd0059a5b477dd842031b1e92265ba966646ac12571b7a53 WatchSource:0}: Error finding container b4ccb16450eb1cbadd0059a5b477dd842031b1e92265ba966646ac12571b7a53: Status 404 returned error can't find the container with id b4ccb16450eb1cbadd0059a5b477dd842031b1e92265ba966646ac12571b7a53
Aug 05 17:22:08.988332 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:08.988286    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-576bd4884b-5qmch" event={"ID":"0f396771-0bb3-4221-a787-c8cc1718032c","Type":"ContainerStarted","Data":"b4ccb16450eb1cbadd0059a5b477dd842031b1e92265ba966646ac12571b7a53"}
Aug 05 17:22:08.989426 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:08.989400    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb" event={"ID":"0cbcfe8b-7d26-444e-9338-c944b590ed0f","Type":"ContainerStarted","Data":"75530dfbe22aad7793aa28c5d3114ac4564b4ae7750847a70f765162982ed940"}
Aug 05 17:22:08.990502 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:08.990479    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96949699d-rvdxv" event={"ID":"7dd286cb-de80-463c-8036-217b255c3518","Type":"ContainerStarted","Data":"f7e5cc0ccb36b8377a89292837dc09cc07e34afe6d13ae0ffa6b6af2e773d1ce"}
Aug 05 17:22:08.993380 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:08.993350    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xhnch" event={"ID":"5bd15c5f-5f78-4cba-b190-2e22851079f8","Type":"ContainerStarted","Data":"81aaa679781802f8d209a85874c409f62537c9630eb08c630aa940f01753aab3"}
Aug 05 17:22:08.994464 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:08.994445    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gxsh5" event={"ID":"8bdd9d59-2279-49b8-882d-210c0d146301","Type":"ContainerStarted","Data":"9c1b9871c1cac4d4710891bdc91df9316b4ec299c7d369e6ca087297bb632c34"}
Aug 05 17:22:09.084557 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:09.084483    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:09.084694 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:09.084632    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 05 17:22:09.084694 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:09.084651    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d8d4888f6-kkxqq: secret "image-registry-tls" not found
Aug 05 17:22:09.084799 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:09.084715    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls podName:034231f3-7707-44c2-ad8b-63307a9c4e81 nodeName:}" failed. No retries permitted until 2026-08-05 17:22:11.084685316 +0000 UTC m=+38.066262735 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls") pod "image-registry-d8d4888f6-kkxqq" (UID: "034231f3-7707-44c2-ad8b-63307a9c4e81") : secret "image-registry-tls" not found
Aug 05 17:22:09.185995 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:09.185960    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/976b5f92-171d-43bc-9117-b2f42f830596-metrics-tls\") pod \"dns-default-6xcf5\" (UID: \"976b5f92-171d-43bc-9117-b2f42f830596\") " pod="openshift-dns/dns-default-6xcf5"
Aug 05 17:22:09.186168 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:09.186107    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 05 17:22:09.186210 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:09.186169    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/976b5f92-171d-43bc-9117-b2f42f830596-metrics-tls podName:976b5f92-171d-43bc-9117-b2f42f830596 nodeName:}" failed. No retries permitted until 2026-08-05 17:22:11.186153905 +0000 UTC m=+38.167731311 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/976b5f92-171d-43bc-9117-b2f42f830596-metrics-tls") pod "dns-default-6xcf5" (UID: "976b5f92-171d-43bc-9117-b2f42f830596") : secret "dns-default-metrics-tls" not found
Aug 05 17:22:10.016680 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:10.015758    2550 generic.go:356] "Generic (PLEG): container finished" podID="5bd15c5f-5f78-4cba-b190-2e22851079f8" containerID="81aaa679781802f8d209a85874c409f62537c9630eb08c630aa940f01753aab3" exitCode=0
Aug 05 17:22:10.016680 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:10.015817    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xhnch" event={"ID":"5bd15c5f-5f78-4cba-b190-2e22851079f8","Type":"ContainerDied","Data":"81aaa679781802f8d209a85874c409f62537c9630eb08c630aa940f01753aab3"}
Aug 05 17:22:11.027188 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:11.026406    2550 generic.go:356] "Generic (PLEG): container finished" podID="5bd15c5f-5f78-4cba-b190-2e22851079f8" containerID="9abc224e1c692ff2fab1d19f22dbb981b8420518784307581ced7dad2db7e625" exitCode=0
Aug 05 17:22:11.027188 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:11.026470    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xhnch" event={"ID":"5bd15c5f-5f78-4cba-b190-2e22851079f8","Type":"ContainerDied","Data":"9abc224e1c692ff2fab1d19f22dbb981b8420518784307581ced7dad2db7e625"}
Aug 05 17:22:11.105082 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:11.104481    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:11.105082 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:11.104646    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 05 17:22:11.105082 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:11.104664    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d8d4888f6-kkxqq: secret "image-registry-tls" not found
Aug 05 17:22:11.105082 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:11.104743    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls podName:034231f3-7707-44c2-ad8b-63307a9c4e81 nodeName:}" failed. No retries permitted until 2026-08-05 17:22:15.104723652 +0000 UTC m=+42.086301061 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls") pod "image-registry-d8d4888f6-kkxqq" (UID: "034231f3-7707-44c2-ad8b-63307a9c4e81") : secret "image-registry-tls" not found
Aug 05 17:22:11.205663 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:11.205118    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/976b5f92-171d-43bc-9117-b2f42f830596-metrics-tls\") pod \"dns-default-6xcf5\" (UID: \"976b5f92-171d-43bc-9117-b2f42f830596\") " pod="openshift-dns/dns-default-6xcf5"
Aug 05 17:22:11.205663 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:11.205650    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 05 17:22:11.205884 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:11.205741    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/976b5f92-171d-43bc-9117-b2f42f830596-metrics-tls podName:976b5f92-171d-43bc-9117-b2f42f830596 nodeName:}" failed. No retries permitted until 2026-08-05 17:22:15.205719518 +0000 UTC m=+42.187296948 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/976b5f92-171d-43bc-9117-b2f42f830596-metrics-tls") pod "dns-default-6xcf5" (UID: "976b5f92-171d-43bc-9117-b2f42f830596") : secret "dns-default-metrics-tls" not found
Aug 05 17:22:12.936238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:12.936026    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:13.004132 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:13.004090    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:15.137953 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:15.137911    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:15.138335 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:15.138057    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 05 17:22:15.138335 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:15.138073    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d8d4888f6-kkxqq: secret "image-registry-tls" not found
Aug 05 17:22:15.138335 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:15.138128    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls podName:034231f3-7707-44c2-ad8b-63307a9c4e81 nodeName:}" failed. No retries permitted until 2026-08-05 17:22:23.138111839 +0000 UTC m=+50.119689257 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls") pod "image-registry-d8d4888f6-kkxqq" (UID: "034231f3-7707-44c2-ad8b-63307a9c4e81") : secret "image-registry-tls" not found
Aug 05 17:22:15.238937 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:15.238894    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/976b5f92-171d-43bc-9117-b2f42f830596-metrics-tls\") pod \"dns-default-6xcf5\" (UID: \"976b5f92-171d-43bc-9117-b2f42f830596\") " pod="openshift-dns/dns-default-6xcf5"
Aug 05 17:22:15.239109 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:15.239074    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 05 17:22:15.239153 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:15.239136    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/976b5f92-171d-43bc-9117-b2f42f830596-metrics-tls podName:976b5f92-171d-43bc-9117-b2f42f830596 nodeName:}" failed. No retries permitted until 2026-08-05 17:22:23.239120284 +0000 UTC m=+50.220697692 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/976b5f92-171d-43bc-9117-b2f42f830596-metrics-tls") pod "dns-default-6xcf5" (UID: "976b5f92-171d-43bc-9117-b2f42f830596") : secret "dns-default-metrics-tls" not found
Aug 05 17:22:16.378572 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:16.378514    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:16.457611 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:16.457561    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:18.045031 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:18.044997    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96949699d-rvdxv" event={"ID":"7dd286cb-de80-463c-8036-217b255c3518","Type":"ContainerStarted","Data":"90cfaf0dc67aafd265c57096d850d4d587ac18560c3b39ac0f6f3cf8f2deefb9"}
Aug 05 17:22:18.048285 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:18.048248    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xhnch" event={"ID":"5bd15c5f-5f78-4cba-b190-2e22851079f8","Type":"ContainerStarted","Data":"1d6e04f8b67b1422f2ed541198bfd529829d30a03ea2fcd0804c9d61eae7e79a"}
Aug 05 17:22:18.049631 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:18.049602    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gxsh5" event={"ID":"8bdd9d59-2279-49b8-882d-210c0d146301","Type":"ContainerStarted","Data":"ee4cf88c0e05beb16de8280850bff2adbba16dcc1a0c03bed7d43a1f600b740a"}
Aug 05 17:22:18.050948 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:18.050928    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-576bd4884b-5qmch" event={"ID":"0f396771-0bb3-4221-a787-c8cc1718032c","Type":"ContainerStarted","Data":"265aa3850aaa108d00553d333dd63c158e200687fcee0fcdbb2051f349c2a042"}
Aug 05 17:22:18.051063 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:18.051047    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-576bd4884b-5qmch"
Aug 05 17:22:18.052880 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:18.052862    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-576bd4884b-5qmch"
Aug 05 17:22:18.053144 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:18.053127    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb" event={"ID":"0cbcfe8b-7d26-444e-9338-c944b590ed0f","Type":"ContainerStarted","Data":"d3898bc7e92466a4a133ffc9ee635c31041f9002c6eff511d26578c8723aa250"}
Aug 05 17:22:18.053189 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:18.053156    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb" event={"ID":"0cbcfe8b-7d26-444e-9338-c944b590ed0f","Type":"ContainerStarted","Data":"2bc4f67f98c5272494442fdc1cc62daffd940ce2a4bccc41f2860cf43a03035d"}
Aug 05 17:22:18.063174 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:18.063102    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96949699d-rvdxv" podStartSLOduration=8.201148917 podStartE2EDuration="17.063083266s" totalImagesPullingTime="8.861934349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:22:18.061888182 +0000 UTC m=+45.043465612" watchObservedRunningTime="2026-08-05 17:22:18.063083266 +0000 UTC m=+45.044660696"
Aug 05 17:22:18.080823 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:18.080778    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-576bd4884b-5qmch" podStartSLOduration=8.203976107 podStartE2EDuration="17.080763427s" totalImagesPullingTime="8.87678732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:22:18.080322454 +0000 UTC m=+45.061899883" watchObservedRunningTime="2026-08-05 17:22:18.080763427 +0000 UTC m=+45.062340856"
Aug 05 17:22:18.100628 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:18.100576    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gxsh5" podStartSLOduration=2.238592753 podStartE2EDuration="11.100558475s" totalImagesPullingTime="8.861965722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:22:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:22:18.099867476 +0000 UTC m=+45.081444905" watchObservedRunningTime="2026-08-05 17:22:18.100558475 +0000 UTC m=+45.082135903"
Aug 05 17:22:18.127493 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:18.127385    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xhnch" podStartSLOduration=14.301934371 podStartE2EDuration="45.127372957s" totalImagesPullingTime="30.825438586s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:21:33 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:22:18.125747927 +0000 UTC m=+45.107325353" watchObservedRunningTime="2026-08-05 17:22:18.127372957 +0000 UTC m=+45.108950385"
Aug 05 17:22:18.250659 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:18.250627    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:18.351988 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:18.351948    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:19.058766 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:19.058726    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb" event={"ID":"0cbcfe8b-7d26-444e-9338-c944b590ed0f","Type":"ContainerStarted","Data":"1befae7d1e96735589e458df0aa14c17d2102a74c60341ab3f13b3bd6f21392a"}
Aug 05 17:22:19.081308 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:19.081255    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb" podStartSLOduration=9.219230516 podStartE2EDuration="18.08123969s" totalImagesPullingTime="8.862009174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:22:19.080450656 +0000 UTC m=+46.062028084" watchObservedRunningTime="2026-08-05 17:22:19.08123969 +0000 UTC m=+46.062817097"
Aug 05 17:22:23.199919 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:23.199880    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:23.200310 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:23.200008    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 05 17:22:23.200310 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:23.200020    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d8d4888f6-kkxqq: secret "image-registry-tls" not found
Aug 05 17:22:23.200310 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:23.200070    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls podName:034231f3-7707-44c2-ad8b-63307a9c4e81 nodeName:}" failed. No retries permitted until 2026-08-05 17:22:39.20005666 +0000 UTC m=+66.181634067 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls") pod "image-registry-d8d4888f6-kkxqq" (UID: "034231f3-7707-44c2-ad8b-63307a9c4e81") : secret "image-registry-tls" not found
Aug 05 17:22:23.300244 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:23.300204    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/976b5f92-171d-43bc-9117-b2f42f830596-metrics-tls\") pod \"dns-default-6xcf5\" (UID: \"976b5f92-171d-43bc-9117-b2f42f830596\") " pod="openshift-dns/dns-default-6xcf5"
Aug 05 17:22:23.300413 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:23.300350    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 05 17:22:23.300413 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:23.300410    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/976b5f92-171d-43bc-9117-b2f42f830596-metrics-tls podName:976b5f92-171d-43bc-9117-b2f42f830596 nodeName:}" failed. No retries permitted until 2026-08-05 17:22:39.300395359 +0000 UTC m=+66.281972766 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/976b5f92-171d-43bc-9117-b2f42f830596-metrics-tls") pod "dns-default-6xcf5" (UID: "976b5f92-171d-43bc-9117-b2f42f830596") : secret "dns-default-metrics-tls" not found
Aug 05 17:22:24.051285 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:24.051251    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:24.126396 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:24.126358    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:24.213510 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:24.213472    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:24.296023 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:24.295990    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:24.368180 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:24.368095    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:24.469893 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:24.469861    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:25.164292 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:25.164257    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:25.221550 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:25.221524    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:25.564532 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:25.564445    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:25.628803 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:25.628775    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:32.141403 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:32.141374    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:32.224488 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:32.224453    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:32.320968 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:32.320933    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:32.421479 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:32.421444    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:32.526017 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:32.525987    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:32.982408 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:32.982380    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-thnnz"
Aug 05 17:22:33.151256 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:33.151222    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:33.213562 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:33.213525    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:33.538332 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:33.538297    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:33.593360 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:33.593332    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:38.312330 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:38.312287    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-original-pull-secret\") pod \"global-pull-secret-syncer-h8vbl\" (UID: \"bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f\") " pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:22:38.312330 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:38.312332    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs\") pod \"network-metrics-daemon-rkln9\" (UID: \"0fca5e3d-e0e1-4b33-909c-46599a84cf4a\") " pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:22:38.315298 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:38.315280    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 05 17:22:38.315407 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:38.315389    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 05 17:22:38.323282 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:38.323257    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Aug 05 17:22:38.323364 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:38.323314    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs podName:0fca5e3d-e0e1-4b33-909c-46599a84cf4a nodeName:}" failed. No retries permitted until 2026-08-05 17:23:42.323297804 +0000 UTC m=+129.304875214 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs") pod "network-metrics-daemon-rkln9" (UID: "0fca5e3d-e0e1-4b33-909c-46599a84cf4a") : secret "metrics-daemon-secret" not found
Aug 05 17:22:38.326056 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:38.326034    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f-original-pull-secret\") pod \"global-pull-secret-syncer-h8vbl\" (UID: \"bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f\") " pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:22:38.411410 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:38.411374    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8vbl"
Aug 05 17:22:38.413694 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:38.413669    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwfdc\" (UniqueName: \"kubernetes.io/projected/f8972708-830a-4ebc-b521-8a16ca1aee39-kube-api-access-kwfdc\") pod \"network-check-target-4jfwd\" (UID: \"f8972708-830a-4ebc-b521-8a16ca1aee39\") " pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:22:38.416639 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:38.416622    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 05 17:22:38.426367 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:38.426342    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 05 17:22:38.436627 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:38.436603    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwfdc\" (UniqueName: \"kubernetes.io/projected/f8972708-830a-4ebc-b521-8a16ca1aee39-kube-api-access-kwfdc\") pod \"network-check-target-4jfwd\" (UID: \"f8972708-830a-4ebc-b521-8a16ca1aee39\") " pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:22:38.555940 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:38.555907    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h8vbl"]
Aug 05 17:22:38.558834 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:22:38.558810    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfbd82d0_51a1_4b7f_ba2c_19f17ad3b35f.slice/crio-856ee7453e9988215a8b14eb1e121ec9a3df758b7996b8b5c62677964f1069f5 WatchSource:0}: Error finding container 856ee7453e9988215a8b14eb1e121ec9a3df758b7996b8b5c62677964f1069f5: Status 404 returned error can't find the container with id 856ee7453e9988215a8b14eb1e121ec9a3df758b7996b8b5c62677964f1069f5
Aug 05 17:22:38.608257 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:38.608224    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8p6dg\""
Aug 05 17:22:38.615583 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:38.615563    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:22:38.759139 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:38.759107    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4jfwd"]
Aug 05 17:22:38.762744 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:22:38.762717    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8972708_830a_4ebc_b521_8a16ca1aee39.slice/crio-0cbe4b664e3ca5994c5f33552079fae2e1d1792466371496109517c3e54f039e WatchSource:0}: Error finding container 0cbe4b664e3ca5994c5f33552079fae2e1d1792466371496109517c3e54f039e: Status 404 returned error can't find the container with id 0cbe4b664e3ca5994c5f33552079fae2e1d1792466371496109517c3e54f039e
Aug 05 17:22:39.120951 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:39.120916    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h8vbl" event={"ID":"bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f","Type":"ContainerStarted","Data":"856ee7453e9988215a8b14eb1e121ec9a3df758b7996b8b5c62677964f1069f5"}
Aug 05 17:22:39.121944 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:39.121923    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4jfwd" event={"ID":"f8972708-830a-4ebc-b521-8a16ca1aee39","Type":"ContainerStarted","Data":"0cbe4b664e3ca5994c5f33552079fae2e1d1792466371496109517c3e54f039e"}
Aug 05 17:22:39.219622 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:39.219588    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:22:39.219823 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:39.219785    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 05 17:22:39.219823 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:39.219809    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d8d4888f6-kkxqq: secret "image-registry-tls" not found
Aug 05 17:22:39.219908 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:39.219872    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls podName:034231f3-7707-44c2-ad8b-63307a9c4e81 nodeName:}" failed. No retries permitted until 2026-08-05 17:23:11.219852655 +0000 UTC m=+98.201430061 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls") pod "image-registry-d8d4888f6-kkxqq" (UID: "034231f3-7707-44c2-ad8b-63307a9c4e81") : secret "image-registry-tls" not found
Aug 05 17:22:39.320543 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:39.320500    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/976b5f92-171d-43bc-9117-b2f42f830596-metrics-tls\") pod \"dns-default-6xcf5\" (UID: \"976b5f92-171d-43bc-9117-b2f42f830596\") " pod="openshift-dns/dns-default-6xcf5"
Aug 05 17:22:39.320958 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:39.320652    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 05 17:22:39.320958 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:22:39.320735    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/976b5f92-171d-43bc-9117-b2f42f830596-metrics-tls podName:976b5f92-171d-43bc-9117-b2f42f830596 nodeName:}" failed. No retries permitted until 2026-08-05 17:23:11.320716138 +0000 UTC m=+98.302293560 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/976b5f92-171d-43bc-9117-b2f42f830596-metrics-tls") pod "dns-default-6xcf5" (UID: "976b5f92-171d-43bc-9117-b2f42f830596") : secret "dns-default-metrics-tls" not found
Aug 05 17:22:44.140030 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:44.139992    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h8vbl" event={"ID":"bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f","Type":"ContainerStarted","Data":"bd44a024713e6751cd7db345a55f6c649ff35ceedfb8f2a359881ba1d40c0628"}
Aug 05 17:22:44.141353 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:44.141328    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4jfwd" event={"ID":"f8972708-830a-4ebc-b521-8a16ca1aee39","Type":"ContainerStarted","Data":"dfda46e15360e6372b5e588f38c91f92f85e358da4e2fae3d713468f83245493"}
Aug 05 17:22:44.141470 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:44.141447    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:22:44.155913 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:44.155870    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-h8vbl" podStartSLOduration=65.340490645 podStartE2EDuration="1m10.15585711s" totalImagesPullingTime="4.815366465s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:21:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:22:44.15562527 +0000 UTC m=+71.137202703" watchObservedRunningTime="2026-08-05 17:22:44.15585711 +0000 UTC m=+71.137434538"
Aug 05 17:22:44.170375 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:44.170303    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4jfwd" podStartSLOduration=65.564412248 podStartE2EDuration="1m10.170284732s" totalImagesPullingTime="4.605872484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:21:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:22:44.170185171 +0000 UTC m=+71.151762601" watchObservedRunningTime="2026-08-05 17:22:44.170284732 +0000 UTC m=+71.151862164"
Aug 05 17:22:46.225290 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:46.225256    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:46.325230 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:46.325187    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:46.390382 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:46.390351    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:47.237516 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:47.237482    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:47.305165 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:47.305125    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:49.182026 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:22:49.181989    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log" limit=1024
Aug 05 17:23:11.251191 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:11.251088    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:23:11.251590 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:11.251236    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 05 17:23:11.251590 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:11.251252    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d8d4888f6-kkxqq: secret "image-registry-tls" not found
Aug 05 17:23:11.251590 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:11.251307    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls podName:034231f3-7707-44c2-ad8b-63307a9c4e81 nodeName:}" failed. No retries permitted until 2026-08-05 17:24:15.251290811 +0000 UTC m=+162.232868218 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls") pod "image-registry-d8d4888f6-kkxqq" (UID: "034231f3-7707-44c2-ad8b-63307a9c4e81") : secret "image-registry-tls" not found
Aug 05 17:23:11.352348 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:11.352298    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/976b5f92-171d-43bc-9117-b2f42f830596-metrics-tls\") pod \"dns-default-6xcf5\" (UID: \"976b5f92-171d-43bc-9117-b2f42f830596\") " pod="openshift-dns/dns-default-6xcf5"
Aug 05 17:23:11.352521 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:11.352452    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 05 17:23:11.352562 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:11.352520    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/976b5f92-171d-43bc-9117-b2f42f830596-metrics-tls podName:976b5f92-171d-43bc-9117-b2f42f830596 nodeName:}" failed. No retries permitted until 2026-08-05 17:24:15.35250604 +0000 UTC m=+162.334083447 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/976b5f92-171d-43bc-9117-b2f42f830596-metrics-tls") pod "dns-default-6xcf5" (UID: "976b5f92-171d-43bc-9117-b2f42f830596") : secret "dns-default-metrics-tls" not found
Aug 05 17:23:15.147603 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:15.147570    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4jfwd"
Aug 05 17:23:36.408346 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:36.408316    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-6fqhl"]
Aug 05 17:23:36.411119 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:36.411102    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-6fqhl"
Aug 05 17:23:36.413325 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:36.413304    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-5wv9w\""
Aug 05 17:23:36.420409 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:36.420385    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-6fqhl"]
Aug 05 17:23:36.531271 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:36.531236    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt7lq\" (UniqueName: \"kubernetes.io/projected/71a4db96-3216-4365-9f12-5e4aeb8c45be-kube-api-access-xt7lq\") pod \"network-check-source-65bdbbff45-6fqhl\" (UID: \"71a4db96-3216-4365-9f12-5e4aeb8c45be\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-6fqhl"
Aug 05 17:23:36.632222 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:36.632193    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt7lq\" (UniqueName: \"kubernetes.io/projected/71a4db96-3216-4365-9f12-5e4aeb8c45be-kube-api-access-xt7lq\") pod \"network-check-source-65bdbbff45-6fqhl\" (UID: \"71a4db96-3216-4365-9f12-5e4aeb8c45be\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-6fqhl"
Aug 05 17:23:36.640894 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:36.640871    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt7lq\" (UniqueName: \"kubernetes.io/projected/71a4db96-3216-4365-9f12-5e4aeb8c45be-kube-api-access-xt7lq\") pod \"network-check-source-65bdbbff45-6fqhl\" (UID: \"71a4db96-3216-4365-9f12-5e4aeb8c45be\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-6fqhl"
Aug 05 17:23:36.719633 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:36.719547    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-6fqhl"
Aug 05 17:23:36.844833 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:36.844804    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-6fqhl"]
Aug 05 17:23:36.848114 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:23:36.848082    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71a4db96_3216_4365_9f12_5e4aeb8c45be.slice/crio-32b6a0e1e8d86843d4d64e64a826a14a081b0dfb46cc64b347d01d4eecad1996 WatchSource:0}: Error finding container 32b6a0e1e8d86843d4d64e64a826a14a081b0dfb46cc64b347d01d4eecad1996: Status 404 returned error can't find the container with id 32b6a0e1e8d86843d4d64e64a826a14a081b0dfb46cc64b347d01d4eecad1996
Aug 05 17:23:37.297992 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:37.297955    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-6fqhl" event={"ID":"71a4db96-3216-4365-9f12-5e4aeb8c45be","Type":"ContainerStarted","Data":"5d1faae002e9d97b5a1bc18647138adf466752df2c54bc5a1d85b992d6917599"}
Aug 05 17:23:37.298171 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:37.298005    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-6fqhl" event={"ID":"71a4db96-3216-4365-9f12-5e4aeb8c45be","Type":"ContainerStarted","Data":"32b6a0e1e8d86843d4d64e64a826a14a081b0dfb46cc64b347d01d4eecad1996"}
Aug 05 17:23:37.313630 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:37.313578    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-6fqhl" podStartSLOduration=1.313562315 podStartE2EDuration="1.313562315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:23:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:23:37.312880664 +0000 UTC m=+124.294458092" watchObservedRunningTime="2026-08-05 17:23:37.313562315 +0000 UTC m=+124.295139744"
Aug 05 17:23:39.509964 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:39.509934    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vgvsc"]
Aug 05 17:23:39.513071 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:39.513052    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vgvsc"
Aug 05 17:23:39.515630 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:39.515612    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Aug 05 17:23:39.515751 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:39.515646    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Aug 05 17:23:39.516733 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:39.516718    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vf4w2\""
Aug 05 17:23:39.516804 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:39.516748    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Aug 05 17:23:39.516804 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:39.516758    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Aug 05 17:23:39.524032 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:39.524011    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vgvsc"]
Aug 05 17:23:39.557163 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:39.554177    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/358a44e2-7356-4ac2-afee-afac2f16f107-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vgvsc\" (UID: \"358a44e2-7356-4ac2-afee-afac2f16f107\") " pod="openshift-insights/insights-runtime-extractor-vgvsc"
Aug 05 17:23:39.557163 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:39.554232    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/358a44e2-7356-4ac2-afee-afac2f16f107-crio-socket\") pod \"insights-runtime-extractor-vgvsc\" (UID: \"358a44e2-7356-4ac2-afee-afac2f16f107\") " pod="openshift-insights/insights-runtime-extractor-vgvsc"
Aug 05 17:23:39.557163 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:39.554283    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/358a44e2-7356-4ac2-afee-afac2f16f107-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vgvsc\" (UID: \"358a44e2-7356-4ac2-afee-afac2f16f107\") " pod="openshift-insights/insights-runtime-extractor-vgvsc"
Aug 05 17:23:39.557163 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:39.554321    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/358a44e2-7356-4ac2-afee-afac2f16f107-data-volume\") pod \"insights-runtime-extractor-vgvsc\" (UID: \"358a44e2-7356-4ac2-afee-afac2f16f107\") " pod="openshift-insights/insights-runtime-extractor-vgvsc"
Aug 05 17:23:39.557163 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:39.554417    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdm9v\" (UniqueName: \"kubernetes.io/projected/358a44e2-7356-4ac2-afee-afac2f16f107-kube-api-access-cdm9v\") pod \"insights-runtime-extractor-vgvsc\" (UID: \"358a44e2-7356-4ac2-afee-afac2f16f107\") " pod="openshift-insights/insights-runtime-extractor-vgvsc"
Aug 05 17:23:39.655475 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:39.655438    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/358a44e2-7356-4ac2-afee-afac2f16f107-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vgvsc\" (UID: \"358a44e2-7356-4ac2-afee-afac2f16f107\") " pod="openshift-insights/insights-runtime-extractor-vgvsc"
Aug 05 17:23:39.655475 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:39.655483    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/358a44e2-7356-4ac2-afee-afac2f16f107-crio-socket\") pod \"insights-runtime-extractor-vgvsc\" (UID: \"358a44e2-7356-4ac2-afee-afac2f16f107\") " pod="openshift-insights/insights-runtime-extractor-vgvsc"
Aug 05 17:23:39.655752 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:39.655518    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/358a44e2-7356-4ac2-afee-afac2f16f107-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vgvsc\" (UID: \"358a44e2-7356-4ac2-afee-afac2f16f107\") " pod="openshift-insights/insights-runtime-extractor-vgvsc"
Aug 05 17:23:39.655752 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:39.655546    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/358a44e2-7356-4ac2-afee-afac2f16f107-data-volume\") pod \"insights-runtime-extractor-vgvsc\" (UID: \"358a44e2-7356-4ac2-afee-afac2f16f107\") " pod="openshift-insights/insights-runtime-extractor-vgvsc"
Aug 05 17:23:39.655752 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:39.655585    2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 05 17:23:39.655752 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:39.655607    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdm9v\" (UniqueName: \"kubernetes.io/projected/358a44e2-7356-4ac2-afee-afac2f16f107-kube-api-access-cdm9v\") pod \"insights-runtime-extractor-vgvsc\" (UID: \"358a44e2-7356-4ac2-afee-afac2f16f107\") " pod="openshift-insights/insights-runtime-extractor-vgvsc"
Aug 05 17:23:39.655752 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:39.655587    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/358a44e2-7356-4ac2-afee-afac2f16f107-crio-socket\") pod \"insights-runtime-extractor-vgvsc\" (UID: \"358a44e2-7356-4ac2-afee-afac2f16f107\") " pod="openshift-insights/insights-runtime-extractor-vgvsc"
Aug 05 17:23:39.655752 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:39.655671    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/358a44e2-7356-4ac2-afee-afac2f16f107-kube-rbac-proxy-cm podName:358a44e2-7356-4ac2-afee-afac2f16f107 nodeName:}" failed. No retries permitted until 2026-08-05 17:23:40.155649105 +0000 UTC m=+127.137226529 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/358a44e2-7356-4ac2-afee-afac2f16f107-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vgvsc" (UID: "358a44e2-7356-4ac2-afee-afac2f16f107") : configmap "kube-rbac-proxy" not found
Aug 05 17:23:39.655752 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:39.655686    2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 05 17:23:39.655752 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:39.655757    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/358a44e2-7356-4ac2-afee-afac2f16f107-insights-runtime-extractor-tls podName:358a44e2-7356-4ac2-afee-afac2f16f107 nodeName:}" failed. No retries permitted until 2026-08-05 17:23:40.155740649 +0000 UTC m=+127.137318059 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/358a44e2-7356-4ac2-afee-afac2f16f107-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vgvsc" (UID: "358a44e2-7356-4ac2-afee-afac2f16f107") : secret "insights-runtime-extractor-tls" not found
Aug 05 17:23:39.656102 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:39.655940    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/358a44e2-7356-4ac2-afee-afac2f16f107-data-volume\") pod \"insights-runtime-extractor-vgvsc\" (UID: \"358a44e2-7356-4ac2-afee-afac2f16f107\") " pod="openshift-insights/insights-runtime-extractor-vgvsc"
Aug 05 17:23:39.666011 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:39.665987    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdm9v\" (UniqueName: \"kubernetes.io/projected/358a44e2-7356-4ac2-afee-afac2f16f107-kube-api-access-cdm9v\") pod \"insights-runtime-extractor-vgvsc\" (UID: \"358a44e2-7356-4ac2-afee-afac2f16f107\") " pod="openshift-insights/insights-runtime-extractor-vgvsc"
Aug 05 17:23:40.159503 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:40.159461    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/358a44e2-7356-4ac2-afee-afac2f16f107-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vgvsc\" (UID: \"358a44e2-7356-4ac2-afee-afac2f16f107\") " pod="openshift-insights/insights-runtime-extractor-vgvsc"
Aug 05 17:23:40.159679 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:40.159514    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/358a44e2-7356-4ac2-afee-afac2f16f107-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vgvsc\" (UID: \"358a44e2-7356-4ac2-afee-afac2f16f107\") " pod="openshift-insights/insights-runtime-extractor-vgvsc"
Aug 05 17:23:40.159679 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:40.159580    2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 05 17:23:40.159679 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:40.159636    2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 05 17:23:40.159679 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:40.159641    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/358a44e2-7356-4ac2-afee-afac2f16f107-kube-rbac-proxy-cm podName:358a44e2-7356-4ac2-afee-afac2f16f107 nodeName:}" failed. No retries permitted until 2026-08-05 17:23:41.159626782 +0000 UTC m=+128.141204188 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/358a44e2-7356-4ac2-afee-afac2f16f107-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vgvsc" (UID: "358a44e2-7356-4ac2-afee-afac2f16f107") : configmap "kube-rbac-proxy" not found
Aug 05 17:23:40.159853 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:40.159692    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/358a44e2-7356-4ac2-afee-afac2f16f107-insights-runtime-extractor-tls podName:358a44e2-7356-4ac2-afee-afac2f16f107 nodeName:}" failed. No retries permitted until 2026-08-05 17:23:41.159680469 +0000 UTC m=+128.141257881 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/358a44e2-7356-4ac2-afee-afac2f16f107-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vgvsc" (UID: "358a44e2-7356-4ac2-afee-afac2f16f107") : secret "insights-runtime-extractor-tls" not found
Aug 05 17:23:41.168880 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:41.168841    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/358a44e2-7356-4ac2-afee-afac2f16f107-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vgvsc\" (UID: \"358a44e2-7356-4ac2-afee-afac2f16f107\") " pod="openshift-insights/insights-runtime-extractor-vgvsc"
Aug 05 17:23:41.169462 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:41.168960    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/358a44e2-7356-4ac2-afee-afac2f16f107-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vgvsc\" (UID: \"358a44e2-7356-4ac2-afee-afac2f16f107\") " pod="openshift-insights/insights-runtime-extractor-vgvsc"
Aug 05 17:23:41.169462 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:41.169008    2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 05 17:23:41.169462 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:41.169033    2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 05 17:23:41.169462 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:41.169084    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/358a44e2-7356-4ac2-afee-afac2f16f107-insights-runtime-extractor-tls podName:358a44e2-7356-4ac2-afee-afac2f16f107 nodeName:}" failed. No retries permitted until 2026-08-05 17:23:43.169062035 +0000 UTC m=+130.150639453 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/358a44e2-7356-4ac2-afee-afac2f16f107-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vgvsc" (UID: "358a44e2-7356-4ac2-afee-afac2f16f107") : secret "insights-runtime-extractor-tls" not found
Aug 05 17:23:41.169462 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:41.169103    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/358a44e2-7356-4ac2-afee-afac2f16f107-kube-rbac-proxy-cm podName:358a44e2-7356-4ac2-afee-afac2f16f107 nodeName:}" failed. No retries permitted until 2026-08-05 17:23:43.169094193 +0000 UTC m=+130.150671602 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/358a44e2-7356-4ac2-afee-afac2f16f107-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vgvsc" (UID: "358a44e2-7356-4ac2-afee-afac2f16f107") : configmap "kube-rbac-proxy" not found
Aug 05 17:23:41.274876 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:41.274844    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2f92r_b382a0b8-435d-430a-a8a2-faa3d0b3bdc1/dns-node-resolver/0.log"
Aug 05 17:23:42.275235 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:42.275207    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bpcrr_610a012b-bd9a-4627-8976-b1f785945d61/node-ca/0.log"
Aug 05 17:23:42.378929 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:42.378878    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs\") pod \"network-metrics-daemon-rkln9\" (UID: \"0fca5e3d-e0e1-4b33-909c-46599a84cf4a\") " pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:23:42.379103 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:42.379026    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Aug 05 17:23:42.379103 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:42.379087    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs podName:0fca5e3d-e0e1-4b33-909c-46599a84cf4a nodeName:}" failed. No retries permitted until 2026-08-05 17:25:44.379072974 +0000 UTC m=+251.360650381 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs") pod "network-metrics-daemon-rkln9" (UID: "0fca5e3d-e0e1-4b33-909c-46599a84cf4a") : secret "metrics-daemon-secret" not found
Aug 05 17:23:43.185124 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:43.185087    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/358a44e2-7356-4ac2-afee-afac2f16f107-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vgvsc\" (UID: \"358a44e2-7356-4ac2-afee-afac2f16f107\") " pod="openshift-insights/insights-runtime-extractor-vgvsc"
Aug 05 17:23:43.185124 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:43.185131    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/358a44e2-7356-4ac2-afee-afac2f16f107-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vgvsc\" (UID: \"358a44e2-7356-4ac2-afee-afac2f16f107\") " pod="openshift-insights/insights-runtime-extractor-vgvsc"
Aug 05 17:23:43.185332 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:43.185211    2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 05 17:23:43.185332 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:43.185277    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/358a44e2-7356-4ac2-afee-afac2f16f107-kube-rbac-proxy-cm podName:358a44e2-7356-4ac2-afee-afac2f16f107 nodeName:}" failed. No retries permitted until 2026-08-05 17:23:47.185261437 +0000 UTC m=+134.166838844 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/358a44e2-7356-4ac2-afee-afac2f16f107-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vgvsc" (UID: "358a44e2-7356-4ac2-afee-afac2f16f107") : configmap "kube-rbac-proxy" not found
Aug 05 17:23:43.185332 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:43.185306    2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 05 17:23:43.185430 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:43.185361    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/358a44e2-7356-4ac2-afee-afac2f16f107-insights-runtime-extractor-tls podName:358a44e2-7356-4ac2-afee-afac2f16f107 nodeName:}" failed. No retries permitted until 2026-08-05 17:23:47.185349396 +0000 UTC m=+134.166926803 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/358a44e2-7356-4ac2-afee-afac2f16f107-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vgvsc" (UID: "358a44e2-7356-4ac2-afee-afac2f16f107") : secret "insights-runtime-extractor-tls" not found
Aug 05 17:23:43.274607 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:43.274570    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gxsh5_8bdd9d59-2279-49b8-882d-210c0d146301/serve-healthcheck-canary/0.log"
Aug 05 17:23:47.216529 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:47.216487    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/358a44e2-7356-4ac2-afee-afac2f16f107-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vgvsc\" (UID: \"358a44e2-7356-4ac2-afee-afac2f16f107\") " pod="openshift-insights/insights-runtime-extractor-vgvsc"
Aug 05 17:23:47.216958 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:47.216541    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/358a44e2-7356-4ac2-afee-afac2f16f107-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vgvsc\" (UID: \"358a44e2-7356-4ac2-afee-afac2f16f107\") " pod="openshift-insights/insights-runtime-extractor-vgvsc"
Aug 05 17:23:47.216958 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:47.216605    2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 05 17:23:47.216958 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:47.216660    2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 05 17:23:47.216958 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:47.216669    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/358a44e2-7356-4ac2-afee-afac2f16f107-kube-rbac-proxy-cm podName:358a44e2-7356-4ac2-afee-afac2f16f107 nodeName:}" failed. No retries permitted until 2026-08-05 17:23:55.216652601 +0000 UTC m=+142.198230008 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/358a44e2-7356-4ac2-afee-afac2f16f107-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vgvsc" (UID: "358a44e2-7356-4ac2-afee-afac2f16f107") : configmap "kube-rbac-proxy" not found
Aug 05 17:23:47.216958 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:47.216731    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/358a44e2-7356-4ac2-afee-afac2f16f107-insights-runtime-extractor-tls podName:358a44e2-7356-4ac2-afee-afac2f16f107 nodeName:}" failed. No retries permitted until 2026-08-05 17:23:55.216716704 +0000 UTC m=+142.198294124 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/358a44e2-7356-4ac2-afee-afac2f16f107-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vgvsc" (UID: "358a44e2-7356-4ac2-afee-afac2f16f107") : secret "insights-runtime-extractor-tls" not found
Aug 05 17:23:55.281695 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:55.281649    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/358a44e2-7356-4ac2-afee-afac2f16f107-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vgvsc\" (UID: \"358a44e2-7356-4ac2-afee-afac2f16f107\") " pod="openshift-insights/insights-runtime-extractor-vgvsc"
Aug 05 17:23:55.282105 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:55.281800    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/358a44e2-7356-4ac2-afee-afac2f16f107-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vgvsc\" (UID: \"358a44e2-7356-4ac2-afee-afac2f16f107\") " pod="openshift-insights/insights-runtime-extractor-vgvsc"
Aug 05 17:23:55.282105 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:55.281966    2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 05 17:23:55.282105 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:23:55.282062    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/358a44e2-7356-4ac2-afee-afac2f16f107-insights-runtime-extractor-tls podName:358a44e2-7356-4ac2-afee-afac2f16f107 nodeName:}" failed. No retries permitted until 2026-08-05 17:24:11.282038986 +0000 UTC m=+158.263616405 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/358a44e2-7356-4ac2-afee-afac2f16f107-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vgvsc" (UID: "358a44e2-7356-4ac2-afee-afac2f16f107") : secret "insights-runtime-extractor-tls" not found
Aug 05 17:23:55.282294 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:55.282271    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/358a44e2-7356-4ac2-afee-afac2f16f107-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vgvsc\" (UID: \"358a44e2-7356-4ac2-afee-afac2f16f107\") " pod="openshift-insights/insights-runtime-extractor-vgvsc"
Aug 05 17:23:57.720102 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:23:57.720056    2550 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb" podUID="0cbcfe8b-7d26-444e-9338-c944b590ed0f" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Aug 05 17:24:07.719884 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:07.719846    2550 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb" podUID="0cbcfe8b-7d26-444e-9338-c944b590ed0f" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Aug 05 17:24:10.384581 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:24:10.384540    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq" podUID="034231f3-7707-44c2-ad8b-63307a9c4e81"
Aug 05 17:24:10.393722 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:10.393687    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:24:10.447536 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:24:10.447490    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-6xcf5" podUID="976b5f92-171d-43bc-9117-b2f42f830596"
Aug 05 17:24:11.299760 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:11.299671    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/358a44e2-7356-4ac2-afee-afac2f16f107-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vgvsc\" (UID: \"358a44e2-7356-4ac2-afee-afac2f16f107\") " pod="openshift-insights/insights-runtime-extractor-vgvsc"
Aug 05 17:24:11.301918 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:11.301898    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/358a44e2-7356-4ac2-afee-afac2f16f107-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vgvsc\" (UID: \"358a44e2-7356-4ac2-afee-afac2f16f107\") " pod="openshift-insights/insights-runtime-extractor-vgvsc"
Aug 05 17:24:11.321883 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:11.321859    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vgvsc"
Aug 05 17:24:11.396178 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:11.396153    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6xcf5"
Aug 05 17:24:11.448438 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:11.448407    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vgvsc"]
Aug 05 17:24:11.451658 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:24:11.451638    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod358a44e2_7356_4ac2_afee_afac2f16f107.slice/crio-9c5e58db771f72c045061c040285de22508d627001b6e7969b54402301cc4be7 WatchSource:0}: Error finding container 9c5e58db771f72c045061c040285de22508d627001b6e7969b54402301cc4be7: Status 404 returned error can't find the container with id 9c5e58db771f72c045061c040285de22508d627001b6e7969b54402301cc4be7
Aug 05 17:24:11.723998 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:24:11.723967    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-rkln9" podUID="0fca5e3d-e0e1-4b33-909c-46599a84cf4a"
Aug 05 17:24:12.400032 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:12.400005    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vgvsc" event={"ID":"358a44e2-7356-4ac2-afee-afac2f16f107","Type":"ContainerStarted","Data":"aa0fc76397be8db8058c4c75758a2f370c547238fe1c8b5125cc32d13ffd4822"}
Aug 05 17:24:12.400351 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:12.400066    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vgvsc" event={"ID":"358a44e2-7356-4ac2-afee-afac2f16f107","Type":"ContainerStarted","Data":"5016ef56dca47d1a2bd048992d1c830a6e5739db522e063ace84cd1f40c7a8c6"}
Aug 05 17:24:12.400351 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:12.400100    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vgvsc" event={"ID":"358a44e2-7356-4ac2-afee-afac2f16f107","Type":"ContainerStarted","Data":"9c5e58db771f72c045061c040285de22508d627001b6e7969b54402301cc4be7"}
Aug 05 17:24:14.407770 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:14.407732    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vgvsc" event={"ID":"358a44e2-7356-4ac2-afee-afac2f16f107","Type":"ContainerStarted","Data":"1803dccfabd6905557bce0752324bffa152b9945ff7006ca1583104479e5175a"}
Aug 05 17:24:14.438885 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:14.438838    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vgvsc" podStartSLOduration=33.373988535 podStartE2EDuration="35.438823428s" totalImagesPullingTime="2.064834893s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:23:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:24:14.438602417 +0000 UTC m=+161.420179846" watchObservedRunningTime="2026-08-05 17:24:14.438823428 +0000 UTC m=+161.420400857"
Aug 05 17:24:15.328651 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:15.328614    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:24:15.331022 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:15.330989    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls\") pod \"image-registry-d8d4888f6-kkxqq\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") " pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:24:15.429925 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:15.429895    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/976b5f92-171d-43bc-9117-b2f42f830596-metrics-tls\") pod \"dns-default-6xcf5\" (UID: \"976b5f92-171d-43bc-9117-b2f42f830596\") " pod="openshift-dns/dns-default-6xcf5"
Aug 05 17:24:15.432008 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:15.431990    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/976b5f92-171d-43bc-9117-b2f42f830596-metrics-tls\") pod \"dns-default-6xcf5\" (UID: \"976b5f92-171d-43bc-9117-b2f42f830596\") " pod="openshift-dns/dns-default-6xcf5"
Aug 05 17:24:15.496499 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:15.496465    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4cfrs\""
Aug 05 17:24:15.504607 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:15.504580    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:24:15.599760 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:15.599731    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hv8kx\""
Aug 05 17:24:15.608151 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:15.608129    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6xcf5"
Aug 05 17:24:15.643945 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:15.643063    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d8d4888f6-kkxqq"]
Aug 05 17:24:15.646291 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:24:15.646265    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod034231f3_7707_44c2_ad8b_63307a9c4e81.slice/crio-6b6a4e93f60d56686dc3eab2b2f82ddea811ccf2cf039064411576f379a422f0 WatchSource:0}: Error finding container 6b6a4e93f60d56686dc3eab2b2f82ddea811ccf2cf039064411576f379a422f0: Status 404 returned error can't find the container with id 6b6a4e93f60d56686dc3eab2b2f82ddea811ccf2cf039064411576f379a422f0
Aug 05 17:24:15.739803 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:15.739773    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6xcf5"]
Aug 05 17:24:15.742426 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:24:15.742396    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod976b5f92_171d_43bc_9117_b2f42f830596.slice/crio-8e92d572be0c6fcb6a012644bbc51b84bf067bfb06ab737fdd115f17508a38d8 WatchSource:0}: Error finding container 8e92d572be0c6fcb6a012644bbc51b84bf067bfb06ab737fdd115f17508a38d8: Status 404 returned error can't find the container with id 8e92d572be0c6fcb6a012644bbc51b84bf067bfb06ab737fdd115f17508a38d8
Aug 05 17:24:16.414547 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:16.414509    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6xcf5" event={"ID":"976b5f92-171d-43bc-9117-b2f42f830596","Type":"ContainerStarted","Data":"8e92d572be0c6fcb6a012644bbc51b84bf067bfb06ab737fdd115f17508a38d8"}
Aug 05 17:24:16.416167 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:16.416128    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq" event={"ID":"034231f3-7707-44c2-ad8b-63307a9c4e81","Type":"ContainerStarted","Data":"ac4c25911cdb65f538db5c19a9975fed3eda8c6295e207a5b50aa0192b1f6d0e"}
Aug 05 17:24:16.416298 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:16.416177    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq" event={"ID":"034231f3-7707-44c2-ad8b-63307a9c4e81","Type":"ContainerStarted","Data":"6b6a4e93f60d56686dc3eab2b2f82ddea811ccf2cf039064411576f379a422f0"}
Aug 05 17:24:16.416298 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:16.416214    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:24:16.439826 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:16.439776    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq" podStartSLOduration=162.439758243 podStartE2EDuration="2m42.439758243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:21:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:24:16.438480933 +0000 UTC m=+163.420058361" watchObservedRunningTime="2026-08-05 17:24:16.439758243 +0000 UTC m=+163.421335675"
Aug 05 17:24:17.420955 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:17.420923    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6xcf5" event={"ID":"976b5f92-171d-43bc-9117-b2f42f830596","Type":"ContainerStarted","Data":"6c913ca1a9b71606ff3fed6a78325a6a5ca1ce572e1f9c7e2d64baf1684bef54"}
Aug 05 17:24:17.421129 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:17.420980    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6xcf5" event={"ID":"976b5f92-171d-43bc-9117-b2f42f830596","Type":"ContainerStarted","Data":"ec85842c088120cb785217eb5e4fc9c098e3802b1f86aa11ded54c30a65cb9b7"}
Aug 05 17:24:17.443467 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:17.443425    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6xcf5" podStartSLOduration=129.075209591 podStartE2EDuration="2m10.443409307s" totalImagesPullingTime="1.368199716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:22:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:24:17.443261314 +0000 UTC m=+164.424838742" watchObservedRunningTime="2026-08-05 17:24:17.443409307 +0000 UTC m=+164.424986736"
Aug 05 17:24:17.720044 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:17.719966    2550 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb" podUID="0cbcfe8b-7d26-444e-9338-c944b590ed0f" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Aug 05 17:24:17.720044 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:17.720034    2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb"
Aug 05 17:24:17.720502 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:17.720476    2550 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"1befae7d1e96735589e458df0aa14c17d2102a74c60341ab3f13b3bd6f21392a"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Aug 05 17:24:17.720542 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:17.720530    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb" podUID="0cbcfe8b-7d26-444e-9338-c944b590ed0f" containerName="service-proxy" containerID="cri-o://1befae7d1e96735589e458df0aa14c17d2102a74c60341ab3f13b3bd6f21392a" gracePeriod=30
Aug 05 17:24:18.051364 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.051324    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-576bd4884b-5qmch" podUID="0f396771-0bb3-4221-a787-c8cc1718032c" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.9:8000/readyz\": dial tcp 10.132.0.9:8000: connect: connection refused"
Aug 05 17:24:18.425435 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.425405    2550 generic.go:356] "Generic (PLEG): container finished" podID="0f396771-0bb3-4221-a787-c8cc1718032c" containerID="265aa3850aaa108d00553d333dd63c158e200687fcee0fcdbb2051f349c2a042" exitCode=1
Aug 05 17:24:18.425589 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.425481    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-576bd4884b-5qmch" event={"ID":"0f396771-0bb3-4221-a787-c8cc1718032c","Type":"ContainerDied","Data":"265aa3850aaa108d00553d333dd63c158e200687fcee0fcdbb2051f349c2a042"}
Aug 05 17:24:18.425866 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.425848    2550 scope.go:117] "RemoveContainer" containerID="265aa3850aaa108d00553d333dd63c158e200687fcee0fcdbb2051f349c2a042"
Aug 05 17:24:18.427840 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.427817    2550 generic.go:356] "Generic (PLEG): container finished" podID="0cbcfe8b-7d26-444e-9338-c944b590ed0f" containerID="1befae7d1e96735589e458df0aa14c17d2102a74c60341ab3f13b3bd6f21392a" exitCode=2
Aug 05 17:24:18.427944 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.427841    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb" event={"ID":"0cbcfe8b-7d26-444e-9338-c944b590ed0f","Type":"ContainerDied","Data":"1befae7d1e96735589e458df0aa14c17d2102a74c60341ab3f13b3bd6f21392a"}
Aug 05 17:24:18.427944 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.427893    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8655fc97b8-gkddb" event={"ID":"0cbcfe8b-7d26-444e-9338-c944b590ed0f","Type":"ContainerStarted","Data":"bc278881bdd281d662497c87bf429556cd73438e428909b0c50345680bbf2959"}
Aug 05 17:24:18.429340 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.429317    2550 generic.go:356] "Generic (PLEG): container finished" podID="7dd286cb-de80-463c-8036-217b255c3518" containerID="90cfaf0dc67aafd265c57096d850d4d587ac18560c3b39ac0f6f3cf8f2deefb9" exitCode=255
Aug 05 17:24:18.429435 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.429388    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96949699d-rvdxv" event={"ID":"7dd286cb-de80-463c-8036-217b255c3518","Type":"ContainerDied","Data":"90cfaf0dc67aafd265c57096d850d4d587ac18560c3b39ac0f6f3cf8f2deefb9"}
Aug 05 17:24:18.429527 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.429512    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6xcf5"
Aug 05 17:24:18.429751 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.429737    2550 scope.go:117] "RemoveContainer" containerID="90cfaf0dc67aafd265c57096d850d4d587ac18560c3b39ac0f6f3cf8f2deefb9"
Aug 05 17:24:18.487224 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.487199    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2m9lf"]
Aug 05 17:24:18.490251 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.490233    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.498475 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.498454    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Aug 05 17:24:18.499550 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.499108    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Aug 05 17:24:18.499550 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.499508    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Aug 05 17:24:18.499792 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.499761    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Aug 05 17:24:18.499946 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.499895    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Aug 05 17:24:18.500053 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.500022    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Aug 05 17:24:18.500053 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.500043    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dw7jg\""
Aug 05 17:24:18.554282 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.554251    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ece264d-044c-441c-8db0-c84744efa149-metrics-client-ca\") pod \"node-exporter-2m9lf\" (UID: \"0ece264d-044c-441c-8db0-c84744efa149\") " pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.554417 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.554315    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0ece264d-044c-441c-8db0-c84744efa149-root\") pod \"node-exporter-2m9lf\" (UID: \"0ece264d-044c-441c-8db0-c84744efa149\") " pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.554578 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.554556    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ece264d-044c-441c-8db0-c84744efa149-sys\") pod \"node-exporter-2m9lf\" (UID: \"0ece264d-044c-441c-8db0-c84744efa149\") " pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.554615 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.554597    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0ece264d-044c-441c-8db0-c84744efa149-node-exporter-wtmp\") pod \"node-exporter-2m9lf\" (UID: \"0ece264d-044c-441c-8db0-c84744efa149\") " pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.554649 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.554626    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0ece264d-044c-441c-8db0-c84744efa149-node-exporter-tls\") pod \"node-exporter-2m9lf\" (UID: \"0ece264d-044c-441c-8db0-c84744efa149\") " pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.554770 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.554752    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0ece264d-044c-441c-8db0-c84744efa149-node-exporter-textfile\") pod \"node-exporter-2m9lf\" (UID: \"0ece264d-044c-441c-8db0-c84744efa149\") " pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.554846 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.554828    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbrsk\" (UniqueName: \"kubernetes.io/projected/0ece264d-044c-441c-8db0-c84744efa149-kube-api-access-zbrsk\") pod \"node-exporter-2m9lf\" (UID: \"0ece264d-044c-441c-8db0-c84744efa149\") " pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.554968 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.554947    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0ece264d-044c-441c-8db0-c84744efa149-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2m9lf\" (UID: \"0ece264d-044c-441c-8db0-c84744efa149\") " pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.555176 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.554972    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0ece264d-044c-441c-8db0-c84744efa149-node-exporter-accelerators-collector-config\") pod \"node-exporter-2m9lf\" (UID: \"0ece264d-044c-441c-8db0-c84744efa149\") " pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.656359 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.656328    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ece264d-044c-441c-8db0-c84744efa149-metrics-client-ca\") pod \"node-exporter-2m9lf\" (UID: \"0ece264d-044c-441c-8db0-c84744efa149\") " pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.656524 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.656367    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0ece264d-044c-441c-8db0-c84744efa149-root\") pod \"node-exporter-2m9lf\" (UID: \"0ece264d-044c-441c-8db0-c84744efa149\") " pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.656524 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.656396    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ece264d-044c-441c-8db0-c84744efa149-sys\") pod \"node-exporter-2m9lf\" (UID: \"0ece264d-044c-441c-8db0-c84744efa149\") " pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.656524 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.656413    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0ece264d-044c-441c-8db0-c84744efa149-node-exporter-wtmp\") pod \"node-exporter-2m9lf\" (UID: \"0ece264d-044c-441c-8db0-c84744efa149\") " pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.656524 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.656437    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0ece264d-044c-441c-8db0-c84744efa149-node-exporter-tls\") pod \"node-exporter-2m9lf\" (UID: \"0ece264d-044c-441c-8db0-c84744efa149\") " pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.656524 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.656453    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0ece264d-044c-441c-8db0-c84744efa149-root\") pod \"node-exporter-2m9lf\" (UID: \"0ece264d-044c-441c-8db0-c84744efa149\") " pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.656524 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.656464    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ece264d-044c-441c-8db0-c84744efa149-sys\") pod \"node-exporter-2m9lf\" (UID: \"0ece264d-044c-441c-8db0-c84744efa149\") " pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.656524 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.656479    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0ece264d-044c-441c-8db0-c84744efa149-node-exporter-textfile\") pod \"node-exporter-2m9lf\" (UID: \"0ece264d-044c-441c-8db0-c84744efa149\") " pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.656870 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.656556    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbrsk\" (UniqueName: \"kubernetes.io/projected/0ece264d-044c-441c-8db0-c84744efa149-kube-api-access-zbrsk\") pod \"node-exporter-2m9lf\" (UID: \"0ece264d-044c-441c-8db0-c84744efa149\") " pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.656870 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.656567    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0ece264d-044c-441c-8db0-c84744efa149-node-exporter-wtmp\") pod \"node-exporter-2m9lf\" (UID: \"0ece264d-044c-441c-8db0-c84744efa149\") " pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.656870 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.656584    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0ece264d-044c-441c-8db0-c84744efa149-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2m9lf\" (UID: \"0ece264d-044c-441c-8db0-c84744efa149\") " pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.656870 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.656606    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0ece264d-044c-441c-8db0-c84744efa149-node-exporter-accelerators-collector-config\") pod \"node-exporter-2m9lf\" (UID: \"0ece264d-044c-441c-8db0-c84744efa149\") " pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.657061 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.656910    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0ece264d-044c-441c-8db0-c84744efa149-node-exporter-textfile\") pod \"node-exporter-2m9lf\" (UID: \"0ece264d-044c-441c-8db0-c84744efa149\") " pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.657061 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.656977    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ece264d-044c-441c-8db0-c84744efa149-metrics-client-ca\") pod \"node-exporter-2m9lf\" (UID: \"0ece264d-044c-441c-8db0-c84744efa149\") " pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.657149 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.657132    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0ece264d-044c-441c-8db0-c84744efa149-node-exporter-accelerators-collector-config\") pod \"node-exporter-2m9lf\" (UID: \"0ece264d-044c-441c-8db0-c84744efa149\") " pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.658835 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.658816    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0ece264d-044c-441c-8db0-c84744efa149-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2m9lf\" (UID: \"0ece264d-044c-441c-8db0-c84744efa149\") " pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.658936 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.658903    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0ece264d-044c-441c-8db0-c84744efa149-node-exporter-tls\") pod \"node-exporter-2m9lf\" (UID: \"0ece264d-044c-441c-8db0-c84744efa149\") " pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.669530 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.669510    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbrsk\" (UniqueName: \"kubernetes.io/projected/0ece264d-044c-441c-8db0-c84744efa149-kube-api-access-zbrsk\") pod \"node-exporter-2m9lf\" (UID: \"0ece264d-044c-441c-8db0-c84744efa149\") " pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.800931 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:18.800866    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2m9lf"
Aug 05 17:24:18.811832 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:24:18.811810    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ece264d_044c_441c_8db0_c84744efa149.slice/crio-4d1cd00d85518059470532c504457f962846d02c9892bac3cb58d6e51b52f804 WatchSource:0}: Error finding container 4d1cd00d85518059470532c504457f962846d02c9892bac3cb58d6e51b52f804: Status 404 returned error can't find the container with id 4d1cd00d85518059470532c504457f962846d02c9892bac3cb58d6e51b52f804
Aug 05 17:24:19.435247 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:19.435207    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96949699d-rvdxv" event={"ID":"7dd286cb-de80-463c-8036-217b255c3518","Type":"ContainerStarted","Data":"23ac2a5dd7e960177dd75e6acdef26edb2bf1806e262f9690fa0d3ace4ff1dba"}
Aug 05 17:24:19.436782 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:19.436760    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2m9lf" event={"ID":"0ece264d-044c-441c-8db0-c84744efa149","Type":"ContainerStarted","Data":"4d1cd00d85518059470532c504457f962846d02c9892bac3cb58d6e51b52f804"}
Aug 05 17:24:19.438743 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:19.438721    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-576bd4884b-5qmch" event={"ID":"0f396771-0bb3-4221-a787-c8cc1718032c","Type":"ContainerStarted","Data":"237ddf28b108725d4e27a03742fbf9d8eb7bf202dc47ecb8f07121876ebdcc39"}
Aug 05 17:24:19.439162 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:19.439140    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-576bd4884b-5qmch"
Aug 05 17:24:19.439669 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:19.439652    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-576bd4884b-5qmch"
Aug 05 17:24:20.443386 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:20.443349    2550 generic.go:356] "Generic (PLEG): container finished" podID="0ece264d-044c-441c-8db0-c84744efa149" containerID="3844d40d51c4f32ce61f3ecaf9d999a8da19aef885a00d75c20d09972d58cfad" exitCode=0
Aug 05 17:24:20.443866 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:20.443437    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2m9lf" event={"ID":"0ece264d-044c-441c-8db0-c84744efa149","Type":"ContainerDied","Data":"3844d40d51c4f32ce61f3ecaf9d999a8da19aef885a00d75c20d09972d58cfad"}
Aug 05 17:24:21.448842 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:21.448801    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2m9lf" event={"ID":"0ece264d-044c-441c-8db0-c84744efa149","Type":"ContainerStarted","Data":"8e0f9482132b24edbbc81fefc3d9b2ab66eb8138a679e36ee70f3276cbf372eb"}
Aug 05 17:24:21.449290 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:21.448855    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2m9lf" event={"ID":"0ece264d-044c-441c-8db0-c84744efa149","Type":"ContainerStarted","Data":"ef957ce6f5be59f91aa5b8b145ab71a53ef418d1a73bd88542bd89707c8bcf20"}
Aug 05 17:24:21.473256 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:21.473212    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2m9lf" podStartSLOduration=2.445059582 podStartE2EDuration="3.473196473s" totalImagesPullingTime="965.482614ms" totalInitContainerRuntime="62.654277ms" isStatefulPod=true podCreationTimestamp="2026-08-05 17:24:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:24:21.471953735 +0000 UTC m=+168.453531181" watchObservedRunningTime="2026-08-05 17:24:21.473196473 +0000 UTC m=+168.454773902"
Aug 05 17:24:22.694686 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:22.694640    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:24:24.477488 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:24.477455    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d8d4888f6-kkxqq"]
Aug 05 17:24:28.441670 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:28.441597    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6xcf5"
Aug 05 17:24:34.483958 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:34.483930    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:24:49.495597 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:49.495548    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq" podUID="034231f3-7707-44c2-ad8b-63307a9c4e81" containerName="registry" containerID="cri-o://ac4c25911cdb65f538db5c19a9975fed3eda8c6295e207a5b50aa0192b1f6d0e" gracePeriod=30
Aug 05 17:24:49.750110 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:49.750053    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:24:49.904306 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:49.904269    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/034231f3-7707-44c2-ad8b-63307a9c4e81-ca-trust-extracted\") pod \"034231f3-7707-44c2-ad8b-63307a9c4e81\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") "
Aug 05 17:24:49.904306 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:49.904314    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/034231f3-7707-44c2-ad8b-63307a9c4e81-trusted-ca\") pod \"034231f3-7707-44c2-ad8b-63307a9c4e81\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") "
Aug 05 17:24:49.904550 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:49.904351    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls\") pod \"034231f3-7707-44c2-ad8b-63307a9c4e81\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") "
Aug 05 17:24:49.904550 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:49.904370    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/034231f3-7707-44c2-ad8b-63307a9c4e81-installation-pull-secrets\") pod \"034231f3-7707-44c2-ad8b-63307a9c4e81\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") "
Aug 05 17:24:49.904550 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:49.904390    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/034231f3-7707-44c2-ad8b-63307a9c4e81-image-registry-private-configuration\") pod \"034231f3-7707-44c2-ad8b-63307a9c4e81\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") "
Aug 05 17:24:49.904550 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:49.904418    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-bound-sa-token\") pod \"034231f3-7707-44c2-ad8b-63307a9c4e81\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") "
Aug 05 17:24:49.904550 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:49.904459    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-certificates\") pod \"034231f3-7707-44c2-ad8b-63307a9c4e81\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") "
Aug 05 17:24:49.904550 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:49.904505    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x24g9\" (UniqueName: \"kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-kube-api-access-x24g9\") pod \"034231f3-7707-44c2-ad8b-63307a9c4e81\" (UID: \"034231f3-7707-44c2-ad8b-63307a9c4e81\") "
Aug 05 17:24:49.905141 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:49.904907    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/034231f3-7707-44c2-ad8b-63307a9c4e81-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "034231f3-7707-44c2-ad8b-63307a9c4e81" (UID: "034231f3-7707-44c2-ad8b-63307a9c4e81"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:24:49.905346 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:49.905320    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "034231f3-7707-44c2-ad8b-63307a9c4e81" (UID: "034231f3-7707-44c2-ad8b-63307a9c4e81"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:24:49.907013 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:49.906960    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "034231f3-7707-44c2-ad8b-63307a9c4e81" (UID: "034231f3-7707-44c2-ad8b-63307a9c4e81"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:24:49.907106 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:49.907035    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/034231f3-7707-44c2-ad8b-63307a9c4e81-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "034231f3-7707-44c2-ad8b-63307a9c4e81" (UID: "034231f3-7707-44c2-ad8b-63307a9c4e81"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:24:49.907106 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:49.907073    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "034231f3-7707-44c2-ad8b-63307a9c4e81" (UID: "034231f3-7707-44c2-ad8b-63307a9c4e81"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:24:49.907219 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:49.907197    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-kube-api-access-x24g9" (OuterVolumeSpecName: "kube-api-access-x24g9") pod "034231f3-7707-44c2-ad8b-63307a9c4e81" (UID: "034231f3-7707-44c2-ad8b-63307a9c4e81"). InnerVolumeSpecName "kube-api-access-x24g9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:24:49.907219 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:49.907207    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/034231f3-7707-44c2-ad8b-63307a9c4e81-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "034231f3-7707-44c2-ad8b-63307a9c4e81" (UID: "034231f3-7707-44c2-ad8b-63307a9c4e81"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:24:49.913569 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:49.913545    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/034231f3-7707-44c2-ad8b-63307a9c4e81-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "034231f3-7707-44c2-ad8b-63307a9c4e81" (UID: "034231f3-7707-44c2-ad8b-63307a9c4e81"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:24:50.005398 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:50.005325    2550 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/034231f3-7707-44c2-ad8b-63307a9c4e81-image-registry-private-configuration\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:24:50.005398 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:50.005351    2550 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-bound-sa-token\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:24:50.005398 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:50.005362    2550 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-certificates\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:24:50.005398 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:50.005382    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x24g9\" (UniqueName: \"kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-kube-api-access-x24g9\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:24:50.005398 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:50.005391    2550 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/034231f3-7707-44c2-ad8b-63307a9c4e81-ca-trust-extracted\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:24:50.005398 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:50.005401    2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/034231f3-7707-44c2-ad8b-63307a9c4e81-trusted-ca\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:24:50.005740 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:50.005410    2550 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/034231f3-7707-44c2-ad8b-63307a9c4e81-registry-tls\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:24:50.005740 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:50.005418    2550 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/034231f3-7707-44c2-ad8b-63307a9c4e81-installation-pull-secrets\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:24:50.328329 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:50.328248    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gxsh5_8bdd9d59-2279-49b8-882d-210c0d146301/serve-healthcheck-canary/0.log"
Aug 05 17:24:50.543659 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:50.543626    2550 generic.go:356] "Generic (PLEG): container finished" podID="034231f3-7707-44c2-ad8b-63307a9c4e81" containerID="ac4c25911cdb65f538db5c19a9975fed3eda8c6295e207a5b50aa0192b1f6d0e" exitCode=0
Aug 05 17:24:50.544107 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:50.543670    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq" event={"ID":"034231f3-7707-44c2-ad8b-63307a9c4e81","Type":"ContainerDied","Data":"ac4c25911cdb65f538db5c19a9975fed3eda8c6295e207a5b50aa0192b1f6d0e"}
Aug 05 17:24:50.544107 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:50.543672    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq"
Aug 05 17:24:50.544107 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:50.543717    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d8d4888f6-kkxqq" event={"ID":"034231f3-7707-44c2-ad8b-63307a9c4e81","Type":"ContainerDied","Data":"6b6a4e93f60d56686dc3eab2b2f82ddea811ccf2cf039064411576f379a422f0"}
Aug 05 17:24:50.544107 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:50.543737    2550 scope.go:117] "RemoveContainer" containerID="ac4c25911cdb65f538db5c19a9975fed3eda8c6295e207a5b50aa0192b1f6d0e"
Aug 05 17:24:50.553886 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:50.553860    2550 scope.go:117] "RemoveContainer" containerID="ac4c25911cdb65f538db5c19a9975fed3eda8c6295e207a5b50aa0192b1f6d0e"
Aug 05 17:24:50.554542 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:24:50.554518    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac4c25911cdb65f538db5c19a9975fed3eda8c6295e207a5b50aa0192b1f6d0e\": container with ID starting with ac4c25911cdb65f538db5c19a9975fed3eda8c6295e207a5b50aa0192b1f6d0e not found: ID does not exist" containerID="ac4c25911cdb65f538db5c19a9975fed3eda8c6295e207a5b50aa0192b1f6d0e"
Aug 05 17:24:50.554628 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:50.554553    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac4c25911cdb65f538db5c19a9975fed3eda8c6295e207a5b50aa0192b1f6d0e"} err="failed to get container status \"ac4c25911cdb65f538db5c19a9975fed3eda8c6295e207a5b50aa0192b1f6d0e\": rpc error: code = NotFound desc = could not find container \"ac4c25911cdb65f538db5c19a9975fed3eda8c6295e207a5b50aa0192b1f6d0e\": container with ID starting with ac4c25911cdb65f538db5c19a9975fed3eda8c6295e207a5b50aa0192b1f6d0e not found: ID does not exist"
Aug 05 17:24:50.575309 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:50.575285    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d8d4888f6-kkxqq"]
Aug 05 17:24:50.578977 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:50.578923    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-d8d4888f6-kkxqq"]
Aug 05 17:24:51.698350 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:24:51.698321    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="034231f3-7707-44c2-ad8b-63307a9c4e81" path="/var/lib/kubelet/pods/034231f3-7707-44c2-ad8b-63307a9c4e81/volumes"
Aug 05 17:25:44.381595 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:25:44.381555    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs\") pod \"network-metrics-daemon-rkln9\" (UID: \"0fca5e3d-e0e1-4b33-909c-46599a84cf4a\") " pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:25:44.383790 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:25:44.383768    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fca5e3d-e0e1-4b33-909c-46599a84cf4a-metrics-certs\") pod \"network-metrics-daemon-rkln9\" (UID: \"0fca5e3d-e0e1-4b33-909c-46599a84cf4a\") " pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:25:44.598437 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:25:44.598399    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l8xxn\""
Aug 05 17:25:44.606298 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:25:44.606273    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkln9"
Aug 05 17:25:44.736271 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:25:44.736199    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rkln9"]
Aug 05 17:25:44.738867 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:25:44.738844    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fca5e3d_e0e1_4b33_909c_46599a84cf4a.slice/crio-141725b6733889b4c684614814f0ede23c570a3d12e51d838cdc63f59fa4a4e0 WatchSource:0}: Error finding container 141725b6733889b4c684614814f0ede23c570a3d12e51d838cdc63f59fa4a4e0: Status 404 returned error can't find the container with id 141725b6733889b4c684614814f0ede23c570a3d12e51d838cdc63f59fa4a4e0
Aug 05 17:25:45.725866 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:25:45.725826    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rkln9" event={"ID":"0fca5e3d-e0e1-4b33-909c-46599a84cf4a","Type":"ContainerStarted","Data":"141725b6733889b4c684614814f0ede23c570a3d12e51d838cdc63f59fa4a4e0"}
Aug 05 17:25:46.730786 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:25:46.730750    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rkln9" event={"ID":"0fca5e3d-e0e1-4b33-909c-46599a84cf4a","Type":"ContainerStarted","Data":"830b21fcf11c0f8c925df6688f2ebc13437882c1822b69a1fdf659a200e089b0"}
Aug 05 17:25:46.731186 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:25:46.730801    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rkln9" event={"ID":"0fca5e3d-e0e1-4b33-909c-46599a84cf4a","Type":"ContainerStarted","Data":"c2be92db1030b6dada70c09370747b9f0de9e39fc872df2d626c9b3f2c1c7b84"}
Aug 05 17:25:46.751558 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:25:46.751507    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rkln9" podStartSLOduration=252.823588147 podStartE2EDuration="4m13.751492708s" totalImagesPullingTime="927.904561ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:21:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:25:46.751048707 +0000 UTC m=+253.732626135" watchObservedRunningTime="2026-08-05 17:25:46.751492708 +0000 UTC m=+253.733070137"
Aug 05 17:26:33.444392 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:26:33.444364    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 17:26:33.451847 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:26:33.451824    2550 kubelet.go:1648] "Image garbage collection succeeded"
Aug 05 17:26:33.605107 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:26:33.605084    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 17:28:15.254596 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:15.254558    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-6pf7v"]
Aug 05 17:28:15.255105 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:15.254844    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="034231f3-7707-44c2-ad8b-63307a9c4e81" containerName="registry"
Aug 05 17:28:15.255105 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:15.254856    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="034231f3-7707-44c2-ad8b-63307a9c4e81" containerName="registry"
Aug 05 17:28:15.255105 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:15.254913    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="034231f3-7707-44c2-ad8b-63307a9c4e81" containerName="registry"
Aug 05 17:28:15.257548 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:15.257532    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6pf7v"
Aug 05 17:28:15.259795 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:15.259778    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Aug 05 17:28:15.260053 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:15.260038    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-sb9kp\""
Aug 05 17:28:15.260768 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:15.260751    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Aug 05 17:28:15.268589 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:15.268564    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-6pf7v"]
Aug 05 17:28:15.399395 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:15.399352    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c0f11634-f92f-41d4-9a8d-48d3e232b499-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-6pf7v\" (UID: \"c0f11634-f92f-41d4-9a8d-48d3e232b499\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6pf7v"
Aug 05 17:28:15.399574 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:15.399407    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffklc\" (UniqueName: \"kubernetes.io/projected/c0f11634-f92f-41d4-9a8d-48d3e232b499-kube-api-access-ffklc\") pod \"cert-manager-cainjector-559cdd8c87-6pf7v\" (UID: \"c0f11634-f92f-41d4-9a8d-48d3e232b499\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6pf7v"
Aug 05 17:28:15.499945 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:15.499908    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffklc\" (UniqueName: \"kubernetes.io/projected/c0f11634-f92f-41d4-9a8d-48d3e232b499-kube-api-access-ffklc\") pod \"cert-manager-cainjector-559cdd8c87-6pf7v\" (UID: \"c0f11634-f92f-41d4-9a8d-48d3e232b499\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6pf7v"
Aug 05 17:28:15.500095 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:15.499970    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c0f11634-f92f-41d4-9a8d-48d3e232b499-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-6pf7v\" (UID: \"c0f11634-f92f-41d4-9a8d-48d3e232b499\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6pf7v"
Aug 05 17:28:15.509202 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:15.509133    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c0f11634-f92f-41d4-9a8d-48d3e232b499-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-6pf7v\" (UID: \"c0f11634-f92f-41d4-9a8d-48d3e232b499\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6pf7v"
Aug 05 17:28:15.509202 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:15.509187    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffklc\" (UniqueName: \"kubernetes.io/projected/c0f11634-f92f-41d4-9a8d-48d3e232b499-kube-api-access-ffklc\") pod \"cert-manager-cainjector-559cdd8c87-6pf7v\" (UID: \"c0f11634-f92f-41d4-9a8d-48d3e232b499\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6pf7v"
Aug 05 17:28:15.566165 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:15.566126    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6pf7v"
Aug 05 17:28:15.699976 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:15.699954    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-6pf7v"]
Aug 05 17:28:15.701366 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:28:15.701343    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0f11634_f92f_41d4_9a8d_48d3e232b499.slice/crio-363ba97cc6af681f35c55a5e1de07915d1a653e6c7093987e90e499622d8411e WatchSource:0}: Error finding container 363ba97cc6af681f35c55a5e1de07915d1a653e6c7093987e90e499622d8411e: Status 404 returned error can't find the container with id 363ba97cc6af681f35c55a5e1de07915d1a653e6c7093987e90e499622d8411e
Aug 05 17:28:15.703209 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:15.703193    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 17:28:16.211051 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:16.210988    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6pf7v" event={"ID":"c0f11634-f92f-41d4-9a8d-48d3e232b499","Type":"ContainerStarted","Data":"363ba97cc6af681f35c55a5e1de07915d1a653e6c7093987e90e499622d8411e"}
Aug 05 17:28:19.223858 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:19.223817    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6pf7v" event={"ID":"c0f11634-f92f-41d4-9a8d-48d3e232b499","Type":"ContainerStarted","Data":"46def43989fe114113ed75edae2e370932f72b9e494b87d525bf80e93211e766"}
Aug 05 17:28:19.240005 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:19.239951    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6pf7v" podStartSLOduration=1.4672956560000001 podStartE2EDuration="4.239932794s" totalImagesPullingTime="2.772637138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:28:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:28:19.239001037 +0000 UTC m=+406.220578466" watchObservedRunningTime="2026-08-05 17:28:19.239932794 +0000 UTC m=+406.221510223"
Aug 05 17:28:50.022685 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:50.022644    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cc69f54b9-qlhch"]
Aug 05 17:28:50.028286 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:50.028261    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7cc69f54b9-qlhch"
Aug 05 17:28:50.031281 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:50.031244    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Aug 05 17:28:50.031530 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:50.031326    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Aug 05 17:28:50.031661 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:50.031639    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Aug 05 17:28:50.031841 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:50.031354    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Aug 05 17:28:50.032498 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:50.032473    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-6px7q\""
Aug 05 17:28:50.044237 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:50.044216    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cc69f54b9-qlhch"]
Aug 05 17:28:50.141690 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:50.141650    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4fd60b10-0d46-4be8-b0d6-73ad8455b81d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cc69f54b9-qlhch\" (UID: \"4fd60b10-0d46-4be8-b0d6-73ad8455b81d\") " pod="opendatahub/opendatahub-operator-controller-manager-7cc69f54b9-qlhch"
Aug 05 17:28:50.141868 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:50.141717    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhknx\" (UniqueName: \"kubernetes.io/projected/4fd60b10-0d46-4be8-b0d6-73ad8455b81d-kube-api-access-rhknx\") pod \"opendatahub-operator-controller-manager-7cc69f54b9-qlhch\" (UID: \"4fd60b10-0d46-4be8-b0d6-73ad8455b81d\") " pod="opendatahub/opendatahub-operator-controller-manager-7cc69f54b9-qlhch"
Aug 05 17:28:50.141868 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:50.141769    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4fd60b10-0d46-4be8-b0d6-73ad8455b81d-manifests\") pod \"opendatahub-operator-controller-manager-7cc69f54b9-qlhch\" (UID: \"4fd60b10-0d46-4be8-b0d6-73ad8455b81d\") " pod="opendatahub/opendatahub-operator-controller-manager-7cc69f54b9-qlhch"
Aug 05 17:28:50.141868 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:50.141800    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4fd60b10-0d46-4be8-b0d6-73ad8455b81d-tmp\") pod \"opendatahub-operator-controller-manager-7cc69f54b9-qlhch\" (UID: \"4fd60b10-0d46-4be8-b0d6-73ad8455b81d\") " pod="opendatahub/opendatahub-operator-controller-manager-7cc69f54b9-qlhch"
Aug 05 17:28:50.141868 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:50.141832    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4fd60b10-0d46-4be8-b0d6-73ad8455b81d-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cc69f54b9-qlhch\" (UID: \"4fd60b10-0d46-4be8-b0d6-73ad8455b81d\") " pod="opendatahub/opendatahub-operator-controller-manager-7cc69f54b9-qlhch"
Aug 05 17:28:50.242987 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:50.242947    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4fd60b10-0d46-4be8-b0d6-73ad8455b81d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cc69f54b9-qlhch\" (UID: \"4fd60b10-0d46-4be8-b0d6-73ad8455b81d\") " pod="opendatahub/opendatahub-operator-controller-manager-7cc69f54b9-qlhch"
Aug 05 17:28:50.243148 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:50.242994    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhknx\" (UniqueName: \"kubernetes.io/projected/4fd60b10-0d46-4be8-b0d6-73ad8455b81d-kube-api-access-rhknx\") pod \"opendatahub-operator-controller-manager-7cc69f54b9-qlhch\" (UID: \"4fd60b10-0d46-4be8-b0d6-73ad8455b81d\") " pod="opendatahub/opendatahub-operator-controller-manager-7cc69f54b9-qlhch"
Aug 05 17:28:50.243148 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:50.243027    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4fd60b10-0d46-4be8-b0d6-73ad8455b81d-manifests\") pod \"opendatahub-operator-controller-manager-7cc69f54b9-qlhch\" (UID: \"4fd60b10-0d46-4be8-b0d6-73ad8455b81d\") " pod="opendatahub/opendatahub-operator-controller-manager-7cc69f54b9-qlhch"
Aug 05 17:28:50.243148 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:50.243064    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4fd60b10-0d46-4be8-b0d6-73ad8455b81d-tmp\") pod \"opendatahub-operator-controller-manager-7cc69f54b9-qlhch\" (UID: \"4fd60b10-0d46-4be8-b0d6-73ad8455b81d\") " pod="opendatahub/opendatahub-operator-controller-manager-7cc69f54b9-qlhch"
Aug 05 17:28:50.243148 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:50.243104    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4fd60b10-0d46-4be8-b0d6-73ad8455b81d-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cc69f54b9-qlhch\" (UID: \"4fd60b10-0d46-4be8-b0d6-73ad8455b81d\") " pod="opendatahub/opendatahub-operator-controller-manager-7cc69f54b9-qlhch"
Aug 05 17:28:50.243568 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:50.243540    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4fd60b10-0d46-4be8-b0d6-73ad8455b81d-manifests\") pod \"opendatahub-operator-controller-manager-7cc69f54b9-qlhch\" (UID: \"4fd60b10-0d46-4be8-b0d6-73ad8455b81d\") " pod="opendatahub/opendatahub-operator-controller-manager-7cc69f54b9-qlhch"
Aug 05 17:28:50.245475 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:50.245438    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4fd60b10-0d46-4be8-b0d6-73ad8455b81d-tmp\") pod \"opendatahub-operator-controller-manager-7cc69f54b9-qlhch\" (UID: \"4fd60b10-0d46-4be8-b0d6-73ad8455b81d\") " pod="opendatahub/opendatahub-operator-controller-manager-7cc69f54b9-qlhch"
Aug 05 17:28:50.245601 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:50.245495    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4fd60b10-0d46-4be8-b0d6-73ad8455b81d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cc69f54b9-qlhch\" (UID: \"4fd60b10-0d46-4be8-b0d6-73ad8455b81d\") " pod="opendatahub/opendatahub-operator-controller-manager-7cc69f54b9-qlhch"
Aug 05 17:28:50.245601 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:50.245535    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4fd60b10-0d46-4be8-b0d6-73ad8455b81d-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cc69f54b9-qlhch\" (UID: \"4fd60b10-0d46-4be8-b0d6-73ad8455b81d\") " pod="opendatahub/opendatahub-operator-controller-manager-7cc69f54b9-qlhch"
Aug 05 17:28:50.251761 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:50.251738    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhknx\" (UniqueName: \"kubernetes.io/projected/4fd60b10-0d46-4be8-b0d6-73ad8455b81d-kube-api-access-rhknx\") pod \"opendatahub-operator-controller-manager-7cc69f54b9-qlhch\" (UID: \"4fd60b10-0d46-4be8-b0d6-73ad8455b81d\") " pod="opendatahub/opendatahub-operator-controller-manager-7cc69f54b9-qlhch"
Aug 05 17:28:50.342771 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:50.342669    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7cc69f54b9-qlhch"
Aug 05 17:28:50.476788 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:50.476751    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cc69f54b9-qlhch"]
Aug 05 17:28:50.479903 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:28:50.479876    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fd60b10_0d46_4be8_b0d6_73ad8455b81d.slice/crio-257373c780dc1f95bf0e9f020cc4f4eb002bebf783bf9332ab77eb5821dd7f5f WatchSource:0}: Error finding container 257373c780dc1f95bf0e9f020cc4f4eb002bebf783bf9332ab77eb5821dd7f5f: Status 404 returned error can't find the container with id 257373c780dc1f95bf0e9f020cc4f4eb002bebf783bf9332ab77eb5821dd7f5f
Aug 05 17:28:51.332387 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:51.332318    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cc69f54b9-qlhch" event={"ID":"4fd60b10-0d46-4be8-b0d6-73ad8455b81d","Type":"ContainerStarted","Data":"257373c780dc1f95bf0e9f020cc4f4eb002bebf783bf9332ab77eb5821dd7f5f"}
Aug 05 17:28:54.346086 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:54.346054    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cc69f54b9-qlhch" event={"ID":"4fd60b10-0d46-4be8-b0d6-73ad8455b81d","Type":"ContainerStarted","Data":"eecab4c62894e7805fdf61701cc54d3afb7db2f9e8749c7a38dbacea96a34411"}
Aug 05 17:28:55.350902 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:55.350867    2550 generic.go:356] "Generic (PLEG): container finished" podID="4fd60b10-0d46-4be8-b0d6-73ad8455b81d" containerID="eecab4c62894e7805fdf61701cc54d3afb7db2f9e8749c7a38dbacea96a34411" exitCode=0
Aug 05 17:28:55.351330 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:55.350941    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cc69f54b9-qlhch" event={"ID":"4fd60b10-0d46-4be8-b0d6-73ad8455b81d","Type":"ContainerDied","Data":"eecab4c62894e7805fdf61701cc54d3afb7db2f9e8749c7a38dbacea96a34411"}
Aug 05 17:28:56.356142 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:56.356099    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cc69f54b9-qlhch" event={"ID":"4fd60b10-0d46-4be8-b0d6-73ad8455b81d","Type":"ContainerStarted","Data":"b55eb55f0cae36a302e577482ad437e31728fb22f26edc9a08f687123253e94f"}
Aug 05 17:28:56.356622 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:56.356249    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7cc69f54b9-qlhch"
Aug 05 17:28:56.377063 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:28:56.377021    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7cc69f54b9-qlhch" podStartSLOduration=3.020685949 podStartE2EDuration="7.377007891s" totalImagesPullingTime="3.340548887s" totalInitContainerRuntime="1.015773055s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:28:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:28:56.376835206 +0000 UTC m=+443.358412636" watchObservedRunningTime="2026-08-05 17:28:56.377007891 +0000 UTC m=+443.358585320"
Aug 05 17:29:07.362401 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:07.362372    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7cc69f54b9-qlhch"
Aug 05 17:29:10.035846 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.035812    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-746fd96bc4-btn27"]
Aug 05 17:29:10.046905 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.046881    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-746fd96bc4-btn27"
Aug 05 17:29:10.049692 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.049663    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Aug 05 17:29:10.049904 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.049865    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-jtrt9\""
Aug 05 17:29:10.050895 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.050843    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Aug 05 17:29:10.050996 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.050918    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Aug 05 17:29:10.050996 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.050931    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Aug 05 17:29:10.052403 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.052384    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-746fd96bc4-btn27"]
Aug 05 17:29:10.060989 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.060967    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bfsdk"]
Aug 05 17:29:10.064115 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.064096    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-bfsdk"
Aug 05 17:29:10.066743 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.066721    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Aug 05 17:29:10.066869 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.066853    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-pl7sg\""
Aug 05 17:29:10.071434 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.071416    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db99dc27-ca54-4ccb-ac40-eec6ddfbb1d5-cert\") pod \"odh-model-controller-7998c67dc4-bfsdk\" (UID: \"db99dc27-ca54-4ccb-ac40-eec6ddfbb1d5\") " pod="opendatahub/odh-model-controller-7998c67dc4-bfsdk"
Aug 05 17:29:10.071524 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.071443    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nssw\" (UniqueName: \"kubernetes.io/projected/db99dc27-ca54-4ccb-ac40-eec6ddfbb1d5-kube-api-access-9nssw\") pod \"odh-model-controller-7998c67dc4-bfsdk\" (UID: \"db99dc27-ca54-4ccb-ac40-eec6ddfbb1d5\") " pod="opendatahub/odh-model-controller-7998c67dc4-bfsdk"
Aug 05 17:29:10.071524 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.071472    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbp9n\" (UniqueName: \"kubernetes.io/projected/8b6db924-1edc-415d-91c3-292040e1f654-kube-api-access-tbp9n\") pod \"kube-auth-proxy-746fd96bc4-btn27\" (UID: \"8b6db924-1edc-415d-91c3-292040e1f654\") " pod="openshift-ingress/kube-auth-proxy-746fd96bc4-btn27"
Aug 05 17:29:10.071524 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.071502    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b6db924-1edc-415d-91c3-292040e1f654-tls-certs\") pod \"kube-auth-proxy-746fd96bc4-btn27\" (UID: \"8b6db924-1edc-415d-91c3-292040e1f654\") " pod="openshift-ingress/kube-auth-proxy-746fd96bc4-btn27"
Aug 05 17:29:10.071626 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.071536    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b6db924-1edc-415d-91c3-292040e1f654-tmp\") pod \"kube-auth-proxy-746fd96bc4-btn27\" (UID: \"8b6db924-1edc-415d-91c3-292040e1f654\") " pod="openshift-ingress/kube-auth-proxy-746fd96bc4-btn27"
Aug 05 17:29:10.074190 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.074172    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bfsdk"]
Aug 05 17:29:10.172334 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.172311    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbp9n\" (UniqueName: \"kubernetes.io/projected/8b6db924-1edc-415d-91c3-292040e1f654-kube-api-access-tbp9n\") pod \"kube-auth-proxy-746fd96bc4-btn27\" (UID: \"8b6db924-1edc-415d-91c3-292040e1f654\") " pod="openshift-ingress/kube-auth-proxy-746fd96bc4-btn27"
Aug 05 17:29:10.172465 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.172350    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b6db924-1edc-415d-91c3-292040e1f654-tls-certs\") pod \"kube-auth-proxy-746fd96bc4-btn27\" (UID: \"8b6db924-1edc-415d-91c3-292040e1f654\") " pod="openshift-ingress/kube-auth-proxy-746fd96bc4-btn27"
Aug 05 17:29:10.172465 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.172374    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b6db924-1edc-415d-91c3-292040e1f654-tmp\") pod \"kube-auth-proxy-746fd96bc4-btn27\" (UID: \"8b6db924-1edc-415d-91c3-292040e1f654\") " pod="openshift-ingress/kube-auth-proxy-746fd96bc4-btn27"
Aug 05 17:29:10.172465 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.172426    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db99dc27-ca54-4ccb-ac40-eec6ddfbb1d5-cert\") pod \"odh-model-controller-7998c67dc4-bfsdk\" (UID: \"db99dc27-ca54-4ccb-ac40-eec6ddfbb1d5\") " pod="opendatahub/odh-model-controller-7998c67dc4-bfsdk"
Aug 05 17:29:10.172465 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.172452    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nssw\" (UniqueName: \"kubernetes.io/projected/db99dc27-ca54-4ccb-ac40-eec6ddfbb1d5-kube-api-access-9nssw\") pod \"odh-model-controller-7998c67dc4-bfsdk\" (UID: \"db99dc27-ca54-4ccb-ac40-eec6ddfbb1d5\") " pod="opendatahub/odh-model-controller-7998c67dc4-bfsdk"
Aug 05 17:29:10.172662 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:29:10.172474    2550 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Aug 05 17:29:10.172662 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:29:10.172540    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b6db924-1edc-415d-91c3-292040e1f654-tls-certs podName:8b6db924-1edc-415d-91c3-292040e1f654 nodeName:}" failed. No retries permitted until 2026-08-05 17:29:10.672518533 +0000 UTC m=+457.654095956 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/8b6db924-1edc-415d-91c3-292040e1f654-tls-certs") pod "kube-auth-proxy-746fd96bc4-btn27" (UID: "8b6db924-1edc-415d-91c3-292040e1f654") : secret "kube-auth-proxy-tls" not found
Aug 05 17:29:10.172662 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:29:10.172548    2550 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Aug 05 17:29:10.172662 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:29:10.172606    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db99dc27-ca54-4ccb-ac40-eec6ddfbb1d5-cert podName:db99dc27-ca54-4ccb-ac40-eec6ddfbb1d5 nodeName:}" failed. No retries permitted until 2026-08-05 17:29:10.672591736 +0000 UTC m=+457.654169143 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/db99dc27-ca54-4ccb-ac40-eec6ddfbb1d5-cert") pod "odh-model-controller-7998c67dc4-bfsdk" (UID: "db99dc27-ca54-4ccb-ac40-eec6ddfbb1d5") : secret "odh-model-controller-webhook-cert" not found
Aug 05 17:29:10.174718 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.174672    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b6db924-1edc-415d-91c3-292040e1f654-tmp\") pod \"kube-auth-proxy-746fd96bc4-btn27\" (UID: \"8b6db924-1edc-415d-91c3-292040e1f654\") " pod="openshift-ingress/kube-auth-proxy-746fd96bc4-btn27"
Aug 05 17:29:10.181069 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.181047    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nssw\" (UniqueName: \"kubernetes.io/projected/db99dc27-ca54-4ccb-ac40-eec6ddfbb1d5-kube-api-access-9nssw\") pod \"odh-model-controller-7998c67dc4-bfsdk\" (UID: \"db99dc27-ca54-4ccb-ac40-eec6ddfbb1d5\") " pod="opendatahub/odh-model-controller-7998c67dc4-bfsdk"
Aug 05 17:29:10.181220 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.181200    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbp9n\" (UniqueName: \"kubernetes.io/projected/8b6db924-1edc-415d-91c3-292040e1f654-kube-api-access-tbp9n\") pod \"kube-auth-proxy-746fd96bc4-btn27\" (UID: \"8b6db924-1edc-415d-91c3-292040e1f654\") " pod="openshift-ingress/kube-auth-proxy-746fd96bc4-btn27"
Aug 05 17:29:10.676809 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.676774    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db99dc27-ca54-4ccb-ac40-eec6ddfbb1d5-cert\") pod \"odh-model-controller-7998c67dc4-bfsdk\" (UID: \"db99dc27-ca54-4ccb-ac40-eec6ddfbb1d5\") " pod="opendatahub/odh-model-controller-7998c67dc4-bfsdk"
Aug 05 17:29:10.676977 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.676828    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b6db924-1edc-415d-91c3-292040e1f654-tls-certs\") pod \"kube-auth-proxy-746fd96bc4-btn27\" (UID: \"8b6db924-1edc-415d-91c3-292040e1f654\") " pod="openshift-ingress/kube-auth-proxy-746fd96bc4-btn27"
Aug 05 17:29:10.679182 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.679151    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db99dc27-ca54-4ccb-ac40-eec6ddfbb1d5-cert\") pod \"odh-model-controller-7998c67dc4-bfsdk\" (UID: \"db99dc27-ca54-4ccb-ac40-eec6ddfbb1d5\") " pod="opendatahub/odh-model-controller-7998c67dc4-bfsdk"
Aug 05 17:29:10.679297 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.679215    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b6db924-1edc-415d-91c3-292040e1f654-tls-certs\") pod \"kube-auth-proxy-746fd96bc4-btn27\" (UID: \"8b6db924-1edc-415d-91c3-292040e1f654\") " pod="openshift-ingress/kube-auth-proxy-746fd96bc4-btn27"
Aug 05 17:29:10.935232 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.935155    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-kxjqm"]
Aug 05 17:29:10.938577 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.938562    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-kxjqm"
Aug 05 17:29:10.940798 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.940776    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Aug 05 17:29:10.947041 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.947011    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-kxjqm"]
Aug 05 17:29:10.957646 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.957629    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-746fd96bc4-btn27"
Aug 05 17:29:10.973190 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.973169    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-bfsdk"
Aug 05 17:29:10.978381 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.978359    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e24fda29-710a-4354-9a13-a35317191c15-tls-certs\") pod \"model-serving-api-54d89bd5fc-kxjqm\" (UID: \"e24fda29-710a-4354-9a13-a35317191c15\") " pod="opendatahub/model-serving-api-54d89bd5fc-kxjqm"
Aug 05 17:29:10.978475 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:10.978404    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvqjp\" (UniqueName: \"kubernetes.io/projected/e24fda29-710a-4354-9a13-a35317191c15-kube-api-access-bvqjp\") pod \"model-serving-api-54d89bd5fc-kxjqm\" (UID: \"e24fda29-710a-4354-9a13-a35317191c15\") " pod="opendatahub/model-serving-api-54d89bd5fc-kxjqm"
Aug 05 17:29:11.079209 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.079176    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e24fda29-710a-4354-9a13-a35317191c15-tls-certs\") pod \"model-serving-api-54d89bd5fc-kxjqm\" (UID: \"e24fda29-710a-4354-9a13-a35317191c15\") " pod="opendatahub/model-serving-api-54d89bd5fc-kxjqm"
Aug 05 17:29:11.079609 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.079233    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvqjp\" (UniqueName: \"kubernetes.io/projected/e24fda29-710a-4354-9a13-a35317191c15-kube-api-access-bvqjp\") pod \"model-serving-api-54d89bd5fc-kxjqm\" (UID: \"e24fda29-710a-4354-9a13-a35317191c15\") " pod="opendatahub/model-serving-api-54d89bd5fc-kxjqm"
Aug 05 17:29:11.079609 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:29:11.079345    2550 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Aug 05 17:29:11.079609 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:29:11.079434    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e24fda29-710a-4354-9a13-a35317191c15-tls-certs podName:e24fda29-710a-4354-9a13-a35317191c15 nodeName:}" failed. No retries permitted until 2026-08-05 17:29:11.57940211 +0000 UTC m=+458.560979536 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/e24fda29-710a-4354-9a13-a35317191c15-tls-certs") pod "model-serving-api-54d89bd5fc-kxjqm" (UID: "e24fda29-710a-4354-9a13-a35317191c15") : secret "model-serving-api-tls" not found
Aug 05 17:29:11.095902 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.095863    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-746fd96bc4-btn27"]
Aug 05 17:29:11.098267 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.098248    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvqjp\" (UniqueName: \"kubernetes.io/projected/e24fda29-710a-4354-9a13-a35317191c15-kube-api-access-bvqjp\") pod \"model-serving-api-54d89bd5fc-kxjqm\" (UID: \"e24fda29-710a-4354-9a13-a35317191c15\") " pod="opendatahub/model-serving-api-54d89bd5fc-kxjqm"
Aug 05 17:29:11.098937 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:29:11.098911    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b6db924_1edc_415d_91c3_292040e1f654.slice/crio-90ba932e92eb4a8e7a9d408dce717572731951e3a5bbe2b282dc860ebcbc9b45 WatchSource:0}: Error finding container 90ba932e92eb4a8e7a9d408dce717572731951e3a5bbe2b282dc860ebcbc9b45: Status 404 returned error can't find the container with id 90ba932e92eb4a8e7a9d408dce717572731951e3a5bbe2b282dc860ebcbc9b45
Aug 05 17:29:11.129124 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.129102    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bfsdk"]
Aug 05 17:29:11.131410 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:29:11.131380    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb99dc27_ca54_4ccb_ac40_eec6ddfbb1d5.slice/crio-eb3baccc9ebd531f021e4dcd28bd8d7a59c2116af5cbcaf0a05eac5bab75932d WatchSource:0}: Error finding container eb3baccc9ebd531f021e4dcd28bd8d7a59c2116af5cbcaf0a05eac5bab75932d: Status 404 returned error can't find the container with id eb3baccc9ebd531f021e4dcd28bd8d7a59c2116af5cbcaf0a05eac5bab75932d
Aug 05 17:29:11.213064 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.212987    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"]
Aug 05 17:29:11.217348 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.217332    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"
Aug 05 17:29:11.219652 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.219627    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Aug 05 17:29:11.219652 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.219622    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Aug 05 17:29:11.219652 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.219635    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-kxsv8\""
Aug 05 17:29:11.219928 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.219659    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Aug 05 17:29:11.219928 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.219791    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Aug 05 17:29:11.228350 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.228324    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"]
Aug 05 17:29:11.280919 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.280874    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rfgk\" (UniqueName: \"kubernetes.io/projected/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-kube-api-access-2rfgk\") pod \"istiod-openshift-gateway-7b7995d5cd-5l5nm\" (UID: \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"
Aug 05 17:29:11.281075 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.280942    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-5l5nm\" (UID: \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"
Aug 05 17:29:11.281075 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.280970    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-5l5nm\" (UID: \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"
Aug 05 17:29:11.281075 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.281007    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-5l5nm\" (UID: \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"
Aug 05 17:29:11.281204 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.281074    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-5l5nm\" (UID: \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"
Aug 05 17:29:11.281204 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.281095    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-5l5nm\" (UID: \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"
Aug 05 17:29:11.281204 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.281128    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-5l5nm\" (UID: \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"
Aug 05 17:29:11.382082 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.382050    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rfgk\" (UniqueName: \"kubernetes.io/projected/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-kube-api-access-2rfgk\") pod \"istiod-openshift-gateway-7b7995d5cd-5l5nm\" (UID: \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"
Aug 05 17:29:11.382224 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.382095    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-5l5nm\" (UID: \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"
Aug 05 17:29:11.382224 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.382206    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-5l5nm\" (UID: \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"
Aug 05 17:29:11.382294 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.382279    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-5l5nm\" (UID: \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"
Aug 05 17:29:11.382423 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.382342    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-5l5nm\" (UID: \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"
Aug 05 17:29:11.382423 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.382376    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-5l5nm\" (UID: \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"
Aug 05 17:29:11.382423 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.382405    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-5l5nm\" (UID: \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"
Aug 05 17:29:11.382970 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.382948    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-5l5nm\" (UID: \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"
Aug 05 17:29:11.384304 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.384283    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-5l5nm\" (UID: \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"
Aug 05 17:29:11.384415 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.384394    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-5l5nm\" (UID: \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"
Aug 05 17:29:11.384511 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.384496    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-5l5nm\" (UID: \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"
Aug 05 17:29:11.384550 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.384531    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-5l5nm\" (UID: \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"
Aug 05 17:29:11.389512 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.389495    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rfgk\" (UniqueName: \"kubernetes.io/projected/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-kube-api-access-2rfgk\") pod \"istiod-openshift-gateway-7b7995d5cd-5l5nm\" (UID: \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"
Aug 05 17:29:11.389745 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.389730    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-5l5nm\" (UID: \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"
Aug 05 17:29:11.409367 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.409344    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-bfsdk" event={"ID":"db99dc27-ca54-4ccb-ac40-eec6ddfbb1d5","Type":"ContainerStarted","Data":"eb3baccc9ebd531f021e4dcd28bd8d7a59c2116af5cbcaf0a05eac5bab75932d"}
Aug 05 17:29:11.410409 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.410389    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-746fd96bc4-btn27" event={"ID":"8b6db924-1edc-415d-91c3-292040e1f654","Type":"ContainerStarted","Data":"90ba932e92eb4a8e7a9d408dce717572731951e3a5bbe2b282dc860ebcbc9b45"}
Aug 05 17:29:11.526868 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.526808    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"
Aug 05 17:29:11.584099 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.584064    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e24fda29-710a-4354-9a13-a35317191c15-tls-certs\") pod \"model-serving-api-54d89bd5fc-kxjqm\" (UID: \"e24fda29-710a-4354-9a13-a35317191c15\") " pod="opendatahub/model-serving-api-54d89bd5fc-kxjqm"
Aug 05 17:29:11.586180 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.586157    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e24fda29-710a-4354-9a13-a35317191c15-tls-certs\") pod \"model-serving-api-54d89bd5fc-kxjqm\" (UID: \"e24fda29-710a-4354-9a13-a35317191c15\") " pod="opendatahub/model-serving-api-54d89bd5fc-kxjqm"
Aug 05 17:29:11.654503 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.654470    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"]
Aug 05 17:29:11.657071 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:29:11.657044    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03ec8c9d_7fb8_428c_a6fe_1112c125cfbf.slice/crio-4fc4d015bc50ce10de31618ba387fdbdc5701361d67c5419e68d9f6a196f8c44 WatchSource:0}: Error finding container 4fc4d015bc50ce10de31618ba387fdbdc5701361d67c5419e68d9f6a196f8c44: Status 404 returned error can't find the container with id 4fc4d015bc50ce10de31618ba387fdbdc5701361d67c5419e68d9f6a196f8c44
Aug 05 17:29:11.850037 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:11.849937    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-kxjqm"
Aug 05 17:29:12.048114 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:12.048082    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-kxjqm"]
Aug 05 17:29:12.050398 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:29:12.050371    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode24fda29_710a_4354_9a13_a35317191c15.slice/crio-dc4720fc44516431f28a873ce32ee7aa269a2af122bf194d7328afe5f660b8dc WatchSource:0}: Error finding container dc4720fc44516431f28a873ce32ee7aa269a2af122bf194d7328afe5f660b8dc: Status 404 returned error can't find the container with id dc4720fc44516431f28a873ce32ee7aa269a2af122bf194d7328afe5f660b8dc
Aug 05 17:29:12.417890 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:12.417522    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm" event={"ID":"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf","Type":"ContainerStarted","Data":"4fc4d015bc50ce10de31618ba387fdbdc5701361d67c5419e68d9f6a196f8c44"}
Aug 05 17:29:12.419481 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:12.419427    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-kxjqm" event={"ID":"e24fda29-710a-4354-9a13-a35317191c15","Type":"ContainerStarted","Data":"dc4720fc44516431f28a873ce32ee7aa269a2af122bf194d7328afe5f660b8dc"}
Aug 05 17:29:17.251883 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:17.251849    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 17:29:17.252140 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:17.251915    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 17:29:17.441637 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:17.441595    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-746fd96bc4-btn27" event={"ID":"8b6db924-1edc-415d-91c3-292040e1f654","Type":"ContainerStarted","Data":"892bdc247397cabba84b9ed77c9b9938a0adb0ed94e8efcbf3734a3c658ef20b"}
Aug 05 17:29:17.443350 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:17.443316    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm" event={"ID":"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf","Type":"ContainerStarted","Data":"1f168aaa326049837d1b90b38339ee6e971917e9c0e76585779402e91706f654"}
Aug 05 17:29:17.443486 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:17.443423    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"
Aug 05 17:29:17.444950 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:17.444921    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-bfsdk" event={"ID":"db99dc27-ca54-4ccb-ac40-eec6ddfbb1d5","Type":"ContainerStarted","Data":"14d549e84b5e7c066fe43467b571c07d39c879c56c4d24424744c19536e3e844"}
Aug 05 17:29:17.445064 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:17.445004    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-bfsdk"
Aug 05 17:29:17.446624 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:17.446598    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-kxjqm" event={"ID":"e24fda29-710a-4354-9a13-a35317191c15","Type":"ContainerStarted","Data":"fcd4e3b46f2c230eb553411660fa5d74c27f492768d112c2ac3053541565347b"}
Aug 05 17:29:17.446766 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:17.446663    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-kxjqm"
Aug 05 17:29:17.474231 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:17.474180    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-746fd96bc4-btn27" podStartSLOduration=1.306319407 podStartE2EDuration="7.474161165s" totalImagesPullingTime="6.167841758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:29:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:29:17.472021488 +0000 UTC m=+464.453598915" watchObservedRunningTime="2026-08-05 17:29:17.474161165 +0000 UTC m=+464.455738597"
Aug 05 17:29:17.504280 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:17.504177    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-kxjqm" podStartSLOduration=2.300591486 podStartE2EDuration="7.504153406s" totalImagesPullingTime="5.20356192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:29:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:29:17.502195028 +0000 UTC m=+464.483772458" watchObservedRunningTime="2026-08-05 17:29:17.504153406 +0000 UTC m=+464.485730817"
Aug 05 17:29:17.534213 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:17.534158    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-bfsdk" podStartSLOduration=1.418958314 podStartE2EDuration="7.53413955s" totalImagesPullingTime="6.115181236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:29:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:29:17.533926784 +0000 UTC m=+464.515504205" watchObservedRunningTime="2026-08-05 17:29:17.53413955 +0000 UTC m=+464.515716980"
Aug 05 17:29:17.569849 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:17.569805    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm" podStartSLOduration=0.977032146 podStartE2EDuration="6.569791443s" totalImagesPullingTime="5.592759297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:29:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:29:17.568602483 +0000 UTC m=+464.550179912" watchObservedRunningTime="2026-08-05 17:29:17.569791443 +0000 UTC m=+464.551368873"
Aug 05 17:29:18.451773 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:18.451739    2550 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-5l5nm container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Aug 05 17:29:18.452107 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:18.451789    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm" podUID="03ec8c9d-7fb8-428c-a6fe-1112c125cfbf" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 05 17:29:18.968275 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:18.968239    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"]
Aug 05 17:29:18.971692 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:18.971670    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:18.975038 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:18.975018    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-mt7qr\""
Aug 05 17:29:18.984001 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:18.983980    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"]
Aug 05 17:29:19.040959 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.040933    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f44bcc3-3f40-403f-bcc8-791f5472340b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c\" (UID: \"3f44bcc3-3f40-403f-bcc8-791f5472340b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:19.041085 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.040962    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f44bcc3-3f40-403f-bcc8-791f5472340b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c\" (UID: \"3f44bcc3-3f40-403f-bcc8-791f5472340b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:19.041085 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.040982    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qqbc\" (UniqueName: \"kubernetes.io/projected/3f44bcc3-3f40-403f-bcc8-791f5472340b-kube-api-access-5qqbc\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c\" (UID: \"3f44bcc3-3f40-403f-bcc8-791f5472340b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:19.041085 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.041015    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f44bcc3-3f40-403f-bcc8-791f5472340b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c\" (UID: \"3f44bcc3-3f40-403f-bcc8-791f5472340b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:19.041085 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.041041    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f44bcc3-3f40-403f-bcc8-791f5472340b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c\" (UID: \"3f44bcc3-3f40-403f-bcc8-791f5472340b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:19.041085 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.041063    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f44bcc3-3f40-403f-bcc8-791f5472340b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c\" (UID: \"3f44bcc3-3f40-403f-bcc8-791f5472340b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:19.041085 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.041081    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f44bcc3-3f40-403f-bcc8-791f5472340b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c\" (UID: \"3f44bcc3-3f40-403f-bcc8-791f5472340b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:19.041278 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.041098    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f44bcc3-3f40-403f-bcc8-791f5472340b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c\" (UID: \"3f44bcc3-3f40-403f-bcc8-791f5472340b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:19.041278 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.041123    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f44bcc3-3f40-403f-bcc8-791f5472340b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c\" (UID: \"3f44bcc3-3f40-403f-bcc8-791f5472340b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:19.142382 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.142358    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f44bcc3-3f40-403f-bcc8-791f5472340b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c\" (UID: \"3f44bcc3-3f40-403f-bcc8-791f5472340b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:19.142506 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.142393    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f44bcc3-3f40-403f-bcc8-791f5472340b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c\" (UID: \"3f44bcc3-3f40-403f-bcc8-791f5472340b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:19.142506 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.142418    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qqbc\" (UniqueName: \"kubernetes.io/projected/3f44bcc3-3f40-403f-bcc8-791f5472340b-kube-api-access-5qqbc\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c\" (UID: \"3f44bcc3-3f40-403f-bcc8-791f5472340b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:19.142506 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.142462    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f44bcc3-3f40-403f-bcc8-791f5472340b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c\" (UID: \"3f44bcc3-3f40-403f-bcc8-791f5472340b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:19.142506 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.142487    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f44bcc3-3f40-403f-bcc8-791f5472340b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c\" (UID: \"3f44bcc3-3f40-403f-bcc8-791f5472340b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:19.142757 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.142512    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f44bcc3-3f40-403f-bcc8-791f5472340b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c\" (UID: \"3f44bcc3-3f40-403f-bcc8-791f5472340b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:19.142757 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.142542    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f44bcc3-3f40-403f-bcc8-791f5472340b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c\" (UID: \"3f44bcc3-3f40-403f-bcc8-791f5472340b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:19.142757 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.142572    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f44bcc3-3f40-403f-bcc8-791f5472340b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c\" (UID: \"3f44bcc3-3f40-403f-bcc8-791f5472340b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:19.142757 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.142606    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f44bcc3-3f40-403f-bcc8-791f5472340b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c\" (UID: \"3f44bcc3-3f40-403f-bcc8-791f5472340b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:19.142976 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.142876    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f44bcc3-3f40-403f-bcc8-791f5472340b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c\" (UID: \"3f44bcc3-3f40-403f-bcc8-791f5472340b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:19.143123 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.143102    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f44bcc3-3f40-403f-bcc8-791f5472340b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c\" (UID: \"3f44bcc3-3f40-403f-bcc8-791f5472340b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:19.143184 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.143101    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f44bcc3-3f40-403f-bcc8-791f5472340b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c\" (UID: \"3f44bcc3-3f40-403f-bcc8-791f5472340b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:19.143222 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.143177    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f44bcc3-3f40-403f-bcc8-791f5472340b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c\" (UID: \"3f44bcc3-3f40-403f-bcc8-791f5472340b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:19.143586 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.143568    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f44bcc3-3f40-403f-bcc8-791f5472340b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c\" (UID: \"3f44bcc3-3f40-403f-bcc8-791f5472340b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:19.144645 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.144628    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f44bcc3-3f40-403f-bcc8-791f5472340b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c\" (UID: \"3f44bcc3-3f40-403f-bcc8-791f5472340b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:19.144917 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.144900    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f44bcc3-3f40-403f-bcc8-791f5472340b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c\" (UID: \"3f44bcc3-3f40-403f-bcc8-791f5472340b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:19.151515 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.151493    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f44bcc3-3f40-403f-bcc8-791f5472340b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c\" (UID: \"3f44bcc3-3f40-403f-bcc8-791f5472340b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:19.151682 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.151659    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qqbc\" (UniqueName: \"kubernetes.io/projected/3f44bcc3-3f40-403f-bcc8-791f5472340b-kube-api-access-5qqbc\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c\" (UID: \"3f44bcc3-3f40-403f-bcc8-791f5472340b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:19.282412 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.282326    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:19.430580 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.430468    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"]
Aug 05 17:29:19.433120 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:29:19.433095    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f44bcc3_3f40_403f_bcc8_791f5472340b.slice/crio-9562e2f5f974a87905669a38cbf3a721d60cedc072bf48b5924606af22bfafdd WatchSource:0}: Error finding container 9562e2f5f974a87905669a38cbf3a721d60cedc072bf48b5924606af22bfafdd: Status 404 returned error can't find the container with id 9562e2f5f974a87905669a38cbf3a721d60cedc072bf48b5924606af22bfafdd
Aug 05 17:29:19.457366 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:19.457341    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c" event={"ID":"3f44bcc3-3f40-403f-bcc8-791f5472340b","Type":"ContainerStarted","Data":"9562e2f5f974a87905669a38cbf3a721d60cedc072bf48b5924606af22bfafdd"}
Aug 05 17:29:21.182118 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:21.182085    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-fbrn7"]
Aug 05 17:29:21.188623 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:21.188593    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-fbrn7"
Aug 05 17:29:21.191087 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:21.191058    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-2ldj9"]
Aug 05 17:29:21.192963 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:21.192940    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Aug 05 17:29:21.193538 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:21.193518    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-4746k\""
Aug 05 17:29:21.194309 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:21.194291    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2ldj9"
Aug 05 17:29:21.196232 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:21.196210    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-fbrn7"]
Aug 05 17:29:21.198261 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:21.198010    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Aug 05 17:29:21.198261 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:21.198208    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-xx9sm\""
Aug 05 17:29:21.207013 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:21.206968    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-2ldj9"]
Aug 05 17:29:21.261623 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:21.261586    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9409ab12-e7ca-44e9-84a5-4bc4d3f46916-cert\") pod \"kserve-controller-manager-d6f4b65b8-fbrn7\" (UID: \"9409ab12-e7ca-44e9-84a5-4bc4d3f46916\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-fbrn7"
Aug 05 17:29:21.261826 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:21.261684    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/56c44a81-44f1-451a-a9ed-add009f98ca7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-2ldj9\" (UID: \"56c44a81-44f1-451a-a9ed-add009f98ca7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2ldj9"
Aug 05 17:29:21.261826 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:21.261799    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8knh5\" (UniqueName: \"kubernetes.io/projected/9409ab12-e7ca-44e9-84a5-4bc4d3f46916-kube-api-access-8knh5\") pod \"kserve-controller-manager-d6f4b65b8-fbrn7\" (UID: \"9409ab12-e7ca-44e9-84a5-4bc4d3f46916\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-fbrn7"
Aug 05 17:29:21.261934 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:21.261834    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkrws\" (UniqueName: \"kubernetes.io/projected/56c44a81-44f1-451a-a9ed-add009f98ca7-kube-api-access-qkrws\") pod \"llmisvc-controller-manager-777c6bd7d5-2ldj9\" (UID: \"56c44a81-44f1-451a-a9ed-add009f98ca7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2ldj9"
Aug 05 17:29:21.363254 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:21.363220    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9409ab12-e7ca-44e9-84a5-4bc4d3f46916-cert\") pod \"kserve-controller-manager-d6f4b65b8-fbrn7\" (UID: \"9409ab12-e7ca-44e9-84a5-4bc4d3f46916\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-fbrn7"
Aug 05 17:29:21.363254 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:21.363266    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/56c44a81-44f1-451a-a9ed-add009f98ca7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-2ldj9\" (UID: \"56c44a81-44f1-451a-a9ed-add009f98ca7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2ldj9"
Aug 05 17:29:21.363496 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:21.363309    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8knh5\" (UniqueName: \"kubernetes.io/projected/9409ab12-e7ca-44e9-84a5-4bc4d3f46916-kube-api-access-8knh5\") pod \"kserve-controller-manager-d6f4b65b8-fbrn7\" (UID: \"9409ab12-e7ca-44e9-84a5-4bc4d3f46916\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-fbrn7"
Aug 05 17:29:21.363496 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:21.363330    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkrws\" (UniqueName: \"kubernetes.io/projected/56c44a81-44f1-451a-a9ed-add009f98ca7-kube-api-access-qkrws\") pod \"llmisvc-controller-manager-777c6bd7d5-2ldj9\" (UID: \"56c44a81-44f1-451a-a9ed-add009f98ca7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2ldj9"
Aug 05 17:29:21.363496 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:29:21.363446    2550 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Aug 05 17:29:21.363644 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:29:21.363533    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56c44a81-44f1-451a-a9ed-add009f98ca7-cert podName:56c44a81-44f1-451a-a9ed-add009f98ca7 nodeName:}" failed. No retries permitted until 2026-08-05 17:29:21.863508796 +0000 UTC m=+468.845086225 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/56c44a81-44f1-451a-a9ed-add009f98ca7-cert") pod "llmisvc-controller-manager-777c6bd7d5-2ldj9" (UID: "56c44a81-44f1-451a-a9ed-add009f98ca7") : secret "llmisvc-webhook-server-cert" not found
Aug 05 17:29:21.365922 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:21.365900    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9409ab12-e7ca-44e9-84a5-4bc4d3f46916-cert\") pod \"kserve-controller-manager-d6f4b65b8-fbrn7\" (UID: \"9409ab12-e7ca-44e9-84a5-4bc4d3f46916\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-fbrn7"
Aug 05 17:29:21.372771 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:21.372745    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8knh5\" (UniqueName: \"kubernetes.io/projected/9409ab12-e7ca-44e9-84a5-4bc4d3f46916-kube-api-access-8knh5\") pod \"kserve-controller-manager-d6f4b65b8-fbrn7\" (UID: \"9409ab12-e7ca-44e9-84a5-4bc4d3f46916\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-fbrn7"
Aug 05 17:29:21.373259 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:21.373237    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkrws\" (UniqueName: \"kubernetes.io/projected/56c44a81-44f1-451a-a9ed-add009f98ca7-kube-api-access-qkrws\") pod \"llmisvc-controller-manager-777c6bd7d5-2ldj9\" (UID: \"56c44a81-44f1-451a-a9ed-add009f98ca7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2ldj9"
Aug 05 17:29:21.451536 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:21.451458    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"
Aug 05 17:29:21.506160 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:21.506130    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-fbrn7"
Aug 05 17:29:21.868484 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:21.868402    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/56c44a81-44f1-451a-a9ed-add009f98ca7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-2ldj9\" (UID: \"56c44a81-44f1-451a-a9ed-add009f98ca7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2ldj9"
Aug 05 17:29:21.870943 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:21.870915    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/56c44a81-44f1-451a-a9ed-add009f98ca7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-2ldj9\" (UID: \"56c44a81-44f1-451a-a9ed-add009f98ca7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2ldj9"
Aug 05 17:29:22.113834 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:22.113799    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2ldj9"
Aug 05 17:29:22.418508 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:22.418451    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 17:29:22.419008 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:22.418553    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 17:29:22.419008 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:22.418600    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 17:29:22.501328 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:29:22.501299    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod56c44a81_44f1_451a_a9ed_add009f98ca7.slice/crio-cf13d00aa2d2ed202d78e0690ddbcb1ada4deee2ce6ad0161518b44edd34ef8f WatchSource:0}: Error finding container cf13d00aa2d2ed202d78e0690ddbcb1ada4deee2ce6ad0161518b44edd34ef8f: Status 404 returned error can't find the container with id cf13d00aa2d2ed202d78e0690ddbcb1ada4deee2ce6ad0161518b44edd34ef8f
Aug 05 17:29:22.506000 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:22.505978    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-2ldj9"]
Aug 05 17:29:22.519787 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:22.519760    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-fbrn7"]
Aug 05 17:29:22.522642 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:29:22.522614    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9409ab12_e7ca_44e9_84a5_4bc4d3f46916.slice/crio-9cc6da9ee6fcc96cdc22b4de7b86d54b9d7b88ee4e795156d1da077e6c44828a WatchSource:0}: Error finding container 9cc6da9ee6fcc96cdc22b4de7b86d54b9d7b88ee4e795156d1da077e6c44828a: Status 404 returned error can't find the container with id 9cc6da9ee6fcc96cdc22b4de7b86d54b9d7b88ee4e795156d1da077e6c44828a
Aug 05 17:29:22.942977 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:22.942947    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"]
Aug 05 17:29:22.943152 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:22.943111    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm" podUID="03ec8c9d-7fb8-428c-a6fe-1112c125cfbf" containerName="discovery" containerID="cri-o://1f168aaa326049837d1b90b38339ee6e971917e9c0e76585779402e91706f654" gracePeriod=30
Aug 05 17:29:23.249379 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.249353    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"
Aug 05 17:29:23.279659 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.279623    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-istio-csr-dns-cert\") pod \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\" (UID: \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\") "
Aug 05 17:29:23.279847 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.279677    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-istio-token\") pod \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\" (UID: \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\") "
Aug 05 17:29:23.279847 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.279733    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2rfgk\" (UniqueName: \"kubernetes.io/projected/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-kube-api-access-2rfgk\") pod \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\" (UID: \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\") "
Aug 05 17:29:23.279847 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.279772    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-local-certs\") pod \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\" (UID: \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\") "
Aug 05 17:29:23.279847 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.279816    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-cacerts\") pod \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\" (UID: \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\") "
Aug 05 17:29:23.280074 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.279887    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-istio-csr-ca-configmap\") pod \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\" (UID: \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\") "
Aug 05 17:29:23.280074 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.279916    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-istio-kubeconfig\") pod \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\" (UID: \"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf\") "
Aug 05 17:29:23.282899 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.282669    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "03ec8c9d-7fb8-428c-a6fe-1112c125cfbf" (UID: "03ec8c9d-7fb8-428c-a6fe-1112c125cfbf"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:29:23.283120 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.283083    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "03ec8c9d-7fb8-428c-a6fe-1112c125cfbf" (UID: "03ec8c9d-7fb8-428c-a6fe-1112c125cfbf"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:29:23.283384 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.283345    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-istio-token" (OuterVolumeSpecName: "istio-token") pod "03ec8c9d-7fb8-428c-a6fe-1112c125cfbf" (UID: "03ec8c9d-7fb8-428c-a6fe-1112c125cfbf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:29:23.285393 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.284596    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "03ec8c9d-7fb8-428c-a6fe-1112c125cfbf" (UID: "03ec8c9d-7fb8-428c-a6fe-1112c125cfbf"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:29:23.285502 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.285457    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-local-certs" (OuterVolumeSpecName: "local-certs") pod "03ec8c9d-7fb8-428c-a6fe-1112c125cfbf" (UID: "03ec8c9d-7fb8-428c-a6fe-1112c125cfbf"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:29:23.285620 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.285591    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-kube-api-access-2rfgk" (OuterVolumeSpecName: "kube-api-access-2rfgk") pod "03ec8c9d-7fb8-428c-a6fe-1112c125cfbf" (UID: "03ec8c9d-7fb8-428c-a6fe-1112c125cfbf"). InnerVolumeSpecName "kube-api-access-2rfgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:29:23.286343 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.286286    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-cacerts" (OuterVolumeSpecName: "cacerts") pod "03ec8c9d-7fb8-428c-a6fe-1112c125cfbf" (UID: "03ec8c9d-7fb8-428c-a6fe-1112c125cfbf"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:29:23.380816 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.380676    2550 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-istio-csr-ca-configmap\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:29:23.380816 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.380726    2550 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-istio-kubeconfig\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:29:23.380816 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.380742    2550 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-istio-csr-dns-cert\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:29:23.380816 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.380755    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-istio-token\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:29:23.380816 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.380769    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rfgk\" (UniqueName: \"kubernetes.io/projected/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-kube-api-access-2rfgk\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:29:23.380816 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.380781    2550 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-local-certs\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:29:23.380816 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.380794    2550 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf-cacerts\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:29:23.479779 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.479694    2550 generic.go:356] "Generic (PLEG): container finished" podID="03ec8c9d-7fb8-428c-a6fe-1112c125cfbf" containerID="1f168aaa326049837d1b90b38339ee6e971917e9c0e76585779402e91706f654" exitCode=0
Aug 05 17:29:23.479779 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.479733    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm" event={"ID":"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf","Type":"ContainerDied","Data":"1f168aaa326049837d1b90b38339ee6e971917e9c0e76585779402e91706f654"}
Aug 05 17:29:23.479779 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.479756    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"
Aug 05 17:29:23.480329 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.479798    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm" event={"ID":"03ec8c9d-7fb8-428c-a6fe-1112c125cfbf","Type":"ContainerDied","Data":"4fc4d015bc50ce10de31618ba387fdbdc5701361d67c5419e68d9f6a196f8c44"}
Aug 05 17:29:23.480329 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.479826    2550 scope.go:117] "RemoveContainer" containerID="1f168aaa326049837d1b90b38339ee6e971917e9c0e76585779402e91706f654"
Aug 05 17:29:23.482392 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.482352    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c" event={"ID":"3f44bcc3-3f40-403f-bcc8-791f5472340b","Type":"ContainerStarted","Data":"6f64c3d4798b56bcf7cade52c135c203abf7d101459a1ba981e430a2b5faf0f3"}
Aug 05 17:29:23.484349 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.484322    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-fbrn7" event={"ID":"9409ab12-e7ca-44e9-84a5-4bc4d3f46916","Type":"ContainerStarted","Data":"9cc6da9ee6fcc96cdc22b4de7b86d54b9d7b88ee4e795156d1da077e6c44828a"}
Aug 05 17:29:23.485799 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.485771    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2ldj9" event={"ID":"56c44a81-44f1-451a-a9ed-add009f98ca7","Type":"ContainerStarted","Data":"cf13d00aa2d2ed202d78e0690ddbcb1ada4deee2ce6ad0161518b44edd34ef8f"}
Aug 05 17:29:23.509584 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.509422    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c" podStartSLOduration=2.526351246 podStartE2EDuration="5.509396608s" totalImagesPullingTime="2.983045362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:29:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:29:23.508316794 +0000 UTC m=+470.489894237" watchObservedRunningTime="2026-08-05 17:29:23.509396608 +0000 UTC m=+470.490974038"
Aug 05 17:29:23.526148 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.526106    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"]
Aug 05 17:29:23.533409 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.533366    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5l5nm"]
Aug 05 17:29:23.703040 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.702821    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03ec8c9d-7fb8-428c-a6fe-1112c125cfbf" path="/var/lib/kubelet/pods/03ec8c9d-7fb8-428c-a6fe-1112c125cfbf/volumes"
Aug 05 17:29:23.706493 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.706238    2550 scope.go:117] "RemoveContainer" containerID="1f168aaa326049837d1b90b38339ee6e971917e9c0e76585779402e91706f654"
Aug 05 17:29:23.706603 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:29:23.706577    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f168aaa326049837d1b90b38339ee6e971917e9c0e76585779402e91706f654\": container with ID starting with 1f168aaa326049837d1b90b38339ee6e971917e9c0e76585779402e91706f654 not found: ID does not exist" containerID="1f168aaa326049837d1b90b38339ee6e971917e9c0e76585779402e91706f654"
Aug 05 17:29:23.706659 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:23.706619    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f168aaa326049837d1b90b38339ee6e971917e9c0e76585779402e91706f654"} err="failed to get container status \"1f168aaa326049837d1b90b38339ee6e971917e9c0e76585779402e91706f654\": rpc error: code = NotFound desc = could not find container \"1f168aaa326049837d1b90b38339ee6e971917e9c0e76585779402e91706f654\": container with ID starting with 1f168aaa326049837d1b90b38339ee6e971917e9c0e76585779402e91706f654 not found: ID does not exist"
Aug 05 17:29:24.283565 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:24.283523    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:24.288991 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:24.288966    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:24.492438 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:24.492405    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:24.493591 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:24.493571    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c"
Aug 05 17:29:25.499392 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:25.499350    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-fbrn7" event={"ID":"9409ab12-e7ca-44e9-84a5-4bc4d3f46916","Type":"ContainerStarted","Data":"83f2763a5081b9f1d3c8b314da52b234aaefb0728d25350fd1b83ffc997056e4"}
Aug 05 17:29:25.518879 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:25.518809    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-fbrn7" podStartSLOduration=2.01838968 podStartE2EDuration="4.518784387s" totalImagesPullingTime="2.500394707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:29:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:29:25.516927353 +0000 UTC m=+472.498504787" watchObservedRunningTime="2026-08-05 17:29:25.518784387 +0000 UTC m=+472.500361818"
Aug 05 17:29:26.505046 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:26.505008    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2ldj9" event={"ID":"56c44a81-44f1-451a-a9ed-add009f98ca7","Type":"ContainerStarted","Data":"508e318a1e0b4fb74b8b33783cbe8905ca0167457b05325ccb3788e7fecfd38c"}
Aug 05 17:29:26.505693 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:26.505222    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2ldj9"
Aug 05 17:29:26.505693 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:26.505364    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-fbrn7"
Aug 05 17:29:26.523353 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:26.523248    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2ldj9" podStartSLOduration=2.401295743 podStartE2EDuration="5.523231975s" totalImagesPullingTime="3.121936232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:29:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:29:26.522098486 +0000 UTC m=+473.503675910" watchObservedRunningTime="2026-08-05 17:29:26.523231975 +0000 UTC m=+473.504809404"
Aug 05 17:29:28.452239 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:28.452206    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-bfsdk"
Aug 05 17:29:28.461181 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:28.456963    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-kxjqm"
Aug 05 17:29:57.511197 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:57.511163    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2ldj9"
Aug 05 17:29:57.514123 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:29:57.514101    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-fbrn7"
Aug 05 17:30:34.402821 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:34.402790    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q"]
Aug 05 17:30:34.403291 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:34.403064    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03ec8c9d-7fb8-428c-a6fe-1112c125cfbf" containerName="discovery"
Aug 05 17:30:34.403291 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:34.403074    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="03ec8c9d-7fb8-428c-a6fe-1112c125cfbf" containerName="discovery"
Aug 05 17:30:34.403291 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:34.403137    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="03ec8c9d-7fb8-428c-a6fe-1112c125cfbf" containerName="discovery"
Aug 05 17:30:34.405932 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:34.405915    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q"
Aug 05 17:30:34.408878 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:34.408853    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Aug 05 17:30:34.408878 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:34.408873    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Aug 05 17:30:34.409158 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:34.409145    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-tq282\""
Aug 05 17:30:34.412799 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:34.412742    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1e7dca21-3a3e-471b-a5b6-3bf940955385-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-r8g4q\" (UID: \"1e7dca21-3a3e-471b-a5b6-3bf940955385\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q"
Aug 05 17:30:34.412884 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:34.412795    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f26s9\" (UniqueName: \"kubernetes.io/projected/1e7dca21-3a3e-471b-a5b6-3bf940955385-kube-api-access-f26s9\") pod \"kuadrant-operator-controller-manager-58fc56757c-r8g4q\" (UID: \"1e7dca21-3a3e-471b-a5b6-3bf940955385\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q"
Aug 05 17:30:34.421719 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:34.421685    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q"]
Aug 05 17:30:34.513794 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:34.513766    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1e7dca21-3a3e-471b-a5b6-3bf940955385-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-r8g4q\" (UID: \"1e7dca21-3a3e-471b-a5b6-3bf940955385\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q"
Aug 05 17:30:34.513937 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:34.513807    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f26s9\" (UniqueName: \"kubernetes.io/projected/1e7dca21-3a3e-471b-a5b6-3bf940955385-kube-api-access-f26s9\") pod \"kuadrant-operator-controller-manager-58fc56757c-r8g4q\" (UID: \"1e7dca21-3a3e-471b-a5b6-3bf940955385\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q"
Aug 05 17:30:34.514075 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:34.514056    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1e7dca21-3a3e-471b-a5b6-3bf940955385-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-r8g4q\" (UID: \"1e7dca21-3a3e-471b-a5b6-3bf940955385\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q"
Aug 05 17:30:34.531172 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:34.531141    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f26s9\" (UniqueName: \"kubernetes.io/projected/1e7dca21-3a3e-471b-a5b6-3bf940955385-kube-api-access-f26s9\") pod \"kuadrant-operator-controller-manager-58fc56757c-r8g4q\" (UID: \"1e7dca21-3a3e-471b-a5b6-3bf940955385\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q"
Aug 05 17:30:34.715766 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:34.715727    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q"
Aug 05 17:30:34.845594 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:34.845564    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q"]
Aug 05 17:30:34.847338 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:30:34.847314    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e7dca21_3a3e_471b_a5b6_3bf940955385.slice/crio-6f18cc134bbdd701f7d21b38099126aa96df3a6fb885c0c1b17c8525ed9ed94e WatchSource:0}: Error finding container 6f18cc134bbdd701f7d21b38099126aa96df3a6fb885c0c1b17c8525ed9ed94e: Status 404 returned error can't find the container with id 6f18cc134bbdd701f7d21b38099126aa96df3a6fb885c0c1b17c8525ed9ed94e
Aug 05 17:30:35.775730 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:35.775678    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q" event={"ID":"1e7dca21-3a3e-471b-a5b6-3bf940955385","Type":"ContainerStarted","Data":"6f18cc134bbdd701f7d21b38099126aa96df3a6fb885c0c1b17c8525ed9ed94e"}
Aug 05 17:30:36.446235 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:36.443959    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-x7lx6"]
Aug 05 17:30:36.448523 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:36.448494    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-x7lx6"
Aug 05 17:30:36.451499 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:36.451476    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-svfgz\""
Aug 05 17:30:36.460155 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:36.460131    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-x7lx6"]
Aug 05 17:30:36.525061 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:36.525030    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9b9t8\" (UniqueName: \"kubernetes.io/projected/3812f436-4df5-4e9a-a1f9-96c356c83b1c-kube-api-access-9b9t8\") pod \"limitador-operator-controller-manager-6c549b6d85-x7lx6\" (UID: \"3812f436-4df5-4e9a-a1f9-96c356c83b1c\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-x7lx6"
Aug 05 17:30:36.626292 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:36.626259    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9b9t8\" (UniqueName: \"kubernetes.io/projected/3812f436-4df5-4e9a-a1f9-96c356c83b1c-kube-api-access-9b9t8\") pod \"limitador-operator-controller-manager-6c549b6d85-x7lx6\" (UID: \"3812f436-4df5-4e9a-a1f9-96c356c83b1c\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-x7lx6"
Aug 05 17:30:36.639084 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:36.639057    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b9t8\" (UniqueName: \"kubernetes.io/projected/3812f436-4df5-4e9a-a1f9-96c356c83b1c-kube-api-access-9b9t8\") pod \"limitador-operator-controller-manager-6c549b6d85-x7lx6\" (UID: \"3812f436-4df5-4e9a-a1f9-96c356c83b1c\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-x7lx6"
Aug 05 17:30:36.761916 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:36.761840    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-x7lx6"
Aug 05 17:30:37.639146 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:37.639120    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-x7lx6"]
Aug 05 17:30:37.641821 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:30:37.641793    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3812f436_4df5_4e9a_a1f9_96c356c83b1c.slice/crio-9d88f1b9278d8e8ea616d5bdf7db163ae3ea7f365dec5e4d6f751aa14e3e7408 WatchSource:0}: Error finding container 9d88f1b9278d8e8ea616d5bdf7db163ae3ea7f365dec5e4d6f751aa14e3e7408: Status 404 returned error can't find the container with id 9d88f1b9278d8e8ea616d5bdf7db163ae3ea7f365dec5e4d6f751aa14e3e7408
Aug 05 17:30:37.788619 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:37.788529    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-x7lx6" event={"ID":"3812f436-4df5-4e9a-a1f9-96c356c83b1c","Type":"ContainerStarted","Data":"9d88f1b9278d8e8ea616d5bdf7db163ae3ea7f365dec5e4d6f751aa14e3e7408"}
Aug 05 17:30:39.131421 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:39.131345    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-qh78n"]
Aug 05 17:30:39.135263 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:39.135238    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-qh78n"
Aug 05 17:30:39.138026 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:39.137994    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-pr5v9\""
Aug 05 17:30:39.144218 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:39.143741    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-qh78n"]
Aug 05 17:30:39.146240 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:39.146211    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lmnb\" (UniqueName: \"kubernetes.io/projected/24913a9d-22fc-451b-b7e3-25cee6d65ce1-kube-api-access-8lmnb\") pod \"authorino-operator-68d88948d8-qh78n\" (UID: \"24913a9d-22fc-451b-b7e3-25cee6d65ce1\") " pod="kuadrant-system/authorino-operator-68d88948d8-qh78n"
Aug 05 17:30:39.247390 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:39.247358    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lmnb\" (UniqueName: \"kubernetes.io/projected/24913a9d-22fc-451b-b7e3-25cee6d65ce1-kube-api-access-8lmnb\") pod \"authorino-operator-68d88948d8-qh78n\" (UID: \"24913a9d-22fc-451b-b7e3-25cee6d65ce1\") " pod="kuadrant-system/authorino-operator-68d88948d8-qh78n"
Aug 05 17:30:39.255239 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:39.255214    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lmnb\" (UniqueName: \"kubernetes.io/projected/24913a9d-22fc-451b-b7e3-25cee6d65ce1-kube-api-access-8lmnb\") pod \"authorino-operator-68d88948d8-qh78n\" (UID: \"24913a9d-22fc-451b-b7e3-25cee6d65ce1\") " pod="kuadrant-system/authorino-operator-68d88948d8-qh78n"
Aug 05 17:30:39.448161 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:39.448127    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-qh78n"
Aug 05 17:30:39.597514 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:39.597481    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-qh78n"]
Aug 05 17:30:39.600416 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:30:39.600395    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24913a9d_22fc_451b_b7e3_25cee6d65ce1.slice/crio-29a807c1e585b3b7e77233066658ac49ced2b3d93e52a5521587e546460b171b WatchSource:0}: Error finding container 29a807c1e585b3b7e77233066658ac49ced2b3d93e52a5521587e546460b171b: Status 404 returned error can't find the container with id 29a807c1e585b3b7e77233066658ac49ced2b3d93e52a5521587e546460b171b
Aug 05 17:30:39.799638 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:39.799542    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q" event={"ID":"1e7dca21-3a3e-471b-a5b6-3bf940955385","Type":"ContainerStarted","Data":"949cd9937c4abe4b32bd9f835712da2ca1dc424aaf1d4827e211853562293188"}
Aug 05 17:30:39.799638 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:39.799610    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q"
Aug 05 17:30:39.800729 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:39.800709    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-qh78n" event={"ID":"24913a9d-22fc-451b-b7e3-25cee6d65ce1","Type":"ContainerStarted","Data":"29a807c1e585b3b7e77233066658ac49ced2b3d93e52a5521587e546460b171b"}
Aug 05 17:30:40.816496 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:40.816450    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-x7lx6" event={"ID":"3812f436-4df5-4e9a-a1f9-96c356c83b1c","Type":"ContainerStarted","Data":"e183e67581b334c1e0952ac55dbd20dc1d26bb1b7dd16b08abfc9f107178b5c3"}
Aug 05 17:30:40.832831 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:40.832770    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q" podStartSLOduration=2.407688412 podStartE2EDuration="6.832754605s" totalImagesPullingTime="4.425066193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:30:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:30:39.821075352 +0000 UTC m=+546.802652781" watchObservedRunningTime="2026-08-05 17:30:40.832754605 +0000 UTC m=+547.814332035"
Aug 05 17:30:41.822162 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:41.822126    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-qh78n" event={"ID":"24913a9d-22fc-451b-b7e3-25cee6d65ce1","Type":"ContainerStarted","Data":"f05a4ba4abbefe73ead54c9e256f89484a1e1356402d768804def1376a2eb06a"}
Aug 05 17:30:41.822622 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:41.822303    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d88948d8-qh78n"
Aug 05 17:30:41.822622 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:41.822345    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-x7lx6"
Aug 05 17:30:41.840404 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:41.840350    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d88948d8-qh78n" podStartSLOduration=1.361179127 podStartE2EDuration="2.840333745s" totalImagesPullingTime="1.479154618s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:30:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:30:41.838756209 +0000 UTC m=+548.820333619" watchObservedRunningTime="2026-08-05 17:30:41.840333745 +0000 UTC m=+548.821911173"
Aug 05 17:30:41.840825 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:41.840801    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-x7lx6" podStartSLOduration=3.229506759 podStartE2EDuration="5.840793806s" totalImagesPullingTime="2.611287047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:30:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:30:40.832011884 +0000 UTC m=+547.813589304" watchObservedRunningTime="2026-08-05 17:30:41.840793806 +0000 UTC m=+548.822371235"
Aug 05 17:30:50.819126 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:50.819095    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q"
Aug 05 17:30:52.556867 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.556832    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q"]
Aug 05 17:30:52.557272 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.557019    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q" podUID="1e7dca21-3a3e-471b-a5b6-3bf940955385" containerName="manager" containerID="cri-o://949cd9937c4abe4b32bd9f835712da2ca1dc424aaf1d4827e211853562293188" gracePeriod=2
Aug 05 17:30:52.570550 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.570515    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q"]
Aug 05 17:30:52.579019 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.578992    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-x7lx6"]
Aug 05 17:30:52.579267 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.579205    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-x7lx6" podUID="3812f436-4df5-4e9a-a1f9-96c356c83b1c" containerName="manager" containerID="cri-o://e183e67581b334c1e0952ac55dbd20dc1d26bb1b7dd16b08abfc9f107178b5c3" gracePeriod=2
Aug 05 17:30:52.581650 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:30:52.581617    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-r8g4q\" is forbidden: User \"system:node:ip-10-0-134-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-46.ec2.internal' and this object" podUID="1e7dca21-3a3e-471b-a5b6-3bf940955385" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q"
Aug 05 17:30:52.583189 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.582576    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vk4xt"]
Aug 05 17:30:52.583189 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.583026    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e7dca21-3a3e-471b-a5b6-3bf940955385" containerName="manager"
Aug 05 17:30:52.583189 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.583045    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e7dca21-3a3e-471b-a5b6-3bf940955385" containerName="manager"
Aug 05 17:30:52.583189 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.583126    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e7dca21-3a3e-471b-a5b6-3bf940955385" containerName="manager"
Aug 05 17:30:52.586218 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.586198    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-x7lx6"
Aug 05 17:30:52.586312 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.586235    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-x7lx6"]
Aug 05 17:30:52.586363 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.586308    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vk4xt"
Aug 05 17:30:52.591338 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:30:52.591294    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-x7lx6\" is forbidden: User \"system:node:ip-10-0-134-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-46.ec2.internal' and this object" podUID="3812f436-4df5-4e9a-a1f9-96c356c83b1c" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-x7lx6"
Aug 05 17:30:52.598679 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.598625    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vk4xt"]
Aug 05 17:30:52.603282 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.603256    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8qt45"]
Aug 05 17:30:52.603616 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.603601    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3812f436-4df5-4e9a-a1f9-96c356c83b1c" containerName="manager"
Aug 05 17:30:52.603684 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.603618    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="3812f436-4df5-4e9a-a1f9-96c356c83b1c" containerName="manager"
Aug 05 17:30:52.603860 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.603839    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="3812f436-4df5-4e9a-a1f9-96c356c83b1c" containerName="manager"
Aug 05 17:30:52.606770 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.606748    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8qt45"
Aug 05 17:30:52.612052 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:30:52.612017    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-r8g4q\" is forbidden: User \"system:node:ip-10-0-134-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-46.ec2.internal' and this object" podUID="1e7dca21-3a3e-471b-a5b6-3bf940955385" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q"
Aug 05 17:30:52.613891 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:30:52.613867    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-x7lx6\" is forbidden: User \"system:node:ip-10-0-134-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-46.ec2.internal' and this object" podUID="3812f436-4df5-4e9a-a1f9-96c356c83b1c" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-x7lx6"
Aug 05 17:30:52.618762 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.618736    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8qt45"]
Aug 05 17:30:52.631552 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:30:52.631517    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-r8g4q\" is forbidden: User \"system:node:ip-10-0-134-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-46.ec2.internal' and this object" podUID="1e7dca21-3a3e-471b-a5b6-3bf940955385" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q"
Aug 05 17:30:52.646993 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.646955    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ad7f602-67a6-4ecf-8e1f-eded3cfb7722-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-vk4xt\" (UID: \"0ad7f602-67a6-4ecf-8e1f-eded3cfb7722\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vk4xt"
Aug 05 17:30:52.647252 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.647085    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5n5h\" (UniqueName: \"kubernetes.io/projected/0ad7f602-67a6-4ecf-8e1f-eded3cfb7722-kube-api-access-l5n5h\") pod \"kuadrant-operator-controller-manager-58fc56757c-vk4xt\" (UID: \"0ad7f602-67a6-4ecf-8e1f-eded3cfb7722\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vk4xt"
Aug 05 17:30:52.647923 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.647410    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92khm\" (UniqueName: \"kubernetes.io/projected/e50c044b-cb40-4340-870e-2fac665bbdf7-kube-api-access-92khm\") pod \"limitador-operator-controller-manager-6c549b6d85-8qt45\" (UID: \"e50c044b-cb40-4340-870e-2fac665bbdf7\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8qt45"
Aug 05 17:30:52.748303 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.748261    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92khm\" (UniqueName: \"kubernetes.io/projected/e50c044b-cb40-4340-870e-2fac665bbdf7-kube-api-access-92khm\") pod \"limitador-operator-controller-manager-6c549b6d85-8qt45\" (UID: \"e50c044b-cb40-4340-870e-2fac665bbdf7\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8qt45"
Aug 05 17:30:52.748471 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.748332    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ad7f602-67a6-4ecf-8e1f-eded3cfb7722-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-vk4xt\" (UID: \"0ad7f602-67a6-4ecf-8e1f-eded3cfb7722\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vk4xt"
Aug 05 17:30:52.748471 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.748397    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5n5h\" (UniqueName: \"kubernetes.io/projected/0ad7f602-67a6-4ecf-8e1f-eded3cfb7722-kube-api-access-l5n5h\") pod \"kuadrant-operator-controller-manager-58fc56757c-vk4xt\" (UID: \"0ad7f602-67a6-4ecf-8e1f-eded3cfb7722\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vk4xt"
Aug 05 17:30:52.749198 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.749170    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ad7f602-67a6-4ecf-8e1f-eded3cfb7722-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-vk4xt\" (UID: \"0ad7f602-67a6-4ecf-8e1f-eded3cfb7722\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vk4xt"
Aug 05 17:30:52.759691 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.759639    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92khm\" (UniqueName: \"kubernetes.io/projected/e50c044b-cb40-4340-870e-2fac665bbdf7-kube-api-access-92khm\") pod \"limitador-operator-controller-manager-6c549b6d85-8qt45\" (UID: \"e50c044b-cb40-4340-870e-2fac665bbdf7\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8qt45"
Aug 05 17:30:52.763398 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.761559    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5n5h\" (UniqueName: \"kubernetes.io/projected/0ad7f602-67a6-4ecf-8e1f-eded3cfb7722-kube-api-access-l5n5h\") pod \"kuadrant-operator-controller-manager-58fc56757c-vk4xt\" (UID: \"0ad7f602-67a6-4ecf-8e1f-eded3cfb7722\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vk4xt"
Aug 05 17:30:52.806309 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.806283    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q"
Aug 05 17:30:52.808377 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:30:52.808345    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-x7lx6\" is forbidden: User \"system:node:ip-10-0-134-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-46.ec2.internal' and this object" podUID="3812f436-4df5-4e9a-a1f9-96c356c83b1c" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-x7lx6"
Aug 05 17:30:52.810115 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:30:52.810090    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-r8g4q\" is forbidden: User \"system:node:ip-10-0-134-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-46.ec2.internal' and this object" podUID="1e7dca21-3a3e-471b-a5b6-3bf940955385" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q"
Aug 05 17:30:52.814440 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.814417    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-x7lx6"
Aug 05 17:30:52.816285 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:30:52.816262    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-x7lx6\" is forbidden: User \"system:node:ip-10-0-134-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-46.ec2.internal' and this object" podUID="3812f436-4df5-4e9a-a1f9-96c356c83b1c" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-x7lx6"
Aug 05 17:30:52.817954 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:30:52.817933    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-r8g4q\" is forbidden: User \"system:node:ip-10-0-134-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-46.ec2.internal' and this object" podUID="1e7dca21-3a3e-471b-a5b6-3bf940955385" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q"
Aug 05 17:30:52.828904 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.828883    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d88948d8-qh78n"
Aug 05 17:30:52.847119 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:30:52.847078    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-r8g4q\" is forbidden: User \"system:node:ip-10-0-134-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-46.ec2.internal' and this object" podUID="1e7dca21-3a3e-471b-a5b6-3bf940955385" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q"
Aug 05 17:30:52.849296 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:30:52.849274    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-x7lx6\" is forbidden: User \"system:node:ip-10-0-134-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-46.ec2.internal' and this object" podUID="3812f436-4df5-4e9a-a1f9-96c356c83b1c" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-x7lx6"
Aug 05 17:30:52.849391 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.849369    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9b9t8\" (UniqueName: \"kubernetes.io/projected/3812f436-4df5-4e9a-a1f9-96c356c83b1c-kube-api-access-9b9t8\") pod \"3812f436-4df5-4e9a-a1f9-96c356c83b1c\" (UID: \"3812f436-4df5-4e9a-a1f9-96c356c83b1c\") "
Aug 05 17:30:52.849470 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.849457    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f26s9\" (UniqueName: \"kubernetes.io/projected/1e7dca21-3a3e-471b-a5b6-3bf940955385-kube-api-access-f26s9\") pod \"1e7dca21-3a3e-471b-a5b6-3bf940955385\" (UID: \"1e7dca21-3a3e-471b-a5b6-3bf940955385\") "
Aug 05 17:30:52.849530 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.849518    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1e7dca21-3a3e-471b-a5b6-3bf940955385-extensions-socket-volume\") pod \"1e7dca21-3a3e-471b-a5b6-3bf940955385\" (UID: \"1e7dca21-3a3e-471b-a5b6-3bf940955385\") "
Aug 05 17:30:52.850003 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.849977    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e7dca21-3a3e-471b-a5b6-3bf940955385-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1e7dca21-3a3e-471b-a5b6-3bf940955385" (UID: "1e7dca21-3a3e-471b-a5b6-3bf940955385"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:30:52.851546 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.851525    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e7dca21-3a3e-471b-a5b6-3bf940955385-kube-api-access-f26s9" (OuterVolumeSpecName: "kube-api-access-f26s9") pod "1e7dca21-3a3e-471b-a5b6-3bf940955385" (UID: "1e7dca21-3a3e-471b-a5b6-3bf940955385"). InnerVolumeSpecName "kube-api-access-f26s9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:30:52.851604 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.851549    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3812f436-4df5-4e9a-a1f9-96c356c83b1c-kube-api-access-9b9t8" (OuterVolumeSpecName: "kube-api-access-9b9t8") pod "3812f436-4df5-4e9a-a1f9-96c356c83b1c" (UID: "3812f436-4df5-4e9a-a1f9-96c356c83b1c"). InnerVolumeSpecName "kube-api-access-9b9t8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:30:52.869650 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.869616    2550 generic.go:356] "Generic (PLEG): container finished" podID="1e7dca21-3a3e-471b-a5b6-3bf940955385" containerID="949cd9937c4abe4b32bd9f835712da2ca1dc424aaf1d4827e211853562293188" exitCode=0
Aug 05 17:30:52.869832 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.869657    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q"
Aug 05 17:30:52.869832 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.869732    2550 scope.go:117] "RemoveContainer" containerID="949cd9937c4abe4b32bd9f835712da2ca1dc424aaf1d4827e211853562293188"
Aug 05 17:30:52.871255 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.871236    2550 generic.go:356] "Generic (PLEG): container finished" podID="3812f436-4df5-4e9a-a1f9-96c356c83b1c" containerID="e183e67581b334c1e0952ac55dbd20dc1d26bb1b7dd16b08abfc9f107178b5c3" exitCode=0
Aug 05 17:30:52.871339 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.871273    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-x7lx6"
Aug 05 17:30:52.872026 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:30:52.871889    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-x7lx6\" is forbidden: User \"system:node:ip-10-0-134-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-46.ec2.internal' and this object" podUID="3812f436-4df5-4e9a-a1f9-96c356c83b1c" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-x7lx6"
Aug 05 17:30:52.873755 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:30:52.873732    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-r8g4q\" is forbidden: User \"system:node:ip-10-0-134-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-46.ec2.internal' and this object" podUID="1e7dca21-3a3e-471b-a5b6-3bf940955385" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q"
Aug 05 17:30:52.875683 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:30:52.875527    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-x7lx6\" is forbidden: User \"system:node:ip-10-0-134-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-46.ec2.internal' and this object" podUID="3812f436-4df5-4e9a-a1f9-96c356c83b1c" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-x7lx6"
Aug 05 17:30:52.877289 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:30:52.877257    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-r8g4q\" is forbidden: User \"system:node:ip-10-0-134-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-46.ec2.internal' and this object" podUID="1e7dca21-3a3e-471b-a5b6-3bf940955385" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q"
Aug 05 17:30:52.883991 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.883974    2550 scope.go:117] "RemoveContainer" containerID="949cd9937c4abe4b32bd9f835712da2ca1dc424aaf1d4827e211853562293188"
Aug 05 17:30:52.884435 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:30:52.884406    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"949cd9937c4abe4b32bd9f835712da2ca1dc424aaf1d4827e211853562293188\": container with ID starting with 949cd9937c4abe4b32bd9f835712da2ca1dc424aaf1d4827e211853562293188 not found: ID does not exist" containerID="949cd9937c4abe4b32bd9f835712da2ca1dc424aaf1d4827e211853562293188"
Aug 05 17:30:52.884522 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.884437    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"949cd9937c4abe4b32bd9f835712da2ca1dc424aaf1d4827e211853562293188"} err="failed to get container status \"949cd9937c4abe4b32bd9f835712da2ca1dc424aaf1d4827e211853562293188\": rpc error: code = NotFound desc = could not find container \"949cd9937c4abe4b32bd9f835712da2ca1dc424aaf1d4827e211853562293188\": container with ID starting with 949cd9937c4abe4b32bd9f835712da2ca1dc424aaf1d4827e211853562293188 not found: ID does not exist"
Aug 05 17:30:52.884522 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.884460    2550 scope.go:117] "RemoveContainer" containerID="e183e67581b334c1e0952ac55dbd20dc1d26bb1b7dd16b08abfc9f107178b5c3"
Aug 05 17:30:52.886731 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:30:52.886686    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-r8g4q\" is forbidden: User \"system:node:ip-10-0-134-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-46.ec2.internal' and this object" podUID="1e7dca21-3a3e-471b-a5b6-3bf940955385" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q"
Aug 05 17:30:52.888398 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:30:52.888377    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-x7lx6\" is forbidden: User \"system:node:ip-10-0-134-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-46.ec2.internal' and this object" podUID="3812f436-4df5-4e9a-a1f9-96c356c83b1c" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-x7lx6"
Aug 05 17:30:52.890166 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:30:52.890146    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-x7lx6\" is forbidden: User \"system:node:ip-10-0-134-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-46.ec2.internal' and this object" podUID="3812f436-4df5-4e9a-a1f9-96c356c83b1c" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-x7lx6"
Aug 05 17:30:52.891766 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:30:52.891742    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-r8g4q\" is forbidden: User \"system:node:ip-10-0-134-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-46.ec2.internal' and this object" podUID="1e7dca21-3a3e-471b-a5b6-3bf940955385" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q"
Aug 05 17:30:52.892100 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.892078    2550 scope.go:117] "RemoveContainer" containerID="e183e67581b334c1e0952ac55dbd20dc1d26bb1b7dd16b08abfc9f107178b5c3"
Aug 05 17:30:52.892344 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:30:52.892326    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e183e67581b334c1e0952ac55dbd20dc1d26bb1b7dd16b08abfc9f107178b5c3\": container with ID starting with e183e67581b334c1e0952ac55dbd20dc1d26bb1b7dd16b08abfc9f107178b5c3 not found: ID does not exist" containerID="e183e67581b334c1e0952ac55dbd20dc1d26bb1b7dd16b08abfc9f107178b5c3"
Aug 05 17:30:52.892388 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.892350    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e183e67581b334c1e0952ac55dbd20dc1d26bb1b7dd16b08abfc9f107178b5c3"} err="failed to get container status \"e183e67581b334c1e0952ac55dbd20dc1d26bb1b7dd16b08abfc9f107178b5c3\": rpc error: code = NotFound desc = could not find container \"e183e67581b334c1e0952ac55dbd20dc1d26bb1b7dd16b08abfc9f107178b5c3\": container with ID starting with e183e67581b334c1e0952ac55dbd20dc1d26bb1b7dd16b08abfc9f107178b5c3 not found: ID does not exist"
Aug 05 17:30:52.950671 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.950630    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f26s9\" (UniqueName: \"kubernetes.io/projected/1e7dca21-3a3e-471b-a5b6-3bf940955385-kube-api-access-f26s9\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:30:52.950671 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.950663    2550 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1e7dca21-3a3e-471b-a5b6-3bf940955385-extensions-socket-volume\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:30:52.950671 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.950672    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9b9t8\" (UniqueName: \"kubernetes.io/projected/3812f436-4df5-4e9a-a1f9-96c356c83b1c-kube-api-access-9b9t8\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:30:52.999984 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:52.999945    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vk4xt"
Aug 05 17:30:53.006228 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:53.006199    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8qt45"
Aug 05 17:30:53.157483 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:53.157445    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vk4xt"]
Aug 05 17:30:53.160748 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:30:53.160721    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ad7f602_67a6_4ecf_8e1f_eded3cfb7722.slice/crio-aa203931d7d021f8cc8eed7a5b4d4e107ca1926999427b2e6d2c468d27103a8f WatchSource:0}: Error finding container aa203931d7d021f8cc8eed7a5b4d4e107ca1926999427b2e6d2c468d27103a8f: Status 404 returned error can't find the container with id aa203931d7d021f8cc8eed7a5b4d4e107ca1926999427b2e6d2c468d27103a8f
Aug 05 17:30:53.183252 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:53.183223    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8qt45"]
Aug 05 17:30:53.196872 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:30:53.196835    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode50c044b_cb40_4340_870e_2fac665bbdf7.slice/crio-4d7ad54fe14c1aa1e102671b6355e829d9381d4666f38409295dfa97b52f5f8c WatchSource:0}: Error finding container 4d7ad54fe14c1aa1e102671b6355e829d9381d4666f38409295dfa97b52f5f8c: Status 404 returned error can't find the container with id 4d7ad54fe14c1aa1e102671b6355e829d9381d4666f38409295dfa97b52f5f8c
Aug 05 17:30:53.697820 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:30:53.697784    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-x7lx6\" is forbidden: User \"system:node:ip-10-0-134-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-46.ec2.internal' and this object" podUID="3812f436-4df5-4e9a-a1f9-96c356c83b1c" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-x7lx6"
Aug 05 17:30:53.699558 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:30:53.699523    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-r8g4q\" is forbidden: User \"system:node:ip-10-0-134-46.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-46.ec2.internal' and this object" podUID="1e7dca21-3a3e-471b-a5b6-3bf940955385" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-r8g4q"
Aug 05 17:30:53.699558 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:53.699543    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e7dca21-3a3e-471b-a5b6-3bf940955385" path="/var/lib/kubelet/pods/1e7dca21-3a3e-471b-a5b6-3bf940955385/volumes"
Aug 05 17:30:53.699896 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:53.699882    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3812f436-4df5-4e9a-a1f9-96c356c83b1c" path="/var/lib/kubelet/pods/3812f436-4df5-4e9a-a1f9-96c356c83b1c/volumes"
Aug 05 17:30:53.878576 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:53.878490    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8qt45" event={"ID":"e50c044b-cb40-4340-870e-2fac665bbdf7","Type":"ContainerStarted","Data":"10ef5591c1094240a63bae36e6b9d73a3f17599cde130b02fc05556e21053323"}
Aug 05 17:30:53.878576 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:53.878540    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8qt45" event={"ID":"e50c044b-cb40-4340-870e-2fac665bbdf7","Type":"ContainerStarted","Data":"4d7ad54fe14c1aa1e102671b6355e829d9381d4666f38409295dfa97b52f5f8c"}
Aug 05 17:30:53.878816 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:53.878581    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8qt45"
Aug 05 17:30:53.880855 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:53.880833    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vk4xt" event={"ID":"0ad7f602-67a6-4ecf-8e1f-eded3cfb7722","Type":"ContainerStarted","Data":"1567fa90fa456e948525392ba7261930823bf9493b821490987720cdfefd29a0"}
Aug 05 17:30:53.880951 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:53.880868    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vk4xt" event={"ID":"0ad7f602-67a6-4ecf-8e1f-eded3cfb7722","Type":"ContainerStarted","Data":"aa203931d7d021f8cc8eed7a5b4d4e107ca1926999427b2e6d2c468d27103a8f"}
Aug 05 17:30:53.880991 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:53.880954    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vk4xt"
Aug 05 17:30:53.897082 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:53.897035    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8qt45" podStartSLOduration=1.8970162 podStartE2EDuration="1.8970162s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:30:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:30:53.89658142 +0000 UTC m=+560.878158850" watchObservedRunningTime="2026-08-05 17:30:53.8970162 +0000 UTC m=+560.878593633"
Aug 05 17:30:53.918390 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:30:53.918343    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vk4xt" podStartSLOduration=1.918326865 podStartE2EDuration="1.918326865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:30:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:30:53.917379243 +0000 UTC m=+560.898956672" watchObservedRunningTime="2026-08-05 17:30:53.918326865 +0000 UTC m=+560.899904311"
Aug 05 17:31:04.888008 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:04.887977    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vk4xt"
Aug 05 17:31:04.888375 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:04.888056    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8qt45"
Aug 05 17:31:11.494461 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:11.494427    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vk4xt"]
Aug 05 17:31:11.495173 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:11.494692    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vk4xt" podUID="0ad7f602-67a6-4ecf-8e1f-eded3cfb7722" containerName="manager" containerID="cri-o://1567fa90fa456e948525392ba7261930823bf9493b821490987720cdfefd29a0" gracePeriod=10
Aug 05 17:31:11.793837 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:11.793806    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vk4xt"
Aug 05 17:31:11.906628 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:11.906594    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ad7f602-67a6-4ecf-8e1f-eded3cfb7722-extensions-socket-volume\") pod \"0ad7f602-67a6-4ecf-8e1f-eded3cfb7722\" (UID: \"0ad7f602-67a6-4ecf-8e1f-eded3cfb7722\") "
Aug 05 17:31:11.906628 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:11.906636    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5n5h\" (UniqueName: \"kubernetes.io/projected/0ad7f602-67a6-4ecf-8e1f-eded3cfb7722-kube-api-access-l5n5h\") pod \"0ad7f602-67a6-4ecf-8e1f-eded3cfb7722\" (UID: \"0ad7f602-67a6-4ecf-8e1f-eded3cfb7722\") "
Aug 05 17:31:11.907018 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:11.906990    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ad7f602-67a6-4ecf-8e1f-eded3cfb7722-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0ad7f602-67a6-4ecf-8e1f-eded3cfb7722" (UID: "0ad7f602-67a6-4ecf-8e1f-eded3cfb7722"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:31:11.908691 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:11.908668    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ad7f602-67a6-4ecf-8e1f-eded3cfb7722-kube-api-access-l5n5h" (OuterVolumeSpecName: "kube-api-access-l5n5h") pod "0ad7f602-67a6-4ecf-8e1f-eded3cfb7722" (UID: "0ad7f602-67a6-4ecf-8e1f-eded3cfb7722"). InnerVolumeSpecName "kube-api-access-l5n5h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:31:11.960252 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:11.960219    2550 generic.go:356] "Generic (PLEG): container finished" podID="0ad7f602-67a6-4ecf-8e1f-eded3cfb7722" containerID="1567fa90fa456e948525392ba7261930823bf9493b821490987720cdfefd29a0" exitCode=0
Aug 05 17:31:11.960448 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:11.960262    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vk4xt"
Aug 05 17:31:11.960448 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:11.960304    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vk4xt" event={"ID":"0ad7f602-67a6-4ecf-8e1f-eded3cfb7722","Type":"ContainerDied","Data":"1567fa90fa456e948525392ba7261930823bf9493b821490987720cdfefd29a0"}
Aug 05 17:31:11.960448 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:11.960357    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vk4xt" event={"ID":"0ad7f602-67a6-4ecf-8e1f-eded3cfb7722","Type":"ContainerDied","Data":"aa203931d7d021f8cc8eed7a5b4d4e107ca1926999427b2e6d2c468d27103a8f"}
Aug 05 17:31:11.960448 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:11.960385    2550 scope.go:117] "RemoveContainer" containerID="1567fa90fa456e948525392ba7261930823bf9493b821490987720cdfefd29a0"
Aug 05 17:31:11.969090 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:11.969068    2550 scope.go:117] "RemoveContainer" containerID="1567fa90fa456e948525392ba7261930823bf9493b821490987720cdfefd29a0"
Aug 05 17:31:11.969393 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:31:11.969370    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1567fa90fa456e948525392ba7261930823bf9493b821490987720cdfefd29a0\": container with ID starting with 1567fa90fa456e948525392ba7261930823bf9493b821490987720cdfefd29a0 not found: ID does not exist" containerID="1567fa90fa456e948525392ba7261930823bf9493b821490987720cdfefd29a0"
Aug 05 17:31:11.969437 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:11.969404    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1567fa90fa456e948525392ba7261930823bf9493b821490987720cdfefd29a0"} err="failed to get container status \"1567fa90fa456e948525392ba7261930823bf9493b821490987720cdfefd29a0\": rpc error: code = NotFound desc = could not find container \"1567fa90fa456e948525392ba7261930823bf9493b821490987720cdfefd29a0\": container with ID starting with 1567fa90fa456e948525392ba7261930823bf9493b821490987720cdfefd29a0 not found: ID does not exist"
Aug 05 17:31:11.988789 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:11.988759    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vk4xt"]
Aug 05 17:31:11.994077 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:11.994050    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vk4xt"]
Aug 05 17:31:12.008035 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:12.007973    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5n5h\" (UniqueName: \"kubernetes.io/projected/0ad7f602-67a6-4ecf-8e1f-eded3cfb7722-kube-api-access-l5n5h\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:31:12.008035 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:12.007998    2550 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ad7f602-67a6-4ecf-8e1f-eded3cfb7722-extensions-socket-volume\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:31:13.700715 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:13.700664    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ad7f602-67a6-4ecf-8e1f-eded3cfb7722" path="/var/lib/kubelet/pods/0ad7f602-67a6-4ecf-8e1f-eded3cfb7722/volumes"
Aug 05 17:31:27.756813 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.756778    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"]
Aug 05 17:31:27.757208 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.757087    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ad7f602-67a6-4ecf-8e1f-eded3cfb7722" containerName="manager"
Aug 05 17:31:27.757208 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.757097    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ad7f602-67a6-4ecf-8e1f-eded3cfb7722" containerName="manager"
Aug 05 17:31:27.757208 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.757150    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ad7f602-67a6-4ecf-8e1f-eded3cfb7722" containerName="manager"
Aug 05 17:31:27.760014 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.759993    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:27.762394 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.762376    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-vxc5c\""
Aug 05 17:31:27.772996 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.772966    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"]
Aug 05 17:31:27.835383 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.835348    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwsvc\" (UniqueName: \"kubernetes.io/projected/1b600ba3-1ae9-4de8-bffd-9bac75f659ad-kube-api-access-pwsvc\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6czx5\" (UID: \"1b600ba3-1ae9-4de8-bffd-9bac75f659ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:27.835383 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.835384    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b600ba3-1ae9-4de8-bffd-9bac75f659ad-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6czx5\" (UID: \"1b600ba3-1ae9-4de8-bffd-9bac75f659ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:27.835602 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.835421    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b600ba3-1ae9-4de8-bffd-9bac75f659ad-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6czx5\" (UID: \"1b600ba3-1ae9-4de8-bffd-9bac75f659ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:27.835602 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.835457    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b600ba3-1ae9-4de8-bffd-9bac75f659ad-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6czx5\" (UID: \"1b600ba3-1ae9-4de8-bffd-9bac75f659ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:27.835602 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.835482    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b600ba3-1ae9-4de8-bffd-9bac75f659ad-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6czx5\" (UID: \"1b600ba3-1ae9-4de8-bffd-9bac75f659ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:27.835602 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.835499    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b600ba3-1ae9-4de8-bffd-9bac75f659ad-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6czx5\" (UID: \"1b600ba3-1ae9-4de8-bffd-9bac75f659ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:27.835782 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.835591    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b600ba3-1ae9-4de8-bffd-9bac75f659ad-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6czx5\" (UID: \"1b600ba3-1ae9-4de8-bffd-9bac75f659ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:27.835782 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.835628    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b600ba3-1ae9-4de8-bffd-9bac75f659ad-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6czx5\" (UID: \"1b600ba3-1ae9-4de8-bffd-9bac75f659ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:27.835782 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.835662    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b600ba3-1ae9-4de8-bffd-9bac75f659ad-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6czx5\" (UID: \"1b600ba3-1ae9-4de8-bffd-9bac75f659ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:27.936985 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.936932    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b600ba3-1ae9-4de8-bffd-9bac75f659ad-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6czx5\" (UID: \"1b600ba3-1ae9-4de8-bffd-9bac75f659ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:27.936985 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.936983    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b600ba3-1ae9-4de8-bffd-9bac75f659ad-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6czx5\" (UID: \"1b600ba3-1ae9-4de8-bffd-9bac75f659ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:27.936985 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.937001    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b600ba3-1ae9-4de8-bffd-9bac75f659ad-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6czx5\" (UID: \"1b600ba3-1ae9-4de8-bffd-9bac75f659ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:27.937268 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.937021    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b600ba3-1ae9-4de8-bffd-9bac75f659ad-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6czx5\" (UID: \"1b600ba3-1ae9-4de8-bffd-9bac75f659ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:27.937268 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.937068    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b600ba3-1ae9-4de8-bffd-9bac75f659ad-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6czx5\" (UID: \"1b600ba3-1ae9-4de8-bffd-9bac75f659ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:27.937268 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.937103    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b600ba3-1ae9-4de8-bffd-9bac75f659ad-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6czx5\" (UID: \"1b600ba3-1ae9-4de8-bffd-9bac75f659ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:27.937268 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.937217    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b600ba3-1ae9-4de8-bffd-9bac75f659ad-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6czx5\" (UID: \"1b600ba3-1ae9-4de8-bffd-9bac75f659ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:27.937462 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.937274    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwsvc\" (UniqueName: \"kubernetes.io/projected/1b600ba3-1ae9-4de8-bffd-9bac75f659ad-kube-api-access-pwsvc\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6czx5\" (UID: \"1b600ba3-1ae9-4de8-bffd-9bac75f659ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:27.937462 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.937310    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b600ba3-1ae9-4de8-bffd-9bac75f659ad-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6czx5\" (UID: \"1b600ba3-1ae9-4de8-bffd-9bac75f659ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:27.937566 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.937487    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b600ba3-1ae9-4de8-bffd-9bac75f659ad-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6czx5\" (UID: \"1b600ba3-1ae9-4de8-bffd-9bac75f659ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:27.937624 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.937565    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b600ba3-1ae9-4de8-bffd-9bac75f659ad-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6czx5\" (UID: \"1b600ba3-1ae9-4de8-bffd-9bac75f659ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:27.937677 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.937656    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b600ba3-1ae9-4de8-bffd-9bac75f659ad-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6czx5\" (UID: \"1b600ba3-1ae9-4de8-bffd-9bac75f659ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:27.937773 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.937746    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b600ba3-1ae9-4de8-bffd-9bac75f659ad-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6czx5\" (UID: \"1b600ba3-1ae9-4de8-bffd-9bac75f659ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:27.937831 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.937791    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b600ba3-1ae9-4de8-bffd-9bac75f659ad-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6czx5\" (UID: \"1b600ba3-1ae9-4de8-bffd-9bac75f659ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:27.939930 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.939910    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b600ba3-1ae9-4de8-bffd-9bac75f659ad-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6czx5\" (UID: \"1b600ba3-1ae9-4de8-bffd-9bac75f659ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:27.940119 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.940103    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b600ba3-1ae9-4de8-bffd-9bac75f659ad-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6czx5\" (UID: \"1b600ba3-1ae9-4de8-bffd-9bac75f659ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:27.946114 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.946078    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b600ba3-1ae9-4de8-bffd-9bac75f659ad-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6czx5\" (UID: \"1b600ba3-1ae9-4de8-bffd-9bac75f659ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:27.946234 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:27.946216    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwsvc\" (UniqueName: \"kubernetes.io/projected/1b600ba3-1ae9-4de8-bffd-9bac75f659ad-kube-api-access-pwsvc\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6czx5\" (UID: \"1b600ba3-1ae9-4de8-bffd-9bac75f659ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:28.070078 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:28.069995    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:28.227794 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:28.227763    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"]
Aug 05 17:31:28.230986 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:31:28.230963    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b600ba3_1ae9_4de8_bffd_9bac75f659ad.slice/crio-854b3db1d93b3ca7a930b25ac43fc1f2ab705fe5b234e224f800c216fca0a1e6 WatchSource:0}: Error finding container 854b3db1d93b3ca7a930b25ac43fc1f2ab705fe5b234e224f800c216fca0a1e6: Status 404 returned error can't find the container with id 854b3db1d93b3ca7a930b25ac43fc1f2ab705fe5b234e224f800c216fca0a1e6
Aug 05 17:31:28.233023 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:28.232983    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 17:31:28.233106 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:28.233066    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 17:31:28.233153 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:28.233107    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 17:31:29.036069 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:29.036035    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5" event={"ID":"1b600ba3-1ae9-4de8-bffd-9bac75f659ad","Type":"ContainerStarted","Data":"fc0d6514a3233a9f2963f9aa4565297bb3502f50553e38284e53ebac1fe96c16"}
Aug 05 17:31:29.036425 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:29.036085    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5" event={"ID":"1b600ba3-1ae9-4de8-bffd-9bac75f659ad","Type":"ContainerStarted","Data":"854b3db1d93b3ca7a930b25ac43fc1f2ab705fe5b234e224f800c216fca0a1e6"}
Aug 05 17:31:29.058570 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:29.058526    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5" podStartSLOduration=2.058509206 podStartE2EDuration="2.058509206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:31:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:31:29.056511494 +0000 UTC m=+596.038088934" watchObservedRunningTime="2026-08-05 17:31:29.058509206 +0000 UTC m=+596.040086634"
Aug 05 17:31:29.070487 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:29.070464    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:29.075154 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:29.075134    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:30.040940 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:30.040909    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:30.041820 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:30.041800    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:31:30.053494 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:30.053469    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-x9pjh"]
Aug 05 17:31:30.056615 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:30.056600    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-x9pjh"
Aug 05 17:31:30.059182 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:30.059163    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-795gl\""
Aug 05 17:31:30.059228 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:30.059200    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Aug 05 17:31:30.068786 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:30.068766    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-x9pjh"]
Aug 05 17:31:30.154493 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:30.154463    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-288fc\" (UniqueName: \"kubernetes.io/projected/33d62cc4-f866-4eb7-b674-ea06f1af9c10-kube-api-access-288fc\") pod \"limitador-limitador-859c4d96d6-x9pjh\" (UID: \"33d62cc4-f866-4eb7-b674-ea06f1af9c10\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-x9pjh"
Aug 05 17:31:30.154669 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:30.154649    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/33d62cc4-f866-4eb7-b674-ea06f1af9c10-config-file\") pod \"limitador-limitador-859c4d96d6-x9pjh\" (UID: \"33d62cc4-f866-4eb7-b674-ea06f1af9c10\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-x9pjh"
Aug 05 17:31:30.171831 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:30.171805    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-x9pjh"]
Aug 05 17:31:30.255225 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:30.255195    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-288fc\" (UniqueName: \"kubernetes.io/projected/33d62cc4-f866-4eb7-b674-ea06f1af9c10-kube-api-access-288fc\") pod \"limitador-limitador-859c4d96d6-x9pjh\" (UID: \"33d62cc4-f866-4eb7-b674-ea06f1af9c10\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-x9pjh"
Aug 05 17:31:30.255361 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:30.255249    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/33d62cc4-f866-4eb7-b674-ea06f1af9c10-config-file\") pod \"limitador-limitador-859c4d96d6-x9pjh\" (UID: \"33d62cc4-f866-4eb7-b674-ea06f1af9c10\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-x9pjh"
Aug 05 17:31:30.255893 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:30.255874    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/33d62cc4-f866-4eb7-b674-ea06f1af9c10-config-file\") pod \"limitador-limitador-859c4d96d6-x9pjh\" (UID: \"33d62cc4-f866-4eb7-b674-ea06f1af9c10\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-x9pjh"
Aug 05 17:31:30.272622 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:30.272594    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-288fc\" (UniqueName: \"kubernetes.io/projected/33d62cc4-f866-4eb7-b674-ea06f1af9c10-kube-api-access-288fc\") pod \"limitador-limitador-859c4d96d6-x9pjh\" (UID: \"33d62cc4-f866-4eb7-b674-ea06f1af9c10\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-x9pjh"
Aug 05 17:31:30.367183 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:30.367113    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-x9pjh"
Aug 05 17:31:30.503436 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:30.503413    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-x9pjh"]
Aug 05 17:31:30.505362 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:31:30.505330    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33d62cc4_f866_4eb7_b674_ea06f1af9c10.slice/crio-72cb0cefd077e4211ce031aa5693171f6add664c6d4c7b41198ee31a72ff5ac5 WatchSource:0}: Error finding container 72cb0cefd077e4211ce031aa5693171f6add664c6d4c7b41198ee31a72ff5ac5: Status 404 returned error can't find the container with id 72cb0cefd077e4211ce031aa5693171f6add664c6d4c7b41198ee31a72ff5ac5
Aug 05 17:31:30.938044 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:30.938008    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b54df94b4-m4pcf"]
Aug 05 17:31:30.942419 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:30.942396    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-m4pcf"
Aug 05 17:31:30.945804 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:30.945783    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-7shrc\""
Aug 05 17:31:30.949197 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:30.949177    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-m4pcf"]
Aug 05 17:31:31.046290 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:31.046255    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-x9pjh" event={"ID":"33d62cc4-f866-4eb7-b674-ea06f1af9c10","Type":"ContainerStarted","Data":"72cb0cefd077e4211ce031aa5693171f6add664c6d4c7b41198ee31a72ff5ac5"}
Aug 05 17:31:31.062966 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:31.062938    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phzfb\" (UniqueName: \"kubernetes.io/projected/4e1f0687-9820-470c-a39a-720f262dfc03-kube-api-access-phzfb\") pod \"authorino-b54df94b4-m4pcf\" (UID: \"4e1f0687-9820-470c-a39a-720f262dfc03\") " pod="kuadrant-system/authorino-b54df94b4-m4pcf"
Aug 05 17:31:31.090478 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:31.090451    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-q6gl6"]
Aug 05 17:31:31.093679 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:31.093660    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-q6gl6"
Aug 05 17:31:31.102161 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:31.102138    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-q6gl6"]
Aug 05 17:31:31.163924 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:31.163886    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phzfb\" (UniqueName: \"kubernetes.io/projected/4e1f0687-9820-470c-a39a-720f262dfc03-kube-api-access-phzfb\") pod \"authorino-b54df94b4-m4pcf\" (UID: \"4e1f0687-9820-470c-a39a-720f262dfc03\") " pod="kuadrant-system/authorino-b54df94b4-m4pcf"
Aug 05 17:31:31.173301 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:31.173277    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phzfb\" (UniqueName: \"kubernetes.io/projected/4e1f0687-9820-470c-a39a-720f262dfc03-kube-api-access-phzfb\") pod \"authorino-b54df94b4-m4pcf\" (UID: \"4e1f0687-9820-470c-a39a-720f262dfc03\") " pod="kuadrant-system/authorino-b54df94b4-m4pcf"
Aug 05 17:31:31.252226 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:31.252146    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-m4pcf"
Aug 05 17:31:31.264683 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:31.264653    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lp87k\" (UniqueName: \"kubernetes.io/projected/820ab9be-563b-4de2-a571-ededa9da35d7-kube-api-access-lp87k\") pod \"authorino-c548b5d7d-q6gl6\" (UID: \"820ab9be-563b-4de2-a571-ededa9da35d7\") " pod="kuadrant-system/authorino-c548b5d7d-q6gl6"
Aug 05 17:31:31.365882 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:31.365853    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lp87k\" (UniqueName: \"kubernetes.io/projected/820ab9be-563b-4de2-a571-ededa9da35d7-kube-api-access-lp87k\") pod \"authorino-c548b5d7d-q6gl6\" (UID: \"820ab9be-563b-4de2-a571-ededa9da35d7\") " pod="kuadrant-system/authorino-c548b5d7d-q6gl6"
Aug 05 17:31:31.375223 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:31.375191    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lp87k\" (UniqueName: \"kubernetes.io/projected/820ab9be-563b-4de2-a571-ededa9da35d7-kube-api-access-lp87k\") pod \"authorino-c548b5d7d-q6gl6\" (UID: \"820ab9be-563b-4de2-a571-ededa9da35d7\") " pod="kuadrant-system/authorino-c548b5d7d-q6gl6"
Aug 05 17:31:31.383541 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:31.383519    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-m4pcf"]
Aug 05 17:31:31.385847 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:31:31.385827    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e1f0687_9820_470c_a39a_720f262dfc03.slice/crio-d65e073490399f80676b6df50d4e6a38d9f782bd3d943dc0f5bc2ccfb480cf2f WatchSource:0}: Error finding container d65e073490399f80676b6df50d4e6a38d9f782bd3d943dc0f5bc2ccfb480cf2f: Status 404 returned error can't find the container with id d65e073490399f80676b6df50d4e6a38d9f782bd3d943dc0f5bc2ccfb480cf2f
Aug 05 17:31:31.403007 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:31.402990    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-q6gl6"
Aug 05 17:31:31.527920 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:31.527893    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-q6gl6"]
Aug 05 17:31:31.529586 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:31:31.529562    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod820ab9be_563b_4de2_a571_ededa9da35d7.slice/crio-e12b115dbc678ef6934db4114216b409202ba662b518f5790d78d1862f077532 WatchSource:0}: Error finding container e12b115dbc678ef6934db4114216b409202ba662b518f5790d78d1862f077532: Status 404 returned error can't find the container with id e12b115dbc678ef6934db4114216b409202ba662b518f5790d78d1862f077532
Aug 05 17:31:32.052742 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:32.052691    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-q6gl6" event={"ID":"820ab9be-563b-4de2-a571-ededa9da35d7","Type":"ContainerStarted","Data":"e12b115dbc678ef6934db4114216b409202ba662b518f5790d78d1862f077532"}
Aug 05 17:31:32.054074 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:32.054042    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-m4pcf" event={"ID":"4e1f0687-9820-470c-a39a-720f262dfc03","Type":"ContainerStarted","Data":"d65e073490399f80676b6df50d4e6a38d9f782bd3d943dc0f5bc2ccfb480cf2f"}
Aug 05 17:31:33.061502 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:33.061463    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-x9pjh" event={"ID":"33d62cc4-f866-4eb7-b674-ea06f1af9c10","Type":"ContainerStarted","Data":"700427367e2f9a176311572501f3ae89bd05e63f46f52841bc7dc2c546da70a7"}
Aug 05 17:31:33.061951 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:33.061551    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-x9pjh"
Aug 05 17:31:33.079531 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:33.079258    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-x9pjh" podStartSLOduration=1.3356502909999999 podStartE2EDuration="3.07923667s" totalImagesPullingTime="1.743586379s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:31:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:31:33.078143152 +0000 UTC m=+600.059720611" watchObservedRunningTime="2026-08-05 17:31:33.07923667 +0000 UTC m=+600.060814102"
Aug 05 17:31:33.473346 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:33.473309    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 17:31:33.630102 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:33.630075    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 17:31:36.075779 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:36.075739    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-m4pcf" event={"ID":"4e1f0687-9820-470c-a39a-720f262dfc03","Type":"ContainerStarted","Data":"7652055aff43a6f028914e5ad4dd33c985e6f3329e04cf90417503d3d893d24f"}
Aug 05 17:31:36.077171 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:36.077149    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-q6gl6" event={"ID":"820ab9be-563b-4de2-a571-ededa9da35d7","Type":"ContainerStarted","Data":"21f40debd3f584686aaab594e26d6c7cb2859fdf062cfa911adadc4963ef7e62"}
Aug 05 17:31:36.098822 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:36.098777    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b54df94b4-m4pcf" podStartSLOduration=2.166929713 podStartE2EDuration="6.098759879s" totalImagesPullingTime="3.931830166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:31:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:31:36.097614762 +0000 UTC m=+603.079192191" watchObservedRunningTime="2026-08-05 17:31:36.098759879 +0000 UTC m=+603.080337312"
Aug 05 17:31:36.121410 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:36.121368    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-q6gl6" podStartSLOduration=1.375487788 podStartE2EDuration="5.121353195s" totalImagesPullingTime="3.745865407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:31:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:31:36.120250338 +0000 UTC m=+603.101827789" watchObservedRunningTime="2026-08-05 17:31:36.121353195 +0000 UTC m=+603.102930627"
Aug 05 17:31:36.158388 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:36.158353    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-m4pcf"]
Aug 05 17:31:38.085235 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:38.085192    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b54df94b4-m4pcf" podUID="4e1f0687-9820-470c-a39a-720f262dfc03" containerName="authorino" containerID="cri-o://7652055aff43a6f028914e5ad4dd33c985e6f3329e04cf90417503d3d893d24f" gracePeriod=30
Aug 05 17:31:38.323102 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:38.323078    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-m4pcf"
Aug 05 17:31:38.428615 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:38.428597    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phzfb\" (UniqueName: \"kubernetes.io/projected/4e1f0687-9820-470c-a39a-720f262dfc03-kube-api-access-phzfb\") pod \"4e1f0687-9820-470c-a39a-720f262dfc03\" (UID: \"4e1f0687-9820-470c-a39a-720f262dfc03\") "
Aug 05 17:31:38.430537 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:38.430516    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e1f0687-9820-470c-a39a-720f262dfc03-kube-api-access-phzfb" (OuterVolumeSpecName: "kube-api-access-phzfb") pod "4e1f0687-9820-470c-a39a-720f262dfc03" (UID: "4e1f0687-9820-470c-a39a-720f262dfc03"). InnerVolumeSpecName "kube-api-access-phzfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:31:38.529173 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:38.529139    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phzfb\" (UniqueName: \"kubernetes.io/projected/4e1f0687-9820-470c-a39a-720f262dfc03-kube-api-access-phzfb\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:31:39.089999 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:39.089967    2550 generic.go:356] "Generic (PLEG): container finished" podID="4e1f0687-9820-470c-a39a-720f262dfc03" containerID="7652055aff43a6f028914e5ad4dd33c985e6f3329e04cf90417503d3d893d24f" exitCode=0
Aug 05 17:31:39.090450 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:39.090005    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-m4pcf"
Aug 05 17:31:39.090450 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:39.090059    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-m4pcf" event={"ID":"4e1f0687-9820-470c-a39a-720f262dfc03","Type":"ContainerDied","Data":"7652055aff43a6f028914e5ad4dd33c985e6f3329e04cf90417503d3d893d24f"}
Aug 05 17:31:39.090450 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:39.090110    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-m4pcf" event={"ID":"4e1f0687-9820-470c-a39a-720f262dfc03","Type":"ContainerDied","Data":"d65e073490399f80676b6df50d4e6a38d9f782bd3d943dc0f5bc2ccfb480cf2f"}
Aug 05 17:31:39.090450 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:39.090136    2550 scope.go:117] "RemoveContainer" containerID="7652055aff43a6f028914e5ad4dd33c985e6f3329e04cf90417503d3d893d24f"
Aug 05 17:31:39.098309 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:39.098293    2550 scope.go:117] "RemoveContainer" containerID="7652055aff43a6f028914e5ad4dd33c985e6f3329e04cf90417503d3d893d24f"
Aug 05 17:31:39.098551 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:31:39.098535    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7652055aff43a6f028914e5ad4dd33c985e6f3329e04cf90417503d3d893d24f\": container with ID starting with 7652055aff43a6f028914e5ad4dd33c985e6f3329e04cf90417503d3d893d24f not found: ID does not exist" containerID="7652055aff43a6f028914e5ad4dd33c985e6f3329e04cf90417503d3d893d24f"
Aug 05 17:31:39.098633 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:39.098559    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7652055aff43a6f028914e5ad4dd33c985e6f3329e04cf90417503d3d893d24f"} err="failed to get container status \"7652055aff43a6f028914e5ad4dd33c985e6f3329e04cf90417503d3d893d24f\": rpc error: code = NotFound desc = could not find container \"7652055aff43a6f028914e5ad4dd33c985e6f3329e04cf90417503d3d893d24f\": container with ID starting with 7652055aff43a6f028914e5ad4dd33c985e6f3329e04cf90417503d3d893d24f not found: ID does not exist"
Aug 05 17:31:39.110816 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:39.110791    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-m4pcf"]
Aug 05 17:31:39.113887 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:39.113865    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b54df94b4-m4pcf"]
Aug 05 17:31:39.700109 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:39.700081    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e1f0687-9820-470c-a39a-720f262dfc03" path="/var/lib/kubelet/pods/4e1f0687-9820-470c-a39a-720f262dfc03/volumes"
Aug 05 17:31:44.066506 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:44.066475    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-x9pjh"
Aug 05 17:31:44.382530 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:44.382445    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-x9pjh"]
Aug 05 17:31:44.382682 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:44.382622    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-x9pjh" podUID="33d62cc4-f866-4eb7-b674-ea06f1af9c10" containerName="limitador" containerID="cri-o://700427367e2f9a176311572501f3ae89bd05e63f46f52841bc7dc2c546da70a7" gracePeriod=30
Aug 05 17:31:44.923971 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:44.923948    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-x9pjh"
Aug 05 17:31:44.973516 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:44.973442    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/33d62cc4-f866-4eb7-b674-ea06f1af9c10-config-file\") pod \"33d62cc4-f866-4eb7-b674-ea06f1af9c10\" (UID: \"33d62cc4-f866-4eb7-b674-ea06f1af9c10\") "
Aug 05 17:31:44.973516 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:44.973510    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-288fc\" (UniqueName: \"kubernetes.io/projected/33d62cc4-f866-4eb7-b674-ea06f1af9c10-kube-api-access-288fc\") pod \"33d62cc4-f866-4eb7-b674-ea06f1af9c10\" (UID: \"33d62cc4-f866-4eb7-b674-ea06f1af9c10\") "
Aug 05 17:31:44.973857 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:44.973831    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33d62cc4-f866-4eb7-b674-ea06f1af9c10-config-file" (OuterVolumeSpecName: "config-file") pod "33d62cc4-f866-4eb7-b674-ea06f1af9c10" (UID: "33d62cc4-f866-4eb7-b674-ea06f1af9c10"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:31:44.975516 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:44.975496    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33d62cc4-f866-4eb7-b674-ea06f1af9c10-kube-api-access-288fc" (OuterVolumeSpecName: "kube-api-access-288fc") pod "33d62cc4-f866-4eb7-b674-ea06f1af9c10" (UID: "33d62cc4-f866-4eb7-b674-ea06f1af9c10"). InnerVolumeSpecName "kube-api-access-288fc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:31:45.074431 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:45.074397    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-288fc\" (UniqueName: \"kubernetes.io/projected/33d62cc4-f866-4eb7-b674-ea06f1af9c10-kube-api-access-288fc\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:31:45.074431 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:45.074423    2550 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/33d62cc4-f866-4eb7-b674-ea06f1af9c10-config-file\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:31:45.116799 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:45.116769    2550 generic.go:356] "Generic (PLEG): container finished" podID="33d62cc4-f866-4eb7-b674-ea06f1af9c10" containerID="700427367e2f9a176311572501f3ae89bd05e63f46f52841bc7dc2c546da70a7" exitCode=0
Aug 05 17:31:45.116944 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:45.116815    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-x9pjh" event={"ID":"33d62cc4-f866-4eb7-b674-ea06f1af9c10","Type":"ContainerDied","Data":"700427367e2f9a176311572501f3ae89bd05e63f46f52841bc7dc2c546da70a7"}
Aug 05 17:31:45.116944 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:45.116821    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-x9pjh"
Aug 05 17:31:45.116944 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:45.116876    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-x9pjh" event={"ID":"33d62cc4-f866-4eb7-b674-ea06f1af9c10","Type":"ContainerDied","Data":"72cb0cefd077e4211ce031aa5693171f6add664c6d4c7b41198ee31a72ff5ac5"}
Aug 05 17:31:45.116944 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:45.116898    2550 scope.go:117] "RemoveContainer" containerID="700427367e2f9a176311572501f3ae89bd05e63f46f52841bc7dc2c546da70a7"
Aug 05 17:31:45.125079 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:45.125061    2550 scope.go:117] "RemoveContainer" containerID="700427367e2f9a176311572501f3ae89bd05e63f46f52841bc7dc2c546da70a7"
Aug 05 17:31:45.125301 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:31:45.125283    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"700427367e2f9a176311572501f3ae89bd05e63f46f52841bc7dc2c546da70a7\": container with ID starting with 700427367e2f9a176311572501f3ae89bd05e63f46f52841bc7dc2c546da70a7 not found: ID does not exist" containerID="700427367e2f9a176311572501f3ae89bd05e63f46f52841bc7dc2c546da70a7"
Aug 05 17:31:45.125387 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:45.125309    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"700427367e2f9a176311572501f3ae89bd05e63f46f52841bc7dc2c546da70a7"} err="failed to get container status \"700427367e2f9a176311572501f3ae89bd05e63f46f52841bc7dc2c546da70a7\": rpc error: code = NotFound desc = could not find container \"700427367e2f9a176311572501f3ae89bd05e63f46f52841bc7dc2c546da70a7\": container with ID starting with 700427367e2f9a176311572501f3ae89bd05e63f46f52841bc7dc2c546da70a7 not found: ID does not exist"
Aug 05 17:31:45.141665 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:45.141646    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-x9pjh"]
Aug 05 17:31:45.144571 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:45.144550    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-x9pjh"]
Aug 05 17:31:45.701294 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:45.701261    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33d62cc4-f866-4eb7-b674-ea06f1af9c10" path="/var/lib/kubelet/pods/33d62cc4-f866-4eb7-b674-ea06f1af9c10/volumes"
Aug 05 17:31:46.644273 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:46.644245    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-tdr8k"]
Aug 05 17:31:46.644626 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:46.644543    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33d62cc4-f866-4eb7-b674-ea06f1af9c10" containerName="limitador"
Aug 05 17:31:46.644626 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:46.644553    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="33d62cc4-f866-4eb7-b674-ea06f1af9c10" containerName="limitador"
Aug 05 17:31:46.644626 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:46.644564    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e1f0687-9820-470c-a39a-720f262dfc03" containerName="authorino"
Aug 05 17:31:46.644626 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:46.644569    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e1f0687-9820-470c-a39a-720f262dfc03" containerName="authorino"
Aug 05 17:31:46.644772 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:46.644628    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e1f0687-9820-470c-a39a-720f262dfc03" containerName="authorino"
Aug 05 17:31:46.644772 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:46.644637    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="33d62cc4-f866-4eb7-b674-ea06f1af9c10" containerName="limitador"
Aug 05 17:31:46.648768 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:46.648749    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tdr8k"
Aug 05 17:31:46.654892 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:46.654628    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Aug 05 17:31:46.654892 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:46.654857    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 05 17:31:46.655261 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:46.655244    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Aug 05 17:31:46.655642 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:46.655627    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-wg6v8\""
Aug 05 17:31:46.664474 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:46.664451    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-tdr8k"]
Aug 05 17:31:46.688210 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:46.688190    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndfpq\" (UniqueName: \"kubernetes.io/projected/93e686c5-3d2a-4ffa-a3f1-0d143bdf724f-kube-api-access-ndfpq\") pod \"postgres-69fb4594cf-tdr8k\" (UID: \"93e686c5-3d2a-4ffa-a3f1-0d143bdf724f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tdr8k"
Aug 05 17:31:46.688312 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:46.688225    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/93e686c5-3d2a-4ffa-a3f1-0d143bdf724f-data\") pod \"postgres-69fb4594cf-tdr8k\" (UID: \"93e686c5-3d2a-4ffa-a3f1-0d143bdf724f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tdr8k"
Aug 05 17:31:46.788747 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:46.788693    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndfpq\" (UniqueName: \"kubernetes.io/projected/93e686c5-3d2a-4ffa-a3f1-0d143bdf724f-kube-api-access-ndfpq\") pod \"postgres-69fb4594cf-tdr8k\" (UID: \"93e686c5-3d2a-4ffa-a3f1-0d143bdf724f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tdr8k"
Aug 05 17:31:46.788911 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:46.788760    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/93e686c5-3d2a-4ffa-a3f1-0d143bdf724f-data\") pod \"postgres-69fb4594cf-tdr8k\" (UID: \"93e686c5-3d2a-4ffa-a3f1-0d143bdf724f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tdr8k"
Aug 05 17:31:46.789098 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:46.789082    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/93e686c5-3d2a-4ffa-a3f1-0d143bdf724f-data\") pod \"postgres-69fb4594cf-tdr8k\" (UID: \"93e686c5-3d2a-4ffa-a3f1-0d143bdf724f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tdr8k"
Aug 05 17:31:46.802748 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:46.802690    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndfpq\" (UniqueName: \"kubernetes.io/projected/93e686c5-3d2a-4ffa-a3f1-0d143bdf724f-kube-api-access-ndfpq\") pod \"postgres-69fb4594cf-tdr8k\" (UID: \"93e686c5-3d2a-4ffa-a3f1-0d143bdf724f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tdr8k"
Aug 05 17:31:46.958326 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:46.958298    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tdr8k"
Aug 05 17:31:47.089859 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:47.089820    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-tdr8k"]
Aug 05 17:31:47.093402 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:31:47.093377    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93e686c5_3d2a_4ffa_a3f1_0d143bdf724f.slice/crio-48d5fc2b6139dc15548bfbdea30e6e3357afbcbb21e13da7ff9bcd7c42e43b06 WatchSource:0}: Error finding container 48d5fc2b6139dc15548bfbdea30e6e3357afbcbb21e13da7ff9bcd7c42e43b06: Status 404 returned error can't find the container with id 48d5fc2b6139dc15548bfbdea30e6e3357afbcbb21e13da7ff9bcd7c42e43b06
Aug 05 17:31:47.127382 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:47.127352    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tdr8k" event={"ID":"93e686c5-3d2a-4ffa-a3f1-0d143bdf724f","Type":"ContainerStarted","Data":"48d5fc2b6139dc15548bfbdea30e6e3357afbcbb21e13da7ff9bcd7c42e43b06"}
Aug 05 17:31:53.160870 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:53.160834    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tdr8k" event={"ID":"93e686c5-3d2a-4ffa-a3f1-0d143bdf724f","Type":"ContainerStarted","Data":"886667a8b263d551c0d8531d2200b65862ac9aeb618c2f93137098c55424f8eb"}
Aug 05 17:31:53.161223 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:53.160965    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tdr8k"
Aug 05 17:31:53.199488 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:53.199440    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tdr8k" podStartSLOduration=2.048393923 podStartE2EDuration="7.199426526s" totalImagesPullingTime="5.151032603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:31:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:31:53.199091008 +0000 UTC m=+620.180668436" watchObservedRunningTime="2026-08-05 17:31:53.199426526 +0000 UTC m=+620.181003990"
Aug 05 17:31:59.193237 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:59.193166    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tdr8k"
Aug 05 17:31:59.738464 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:59.738433    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-6xnk4"]
Aug 05 17:31:59.741772 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:59.741753    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-6xnk4"
Aug 05 17:31:59.753168 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:59.753144    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-6xnk4"]
Aug 05 17:31:59.796443 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:59.796416    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd8vl\" (UniqueName: \"kubernetes.io/projected/889ce1f4-7a10-4871-9c8e-b6b7c49a0b9b-kube-api-access-vd8vl\") pod \"authorino-5dc7ccb5f-6xnk4\" (UID: \"889ce1f4-7a10-4871-9c8e-b6b7c49a0b9b\") " pod="kuadrant-system/authorino-5dc7ccb5f-6xnk4"
Aug 05 17:31:59.842375 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:59.842344    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-6xnk4"]
Aug 05 17:31:59.842586 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:31:59.842567    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-vd8vl], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5dc7ccb5f-6xnk4" podUID="889ce1f4-7a10-4871-9c8e-b6b7c49a0b9b"
Aug 05 17:31:59.897367 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:59.897341    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd8vl\" (UniqueName: \"kubernetes.io/projected/889ce1f4-7a10-4871-9c8e-b6b7c49a0b9b-kube-api-access-vd8vl\") pod \"authorino-5dc7ccb5f-6xnk4\" (UID: \"889ce1f4-7a10-4871-9c8e-b6b7c49a0b9b\") " pod="kuadrant-system/authorino-5dc7ccb5f-6xnk4"
Aug 05 17:31:59.918606 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:31:59.918586    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd8vl\" (UniqueName: \"kubernetes.io/projected/889ce1f4-7a10-4871-9c8e-b6b7c49a0b9b-kube-api-access-vd8vl\") pod \"authorino-5dc7ccb5f-6xnk4\" (UID: \"889ce1f4-7a10-4871-9c8e-b6b7c49a0b9b\") " pod="kuadrant-system/authorino-5dc7ccb5f-6xnk4"
Aug 05 17:32:00.071161 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:00.071080    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-cdd8d8969-b7k2d"]
Aug 05 17:32:00.074437 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:00.074418    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cdd8d8969-b7k2d"
Aug 05 17:32:00.077778 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:00.077760    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Aug 05 17:32:00.091292 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:00.091266    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-cdd8d8969-b7k2d"]
Aug 05 17:32:00.099086 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:00.099062    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d7d8973b-00ff-4516-9add-786c207b6d53-tls-cert\") pod \"authorino-cdd8d8969-b7k2d\" (UID: \"d7d8973b-00ff-4516-9add-786c207b6d53\") " pod="kuadrant-system/authorino-cdd8d8969-b7k2d"
Aug 05 17:32:00.099215 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:00.099096    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8g9q\" (UniqueName: \"kubernetes.io/projected/d7d8973b-00ff-4516-9add-786c207b6d53-kube-api-access-c8g9q\") pod \"authorino-cdd8d8969-b7k2d\" (UID: \"d7d8973b-00ff-4516-9add-786c207b6d53\") " pod="kuadrant-system/authorino-cdd8d8969-b7k2d"
Aug 05 17:32:00.192160 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:00.192130    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-6xnk4"
Aug 05 17:32:00.200004 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:00.199980    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d7d8973b-00ff-4516-9add-786c207b6d53-tls-cert\") pod \"authorino-cdd8d8969-b7k2d\" (UID: \"d7d8973b-00ff-4516-9add-786c207b6d53\") " pod="kuadrant-system/authorino-cdd8d8969-b7k2d"
Aug 05 17:32:00.200366 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:00.200019    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8g9q\" (UniqueName: \"kubernetes.io/projected/d7d8973b-00ff-4516-9add-786c207b6d53-kube-api-access-c8g9q\") pod \"authorino-cdd8d8969-b7k2d\" (UID: \"d7d8973b-00ff-4516-9add-786c207b6d53\") " pod="kuadrant-system/authorino-cdd8d8969-b7k2d"
Aug 05 17:32:00.201859 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:00.201840    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-6xnk4"
Aug 05 17:32:00.202787 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:00.202768    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d7d8973b-00ff-4516-9add-786c207b6d53-tls-cert\") pod \"authorino-cdd8d8969-b7k2d\" (UID: \"d7d8973b-00ff-4516-9add-786c207b6d53\") " pod="kuadrant-system/authorino-cdd8d8969-b7k2d"
Aug 05 17:32:00.209574 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:00.209545    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8g9q\" (UniqueName: \"kubernetes.io/projected/d7d8973b-00ff-4516-9add-786c207b6d53-kube-api-access-c8g9q\") pod \"authorino-cdd8d8969-b7k2d\" (UID: \"d7d8973b-00ff-4516-9add-786c207b6d53\") " pod="kuadrant-system/authorino-cdd8d8969-b7k2d"
Aug 05 17:32:00.301104 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:00.301073    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vd8vl\" (UniqueName: \"kubernetes.io/projected/889ce1f4-7a10-4871-9c8e-b6b7c49a0b9b-kube-api-access-vd8vl\") pod \"889ce1f4-7a10-4871-9c8e-b6b7c49a0b9b\" (UID: \"889ce1f4-7a10-4871-9c8e-b6b7c49a0b9b\") "
Aug 05 17:32:00.303087 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:00.303061    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/889ce1f4-7a10-4871-9c8e-b6b7c49a0b9b-kube-api-access-vd8vl" (OuterVolumeSpecName: "kube-api-access-vd8vl") pod "889ce1f4-7a10-4871-9c8e-b6b7c49a0b9b" (UID: "889ce1f4-7a10-4871-9c8e-b6b7c49a0b9b"). InnerVolumeSpecName "kube-api-access-vd8vl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:32:00.383613 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:00.383536    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cdd8d8969-b7k2d"
Aug 05 17:32:00.402227 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:00.402201    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vd8vl\" (UniqueName: \"kubernetes.io/projected/889ce1f4-7a10-4871-9c8e-b6b7c49a0b9b-kube-api-access-vd8vl\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:32:00.521635 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:00.521602    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-cdd8d8969-b7k2d"]
Aug 05 17:32:00.523511 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:32:00.523487    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd7d8973b_00ff_4516_9add_786c207b6d53.slice/crio-f00f719304c59b940e2bd79d4ad2d5f6a8b0b98224f87b96bd51d3b2f10c45ad WatchSource:0}: Error finding container f00f719304c59b940e2bd79d4ad2d5f6a8b0b98224f87b96bd51d3b2f10c45ad: Status 404 returned error can't find the container with id f00f719304c59b940e2bd79d4ad2d5f6a8b0b98224f87b96bd51d3b2f10c45ad
Aug 05 17:32:01.196830 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:01.196793    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cdd8d8969-b7k2d" event={"ID":"d7d8973b-00ff-4516-9add-786c207b6d53","Type":"ContainerStarted","Data":"f00f719304c59b940e2bd79d4ad2d5f6a8b0b98224f87b96bd51d3b2f10c45ad"}
Aug 05 17:32:01.196830 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:01.196803    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-6xnk4"
Aug 05 17:32:01.249151 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:01.249121    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-6xnk4"]
Aug 05 17:32:01.251336 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:01.251309    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-6xnk4"]
Aug 05 17:32:01.699524 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:01.699492    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="889ce1f4-7a10-4871-9c8e-b6b7c49a0b9b" path="/var/lib/kubelet/pods/889ce1f4-7a10-4871-9c8e-b6b7c49a0b9b/volumes"
Aug 05 17:32:02.202174 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:02.202134    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cdd8d8969-b7k2d" event={"ID":"d7d8973b-00ff-4516-9add-786c207b6d53","Type":"ContainerStarted","Data":"bd7be98aa90c74a7f3360b469268c74a2f42023dba2038a30176c330aee9daae"}
Aug 05 17:32:02.232873 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:02.232821    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-cdd8d8969-b7k2d" podStartSLOduration=1.444559676 podStartE2EDuration="2.232804532s" totalImagesPullingTime="788.244856ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:32:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:32:02.232230896 +0000 UTC m=+629.213808327" watchObservedRunningTime="2026-08-05 17:32:02.232804532 +0000 UTC m=+629.214381961"
Aug 05 17:32:02.294375 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:02.294336    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-q6gl6"]
Aug 05 17:32:02.294838 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:02.294551    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-q6gl6" podUID="820ab9be-563b-4de2-a571-ededa9da35d7" containerName="authorino" containerID="cri-o://21f40debd3f584686aaab594e26d6c7cb2859fdf062cfa911adadc4963ef7e62" gracePeriod=30
Aug 05 17:32:02.561880 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:02.561850    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-q6gl6"
Aug 05 17:32:02.619294 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:02.619266    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lp87k\" (UniqueName: \"kubernetes.io/projected/820ab9be-563b-4de2-a571-ededa9da35d7-kube-api-access-lp87k\") pod \"820ab9be-563b-4de2-a571-ededa9da35d7\" (UID: \"820ab9be-563b-4de2-a571-ededa9da35d7\") "
Aug 05 17:32:02.621167 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:02.621142    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/820ab9be-563b-4de2-a571-ededa9da35d7-kube-api-access-lp87k" (OuterVolumeSpecName: "kube-api-access-lp87k") pod "820ab9be-563b-4de2-a571-ededa9da35d7" (UID: "820ab9be-563b-4de2-a571-ededa9da35d7"). InnerVolumeSpecName "kube-api-access-lp87k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:32:02.720569 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:02.720490    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lp87k\" (UniqueName: \"kubernetes.io/projected/820ab9be-563b-4de2-a571-ededa9da35d7-kube-api-access-lp87k\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:32:03.207442 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:03.207402    2550 generic.go:356] "Generic (PLEG): container finished" podID="820ab9be-563b-4de2-a571-ededa9da35d7" containerID="21f40debd3f584686aaab594e26d6c7cb2859fdf062cfa911adadc4963ef7e62" exitCode=0
Aug 05 17:32:03.207442 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:03.207442    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-q6gl6"
Aug 05 17:32:03.207665 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:03.207479    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-q6gl6" event={"ID":"820ab9be-563b-4de2-a571-ededa9da35d7","Type":"ContainerDied","Data":"21f40debd3f584686aaab594e26d6c7cb2859fdf062cfa911adadc4963ef7e62"}
Aug 05 17:32:03.207665 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:03.207525    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-q6gl6" event={"ID":"820ab9be-563b-4de2-a571-ededa9da35d7","Type":"ContainerDied","Data":"e12b115dbc678ef6934db4114216b409202ba662b518f5790d78d1862f077532"}
Aug 05 17:32:03.207665 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:03.207544    2550 scope.go:117] "RemoveContainer" containerID="21f40debd3f584686aaab594e26d6c7cb2859fdf062cfa911adadc4963ef7e62"
Aug 05 17:32:03.217841 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:03.217824    2550 scope.go:117] "RemoveContainer" containerID="21f40debd3f584686aaab594e26d6c7cb2859fdf062cfa911adadc4963ef7e62"
Aug 05 17:32:03.218247 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:32:03.218227    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21f40debd3f584686aaab594e26d6c7cb2859fdf062cfa911adadc4963ef7e62\": container with ID starting with 21f40debd3f584686aaab594e26d6c7cb2859fdf062cfa911adadc4963ef7e62 not found: ID does not exist" containerID="21f40debd3f584686aaab594e26d6c7cb2859fdf062cfa911adadc4963ef7e62"
Aug 05 17:32:03.218331 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:03.218258    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21f40debd3f584686aaab594e26d6c7cb2859fdf062cfa911adadc4963ef7e62"} err="failed to get container status \"21f40debd3f584686aaab594e26d6c7cb2859fdf062cfa911adadc4963ef7e62\": rpc error: code = NotFound desc = could not find container \"21f40debd3f584686aaab594e26d6c7cb2859fdf062cfa911adadc4963ef7e62\": container with ID starting with 21f40debd3f584686aaab594e26d6c7cb2859fdf062cfa911adadc4963ef7e62 not found: ID does not exist"
Aug 05 17:32:03.250881 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:03.250857    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-q6gl6"]
Aug 05 17:32:03.257661 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:03.257644    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-q6gl6"]
Aug 05 17:32:03.700240 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:03.700206    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="820ab9be-563b-4de2-a571-ededa9da35d7" path="/var/lib/kubelet/pods/820ab9be-563b-4de2-a571-ededa9da35d7/volumes"
Aug 05 17:32:07.226406 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:07.226375    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b999f6f6f-gc72f"]
Aug 05 17:32:07.226890 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:07.226757    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="820ab9be-563b-4de2-a571-ededa9da35d7" containerName="authorino"
Aug 05 17:32:07.226890 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:07.226776    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="820ab9be-563b-4de2-a571-ededa9da35d7" containerName="authorino"
Aug 05 17:32:07.226890 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:07.226855    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="820ab9be-563b-4de2-a571-ededa9da35d7" containerName="authorino"
Aug 05 17:32:07.231315 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:07.231294    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b999f6f6f-gc72f"
Aug 05 17:32:07.233971 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:07.233950    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-567ln\""
Aug 05 17:32:07.234140 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:07.233999    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 05 17:32:07.234737 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:07.234716    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 05 17:32:07.242065 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:07.242042    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b999f6f6f-gc72f"]
Aug 05 17:32:07.363145 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:07.363111    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dts2w\" (UniqueName: \"kubernetes.io/projected/6bdef447-b097-4e13-ad34-78a5adcaac56-kube-api-access-dts2w\") pod \"maas-controller-6b999f6f6f-gc72f\" (UID: \"6bdef447-b097-4e13-ad34-78a5adcaac56\") " pod="opendatahub/maas-controller-6b999f6f6f-gc72f"
Aug 05 17:32:07.363312 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:07.363195    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bdef447-b097-4e13-ad34-78a5adcaac56-webhook-cert\") pod \"maas-controller-6b999f6f6f-gc72f\" (UID: \"6bdef447-b097-4e13-ad34-78a5adcaac56\") " pod="opendatahub/maas-controller-6b999f6f6f-gc72f"
Aug 05 17:32:07.416985 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:07.416953    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6bb68d8457-dbh88"]
Aug 05 17:32:07.420051 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:07.420033    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bb68d8457-dbh88"
Aug 05 17:32:07.442134 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:07.442101    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bb68d8457-dbh88"]
Aug 05 17:32:07.463770 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:07.463735    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bdef447-b097-4e13-ad34-78a5adcaac56-webhook-cert\") pod \"maas-controller-6b999f6f6f-gc72f\" (UID: \"6bdef447-b097-4e13-ad34-78a5adcaac56\") " pod="opendatahub/maas-controller-6b999f6f6f-gc72f"
Aug 05 17:32:07.463957 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:07.463798    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dts2w\" (UniqueName: \"kubernetes.io/projected/6bdef447-b097-4e13-ad34-78a5adcaac56-kube-api-access-dts2w\") pod \"maas-controller-6b999f6f6f-gc72f\" (UID: \"6bdef447-b097-4e13-ad34-78a5adcaac56\") " pod="opendatahub/maas-controller-6b999f6f6f-gc72f"
Aug 05 17:32:07.463957 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:07.463848    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9qpx\" (UniqueName: \"kubernetes.io/projected/d2fd17de-f4fc-4330-8f80-604352d6395e-kube-api-access-g9qpx\") pod \"maas-controller-6bb68d8457-dbh88\" (UID: \"d2fd17de-f4fc-4330-8f80-604352d6395e\") " pod="opendatahub/maas-controller-6bb68d8457-dbh88"
Aug 05 17:32:07.463957 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:07.463888    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2fd17de-f4fc-4330-8f80-604352d6395e-webhook-cert\") pod \"maas-controller-6bb68d8457-dbh88\" (UID: \"d2fd17de-f4fc-4330-8f80-604352d6395e\") " pod="opendatahub/maas-controller-6bb68d8457-dbh88"
Aug 05 17:32:07.466153 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:07.466128    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bdef447-b097-4e13-ad34-78a5adcaac56-webhook-cert\") pod \"maas-controller-6b999f6f6f-gc72f\" (UID: \"6bdef447-b097-4e13-ad34-78a5adcaac56\") " pod="opendatahub/maas-controller-6b999f6f6f-gc72f"
Aug 05 17:32:07.477565 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:07.477505    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dts2w\" (UniqueName: \"kubernetes.io/projected/6bdef447-b097-4e13-ad34-78a5adcaac56-kube-api-access-dts2w\") pod \"maas-controller-6b999f6f6f-gc72f\" (UID: \"6bdef447-b097-4e13-ad34-78a5adcaac56\") " pod="opendatahub/maas-controller-6b999f6f6f-gc72f"
Aug 05 17:32:07.542508 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:07.542469    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b999f6f6f-gc72f"
Aug 05 17:32:07.564680 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:07.564649    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9qpx\" (UniqueName: \"kubernetes.io/projected/d2fd17de-f4fc-4330-8f80-604352d6395e-kube-api-access-g9qpx\") pod \"maas-controller-6bb68d8457-dbh88\" (UID: \"d2fd17de-f4fc-4330-8f80-604352d6395e\") " pod="opendatahub/maas-controller-6bb68d8457-dbh88"
Aug 05 17:32:07.564853 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:07.564711    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2fd17de-f4fc-4330-8f80-604352d6395e-webhook-cert\") pod \"maas-controller-6bb68d8457-dbh88\" (UID: \"d2fd17de-f4fc-4330-8f80-604352d6395e\") " pod="opendatahub/maas-controller-6bb68d8457-dbh88"
Aug 05 17:32:07.567225 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:07.567199    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2fd17de-f4fc-4330-8f80-604352d6395e-webhook-cert\") pod \"maas-controller-6bb68d8457-dbh88\" (UID: \"d2fd17de-f4fc-4330-8f80-604352d6395e\") " pod="opendatahub/maas-controller-6bb68d8457-dbh88"
Aug 05 17:32:07.575946 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:07.575921    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9qpx\" (UniqueName: \"kubernetes.io/projected/d2fd17de-f4fc-4330-8f80-604352d6395e-kube-api-access-g9qpx\") pod \"maas-controller-6bb68d8457-dbh88\" (UID: \"d2fd17de-f4fc-4330-8f80-604352d6395e\") " pod="opendatahub/maas-controller-6bb68d8457-dbh88"
Aug 05 17:32:07.685672 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:07.685640    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b999f6f6f-gc72f"]
Aug 05 17:32:07.689122 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:32:07.689087    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bdef447_b097_4e13_ad34_78a5adcaac56.slice/crio-31f9caa5fdb87f891d343351974c8ed9ebd4c4affe6c82f0dbd6fd6790d82c00 WatchSource:0}: Error finding container 31f9caa5fdb87f891d343351974c8ed9ebd4c4affe6c82f0dbd6fd6790d82c00: Status 404 returned error can't find the container with id 31f9caa5fdb87f891d343351974c8ed9ebd4c4affe6c82f0dbd6fd6790d82c00
Aug 05 17:32:07.729660 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:07.729588    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bb68d8457-dbh88"
Aug 05 17:32:07.865506 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:07.865451    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bb68d8457-dbh88"]
Aug 05 17:32:07.868044 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:32:07.868023    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2fd17de_f4fc_4330_8f80_604352d6395e.slice/crio-325b3af4ea0d5a1d2a347a2e1aaf51836c4f049f4f35a1cad35fa31535f1590a WatchSource:0}: Error finding container 325b3af4ea0d5a1d2a347a2e1aaf51836c4f049f4f35a1cad35fa31535f1590a: Status 404 returned error can't find the container with id 325b3af4ea0d5a1d2a347a2e1aaf51836c4f049f4f35a1cad35fa31535f1590a
Aug 05 17:32:08.232761 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:08.232728    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bb68d8457-dbh88" event={"ID":"d2fd17de-f4fc-4330-8f80-604352d6395e","Type":"ContainerStarted","Data":"325b3af4ea0d5a1d2a347a2e1aaf51836c4f049f4f35a1cad35fa31535f1590a"}
Aug 05 17:32:08.233817 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:08.233790    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b999f6f6f-gc72f" event={"ID":"6bdef447-b097-4e13-ad34-78a5adcaac56","Type":"ContainerStarted","Data":"31f9caa5fdb87f891d343351974c8ed9ebd4c4affe6c82f0dbd6fd6790d82c00"}
Aug 05 17:32:11.250964 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:11.250923    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b999f6f6f-gc72f" event={"ID":"6bdef447-b097-4e13-ad34-78a5adcaac56","Type":"ContainerStarted","Data":"6fe22e3b988ae75097fa2ccea8aa4c73504873c3f0515119bf5d0eeafc58174a"}
Aug 05 17:32:11.253410 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:11.250992    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b999f6f6f-gc72f"
Aug 05 17:32:11.253410 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:11.252304    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bb68d8457-dbh88" event={"ID":"d2fd17de-f4fc-4330-8f80-604352d6395e","Type":"ContainerStarted","Data":"8d0db0cb4a884094356fbe5aaa9fb1650e6fcc1558d5d9ee29b7ae3a69da9008"}
Aug 05 17:32:11.253410 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:11.252357    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6bb68d8457-dbh88"
Aug 05 17:32:11.282357 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:11.282281    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6b999f6f6f-gc72f" podStartSLOduration=1.738878931 podStartE2EDuration="4.282262541s" totalImagesPullingTime="2.54338361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:32:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:32:11.282249891 +0000 UTC m=+638.263827321" watchObservedRunningTime="2026-08-05 17:32:11.282262541 +0000 UTC m=+638.263839971"
Aug 05 17:32:11.318235 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:11.318179    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6bb68d8457-dbh88" podStartSLOduration=1.9609290069999998 podStartE2EDuration="4.318159666s" totalImagesPullingTime="2.357230659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:32:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:32:11.317517435 +0000 UTC m=+638.299094858" watchObservedRunningTime="2026-08-05 17:32:11.318159666 +0000 UTC m=+638.299737168"
Aug 05 17:32:22.259818 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:22.259785    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b999f6f6f-gc72f"
Aug 05 17:32:22.260185 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:22.259858    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6bb68d8457-dbh88"
Aug 05 17:32:22.351818 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:22.351779    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b999f6f6f-gc72f"]
Aug 05 17:32:22.352420 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:22.351989    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6b999f6f6f-gc72f" podUID="6bdef447-b097-4e13-ad34-78a5adcaac56" containerName="manager" containerID="cri-o://6fe22e3b988ae75097fa2ccea8aa4c73504873c3f0515119bf5d0eeafc58174a" gracePeriod=30
Aug 05 17:32:22.600809 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:22.600786    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b999f6f6f-gc72f"
Aug 05 17:32:22.687693 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:22.687654    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dts2w\" (UniqueName: \"kubernetes.io/projected/6bdef447-b097-4e13-ad34-78a5adcaac56-kube-api-access-dts2w\") pod \"6bdef447-b097-4e13-ad34-78a5adcaac56\" (UID: \"6bdef447-b097-4e13-ad34-78a5adcaac56\") "
Aug 05 17:32:22.687885 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:22.687835    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bdef447-b097-4e13-ad34-78a5adcaac56-webhook-cert\") pod \"6bdef447-b097-4e13-ad34-78a5adcaac56\" (UID: \"6bdef447-b097-4e13-ad34-78a5adcaac56\") "
Aug 05 17:32:22.689980 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:22.689952    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bdef447-b097-4e13-ad34-78a5adcaac56-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6bdef447-b097-4e13-ad34-78a5adcaac56" (UID: "6bdef447-b097-4e13-ad34-78a5adcaac56"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:32:22.690087 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:22.689974    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bdef447-b097-4e13-ad34-78a5adcaac56-kube-api-access-dts2w" (OuterVolumeSpecName: "kube-api-access-dts2w") pod "6bdef447-b097-4e13-ad34-78a5adcaac56" (UID: "6bdef447-b097-4e13-ad34-78a5adcaac56"). InnerVolumeSpecName "kube-api-access-dts2w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:32:22.788763 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:22.788655    2550 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bdef447-b097-4e13-ad34-78a5adcaac56-webhook-cert\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:32:22.788763 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:22.788684    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dts2w\" (UniqueName: \"kubernetes.io/projected/6bdef447-b097-4e13-ad34-78a5adcaac56-kube-api-access-dts2w\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:32:23.311193 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:23.311159    2550 generic.go:356] "Generic (PLEG): container finished" podID="6bdef447-b097-4e13-ad34-78a5adcaac56" containerID="6fe22e3b988ae75097fa2ccea8aa4c73504873c3f0515119bf5d0eeafc58174a" exitCode=0
Aug 05 17:32:23.311628 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:23.311204    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b999f6f6f-gc72f"
Aug 05 17:32:23.311628 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:23.311203    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b999f6f6f-gc72f" event={"ID":"6bdef447-b097-4e13-ad34-78a5adcaac56","Type":"ContainerDied","Data":"6fe22e3b988ae75097fa2ccea8aa4c73504873c3f0515119bf5d0eeafc58174a"}
Aug 05 17:32:23.311628 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:23.311333    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b999f6f6f-gc72f" event={"ID":"6bdef447-b097-4e13-ad34-78a5adcaac56","Type":"ContainerDied","Data":"31f9caa5fdb87f891d343351974c8ed9ebd4c4affe6c82f0dbd6fd6790d82c00"}
Aug 05 17:32:23.311628 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:23.311362    2550 scope.go:117] "RemoveContainer" containerID="6fe22e3b988ae75097fa2ccea8aa4c73504873c3f0515119bf5d0eeafc58174a"
Aug 05 17:32:23.324035 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:23.320972    2550 scope.go:117] "RemoveContainer" containerID="6fe22e3b988ae75097fa2ccea8aa4c73504873c3f0515119bf5d0eeafc58174a"
Aug 05 17:32:23.324035 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:32:23.322037    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fe22e3b988ae75097fa2ccea8aa4c73504873c3f0515119bf5d0eeafc58174a\": container with ID starting with 6fe22e3b988ae75097fa2ccea8aa4c73504873c3f0515119bf5d0eeafc58174a not found: ID does not exist" containerID="6fe22e3b988ae75097fa2ccea8aa4c73504873c3f0515119bf5d0eeafc58174a"
Aug 05 17:32:23.324035 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:23.322076    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fe22e3b988ae75097fa2ccea8aa4c73504873c3f0515119bf5d0eeafc58174a"} err="failed to get container status \"6fe22e3b988ae75097fa2ccea8aa4c73504873c3f0515119bf5d0eeafc58174a\": rpc error: code = NotFound desc = could not find container \"6fe22e3b988ae75097fa2ccea8aa4c73504873c3f0515119bf5d0eeafc58174a\": container with ID starting with 6fe22e3b988ae75097fa2ccea8aa4c73504873c3f0515119bf5d0eeafc58174a not found: ID does not exist"
Aug 05 17:32:23.334965 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:23.334939    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b999f6f6f-gc72f"]
Aug 05 17:32:23.341243 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:23.341221    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6b999f6f6f-gc72f"]
Aug 05 17:32:23.700610 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:23.700576    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bdef447-b097-4e13-ad34-78a5adcaac56" path="/var/lib/kubelet/pods/6bdef447-b097-4e13-ad34-78a5adcaac56/volumes"
Aug 05 17:32:43.943497 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:43.943464    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6475fbf58c-ptfhq"]
Aug 05 17:32:43.943970 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:43.943828    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bdef447-b097-4e13-ad34-78a5adcaac56" containerName="manager"
Aug 05 17:32:43.943970 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:43.943841    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bdef447-b097-4e13-ad34-78a5adcaac56" containerName="manager"
Aug 05 17:32:43.943970 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:43.943900    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bdef447-b097-4e13-ad34-78a5adcaac56" containerName="manager"
Aug 05 17:32:43.948219 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:43.948197    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6475fbf58c-ptfhq"
Aug 05 17:32:43.950679 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:43.950651    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Aug 05 17:32:43.952381 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:43.952362    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9f8vc\""
Aug 05 17:32:43.960896 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:43.960872    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bef8aee-313c-409f-879e-e453179a9f76-maas-api-tls\") pod \"maas-api-6475fbf58c-ptfhq\" (UID: \"5bef8aee-313c-409f-879e-e453179a9f76\") " pod="odh-ai-gateway-infra/maas-api-6475fbf58c-ptfhq"
Aug 05 17:32:43.961008 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:43.960912    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsw9k\" (UniqueName: \"kubernetes.io/projected/5bef8aee-313c-409f-879e-e453179a9f76-kube-api-access-qsw9k\") pod \"maas-api-6475fbf58c-ptfhq\" (UID: \"5bef8aee-313c-409f-879e-e453179a9f76\") " pod="odh-ai-gateway-infra/maas-api-6475fbf58c-ptfhq"
Aug 05 17:32:43.961008 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:43.960997    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bef8aee-313c-409f-879e-e453179a9f76-openshift-service-ca\") pod \"maas-api-6475fbf58c-ptfhq\" (UID: \"5bef8aee-313c-409f-879e-e453179a9f76\") " pod="odh-ai-gateway-infra/maas-api-6475fbf58c-ptfhq"
Aug 05 17:32:43.964743 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:43.964718    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6475fbf58c-ptfhq"]
Aug 05 17:32:44.061780 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:44.061732    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bef8aee-313c-409f-879e-e453179a9f76-maas-api-tls\") pod \"maas-api-6475fbf58c-ptfhq\" (UID: \"5bef8aee-313c-409f-879e-e453179a9f76\") " pod="odh-ai-gateway-infra/maas-api-6475fbf58c-ptfhq"
Aug 05 17:32:44.061984 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:44.061805    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsw9k\" (UniqueName: \"kubernetes.io/projected/5bef8aee-313c-409f-879e-e453179a9f76-kube-api-access-qsw9k\") pod \"maas-api-6475fbf58c-ptfhq\" (UID: \"5bef8aee-313c-409f-879e-e453179a9f76\") " pod="odh-ai-gateway-infra/maas-api-6475fbf58c-ptfhq"
Aug 05 17:32:44.061984 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:32:44.061826    2550 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Aug 05 17:32:44.061984 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:44.061843    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bef8aee-313c-409f-879e-e453179a9f76-openshift-service-ca\") pod \"maas-api-6475fbf58c-ptfhq\" (UID: \"5bef8aee-313c-409f-879e-e453179a9f76\") " pod="odh-ai-gateway-infra/maas-api-6475fbf58c-ptfhq"
Aug 05 17:32:44.061984 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:32:44.061930    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bef8aee-313c-409f-879e-e453179a9f76-maas-api-tls podName:5bef8aee-313c-409f-879e-e453179a9f76 nodeName:}" failed. No retries permitted until 2026-08-05 17:32:44.561908378 +0000 UTC m=+671.543485785 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5bef8aee-313c-409f-879e-e453179a9f76-maas-api-tls") pod "maas-api-6475fbf58c-ptfhq" (UID: "5bef8aee-313c-409f-879e-e453179a9f76") : secret "maas-api-serving-cert" not found
Aug 05 17:32:44.062482 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:44.062462    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bef8aee-313c-409f-879e-e453179a9f76-openshift-service-ca\") pod \"maas-api-6475fbf58c-ptfhq\" (UID: \"5bef8aee-313c-409f-879e-e453179a9f76\") " pod="odh-ai-gateway-infra/maas-api-6475fbf58c-ptfhq"
Aug 05 17:32:44.074639 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:44.074610    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsw9k\" (UniqueName: \"kubernetes.io/projected/5bef8aee-313c-409f-879e-e453179a9f76-kube-api-access-qsw9k\") pod \"maas-api-6475fbf58c-ptfhq\" (UID: \"5bef8aee-313c-409f-879e-e453179a9f76\") " pod="odh-ai-gateway-infra/maas-api-6475fbf58c-ptfhq"
Aug 05 17:32:44.539564 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:44.539526    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-crrck"]
Aug 05 17:32:44.547490 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:44.547465    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-crrck"
Aug 05 17:32:44.551352 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:44.551332    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Aug 05 17:32:44.558031 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:44.558006    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-crrck"]
Aug 05 17:32:44.567039 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:44.567009    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b762d708-104b-4bbb-ba46-13a3aff27b83-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-crrck\" (UID: \"b762d708-104b-4bbb-ba46-13a3aff27b83\") " pod="openshift-ingress/payload-processing-5cc6cd444b-crrck"
Aug 05 17:32:44.567208 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:44.567083    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bef8aee-313c-409f-879e-e453179a9f76-maas-api-tls\") pod \"maas-api-6475fbf58c-ptfhq\" (UID: \"5bef8aee-313c-409f-879e-e453179a9f76\") " pod="odh-ai-gateway-infra/maas-api-6475fbf58c-ptfhq"
Aug 05 17:32:44.567208 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:44.567159    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwfv2\" (UniqueName: \"kubernetes.io/projected/b762d708-104b-4bbb-ba46-13a3aff27b83-kube-api-access-kwfv2\") pod \"payload-processing-5cc6cd444b-crrck\" (UID: \"b762d708-104b-4bbb-ba46-13a3aff27b83\") " pod="openshift-ingress/payload-processing-5cc6cd444b-crrck"
Aug 05 17:32:44.569627 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:44.569605    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bef8aee-313c-409f-879e-e453179a9f76-maas-api-tls\") pod \"maas-api-6475fbf58c-ptfhq\" (UID: \"5bef8aee-313c-409f-879e-e453179a9f76\") " pod="odh-ai-gateway-infra/maas-api-6475fbf58c-ptfhq"
Aug 05 17:32:44.667552 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:44.667521    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwfv2\" (UniqueName: \"kubernetes.io/projected/b762d708-104b-4bbb-ba46-13a3aff27b83-kube-api-access-kwfv2\") pod \"payload-processing-5cc6cd444b-crrck\" (UID: \"b762d708-104b-4bbb-ba46-13a3aff27b83\") " pod="openshift-ingress/payload-processing-5cc6cd444b-crrck"
Aug 05 17:32:44.667773 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:44.667577    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b762d708-104b-4bbb-ba46-13a3aff27b83-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-crrck\" (UID: \"b762d708-104b-4bbb-ba46-13a3aff27b83\") " pod="openshift-ingress/payload-processing-5cc6cd444b-crrck"
Aug 05 17:32:44.668237 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:44.668217    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b762d708-104b-4bbb-ba46-13a3aff27b83-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-crrck\" (UID: \"b762d708-104b-4bbb-ba46-13a3aff27b83\") " pod="openshift-ingress/payload-processing-5cc6cd444b-crrck"
Aug 05 17:32:44.679602 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:44.679564    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwfv2\" (UniqueName: \"kubernetes.io/projected/b762d708-104b-4bbb-ba46-13a3aff27b83-kube-api-access-kwfv2\") pod \"payload-processing-5cc6cd444b-crrck\" (UID: \"b762d708-104b-4bbb-ba46-13a3aff27b83\") " pod="openshift-ingress/payload-processing-5cc6cd444b-crrck"
Aug 05 17:32:44.858208 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:44.858122    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6475fbf58c-ptfhq"
Aug 05 17:32:44.859200 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:44.859181    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-crrck"
Aug 05 17:32:45.026157 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:45.026127    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6475fbf58c-ptfhq"]
Aug 05 17:32:45.028425 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:32:45.028399    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bef8aee_313c_409f_879e_e453179a9f76.slice/crio-681cb8d783b96a08290c8b2ff6348873148349e57166c33bfc2d7fca9fbc7af0 WatchSource:0}: Error finding container 681cb8d783b96a08290c8b2ff6348873148349e57166c33bfc2d7fca9fbc7af0: Status 404 returned error can't find the container with id 681cb8d783b96a08290c8b2ff6348873148349e57166c33bfc2d7fca9fbc7af0
Aug 05 17:32:45.055052 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:45.055023    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-crrck"]
Aug 05 17:32:45.057096 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:32:45.057070    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb762d708_104b_4bbb_ba46_13a3aff27b83.slice/crio-c2b5cb400e6a41114ec750e196dec56d60b4b43532d9f5cd03de1cdf6384dfba WatchSource:0}: Error finding container c2b5cb400e6a41114ec750e196dec56d60b4b43532d9f5cd03de1cdf6384dfba: Status 404 returned error can't find the container with id c2b5cb400e6a41114ec750e196dec56d60b4b43532d9f5cd03de1cdf6384dfba
Aug 05 17:32:45.410329 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:45.410284    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6475fbf58c-ptfhq" event={"ID":"5bef8aee-313c-409f-879e-e453179a9f76","Type":"ContainerStarted","Data":"681cb8d783b96a08290c8b2ff6348873148349e57166c33bfc2d7fca9fbc7af0"}
Aug 05 17:32:45.415016 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:45.414972    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-crrck" event={"ID":"b762d708-104b-4bbb-ba46-13a3aff27b83","Type":"ContainerStarted","Data":"c2b5cb400e6a41114ec750e196dec56d60b4b43532d9f5cd03de1cdf6384dfba"}
Aug 05 17:32:47.444310 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:47.444270    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6475fbf58c-ptfhq" event={"ID":"5bef8aee-313c-409f-879e-e453179a9f76","Type":"ContainerStarted","Data":"da5a6c3f9cc118d83521ff184afaed92bb39a5adfc8fa95aec1464e43bffd5d7"}
Aug 05 17:32:47.444716 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:47.444429    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6475fbf58c-ptfhq"
Aug 05 17:32:47.446102 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:47.446076    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-crrck" event={"ID":"b762d708-104b-4bbb-ba46-13a3aff27b83","Type":"ContainerStarted","Data":"a889f3e3c129cc1d98565241abe98b65ba1e03679edded418b810255fcdff89d"}
Aug 05 17:32:47.446234 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:47.446217    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-crrck"
Aug 05 17:32:47.464642 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:47.464584    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6475fbf58c-ptfhq" podStartSLOduration=2.24619397 podStartE2EDuration="4.464563054s" totalImagesPullingTime="2.218369084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:32:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:32:47.463946505 +0000 UTC m=+674.445523932" watchObservedRunningTime="2026-08-05 17:32:47.464563054 +0000 UTC m=+674.446140485"
Aug 05 17:32:47.487312 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:47.487253    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-crrck" podStartSLOduration=1.291470061 podStartE2EDuration="3.487234522s" totalImagesPullingTime="2.195764461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:32:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:32:47.486297418 +0000 UTC m=+674.467874848" watchObservedRunningTime="2026-08-05 17:32:47.487234522 +0000 UTC m=+674.468811950"
Aug 05 17:32:53.456343 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:53.456316    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6475fbf58c-ptfhq"
Aug 05 17:32:54.476817 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:54.476782    2550 generic.go:356] "Generic (PLEG): container finished" podID="1b600ba3-1ae9-4de8-bffd-9bac75f659ad" containerID="fc0d6514a3233a9f2963f9aa4565297bb3502f50553e38284e53ebac1fe96c16" exitCode=0
Aug 05 17:32:54.477182 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:54.476855    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5" event={"ID":"1b600ba3-1ae9-4de8-bffd-9bac75f659ad","Type":"ContainerDied","Data":"fc0d6514a3233a9f2963f9aa4565297bb3502f50553e38284e53ebac1fe96c16"}
Aug 05 17:32:54.477228 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:54.477211    2550 scope.go:117] "RemoveContainer" containerID="fc0d6514a3233a9f2963f9aa4565297bb3502f50553e38284e53ebac1fe96c16"
Aug 05 17:32:54.478266 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:54.478242    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 17:32:54.478340 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:54.478323    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 17:32:54.478395 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:54.478379    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 17:32:55.483132 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:55.483101    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5" event={"ID":"1b600ba3-1ae9-4de8-bffd-9bac75f659ad","Type":"ContainerStarted","Data":"5d5c1ebd01a1e598fd915a4b7caade626b2ab2cb5cbf0a0600ae97483c3c6949"}
Aug 05 17:32:56.070963 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:56.070932    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:32:56.071904 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:56.071878    2550 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-6czx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused" start-of-body=
Aug 05 17:32:56.072023 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:56.071924    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5" podUID="1b600ba3-1ae9-4de8-bffd-9bac75f659ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused"
Aug 05 17:32:57.071060 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:57.071030    2550 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-6czx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused" start-of-body=
Aug 05 17:32:57.071432 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:57.071085    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5" podUID="1b600ba3-1ae9-4de8-bffd-9bac75f659ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused"
Aug 05 17:32:58.071233 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:58.071199    2550 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-6czx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused" start-of-body=
Aug 05 17:32:58.071593 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:58.071255    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5" podUID="1b600ba3-1ae9-4de8-bffd-9bac75f659ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused"
Aug 05 17:32:58.453021 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:58.452992    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-crrck"
Aug 05 17:32:59.071156 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:59.071120    2550 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-6czx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused" start-of-body=
Aug 05 17:32:59.071334 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:32:59.071174    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5" podUID="1b600ba3-1ae9-4de8-bffd-9bac75f659ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused"
Aug 05 17:33:00.041498 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:00.041460    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:33:00.070999 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:00.070966    2550 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-6czx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused" start-of-body=
Aug 05 17:33:00.071114 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:00.071036    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5" podUID="1b600ba3-1ae9-4de8-bffd-9bac75f659ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused"
Aug 05 17:33:01.074670 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:01.074639    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:33:01.075609 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:01.075587    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6czx5"
Aug 05 17:33:47.776682 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:47.776588    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq"]
Aug 05 17:33:47.780107 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:47.780086    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq"
Aug 05 17:33:47.783418 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:47.783390    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Aug 05 17:33:47.783541 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:47.783523    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Aug 05 17:33:47.783604 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:47.783544    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Aug 05 17:33:47.783664 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:47.783549    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6zzxp\""
Aug 05 17:33:47.788364 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:47.788338    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq"]
Aug 05 17:33:47.800352 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:47.800322    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84b646e7-28f3-4a91-af8d-74a4305d8e92-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ks5mq\" (UID: \"84b646e7-28f3-4a91-af8d-74a4305d8e92\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq"
Aug 05 17:33:47.800489 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:47.800366    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84b646e7-28f3-4a91-af8d-74a4305d8e92-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ks5mq\" (UID: \"84b646e7-28f3-4a91-af8d-74a4305d8e92\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq"
Aug 05 17:33:47.800489 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:47.800453    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84b646e7-28f3-4a91-af8d-74a4305d8e92-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ks5mq\" (UID: \"84b646e7-28f3-4a91-af8d-74a4305d8e92\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq"
Aug 05 17:33:47.800602 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:47.800498    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84b646e7-28f3-4a91-af8d-74a4305d8e92-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ks5mq\" (UID: \"84b646e7-28f3-4a91-af8d-74a4305d8e92\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq"
Aug 05 17:33:47.800602 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:47.800568    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dndlr\" (UniqueName: \"kubernetes.io/projected/84b646e7-28f3-4a91-af8d-74a4305d8e92-kube-api-access-dndlr\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ks5mq\" (UID: \"84b646e7-28f3-4a91-af8d-74a4305d8e92\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq"
Aug 05 17:33:47.800690 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:47.800633    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84b646e7-28f3-4a91-af8d-74a4305d8e92-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ks5mq\" (UID: \"84b646e7-28f3-4a91-af8d-74a4305d8e92\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq"
Aug 05 17:33:47.901100 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:47.901063    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84b646e7-28f3-4a91-af8d-74a4305d8e92-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ks5mq\" (UID: \"84b646e7-28f3-4a91-af8d-74a4305d8e92\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq"
Aug 05 17:33:47.901100 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:47.901102    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84b646e7-28f3-4a91-af8d-74a4305d8e92-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ks5mq\" (UID: \"84b646e7-28f3-4a91-af8d-74a4305d8e92\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq"
Aug 05 17:33:47.901351 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:47.901140    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84b646e7-28f3-4a91-af8d-74a4305d8e92-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ks5mq\" (UID: \"84b646e7-28f3-4a91-af8d-74a4305d8e92\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq"
Aug 05 17:33:47.901351 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:47.901170    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84b646e7-28f3-4a91-af8d-74a4305d8e92-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ks5mq\" (UID: \"84b646e7-28f3-4a91-af8d-74a4305d8e92\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq"
Aug 05 17:33:47.901351 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:47.901208    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dndlr\" (UniqueName: \"kubernetes.io/projected/84b646e7-28f3-4a91-af8d-74a4305d8e92-kube-api-access-dndlr\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ks5mq\" (UID: \"84b646e7-28f3-4a91-af8d-74a4305d8e92\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq"
Aug 05 17:33:47.901351 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:47.901258    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84b646e7-28f3-4a91-af8d-74a4305d8e92-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ks5mq\" (UID: \"84b646e7-28f3-4a91-af8d-74a4305d8e92\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq"
Aug 05 17:33:47.901557 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:47.901530    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84b646e7-28f3-4a91-af8d-74a4305d8e92-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ks5mq\" (UID: \"84b646e7-28f3-4a91-af8d-74a4305d8e92\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq"
Aug 05 17:33:47.901610 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:47.901552    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84b646e7-28f3-4a91-af8d-74a4305d8e92-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ks5mq\" (UID: \"84b646e7-28f3-4a91-af8d-74a4305d8e92\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq"
Aug 05 17:33:47.901610 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:47.901583    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84b646e7-28f3-4a91-af8d-74a4305d8e92-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ks5mq\" (UID: \"84b646e7-28f3-4a91-af8d-74a4305d8e92\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq"
Aug 05 17:33:47.903579 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:47.903553    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84b646e7-28f3-4a91-af8d-74a4305d8e92-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ks5mq\" (UID: \"84b646e7-28f3-4a91-af8d-74a4305d8e92\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq"
Aug 05 17:33:47.903903 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:47.903882    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84b646e7-28f3-4a91-af8d-74a4305d8e92-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ks5mq\" (UID: \"84b646e7-28f3-4a91-af8d-74a4305d8e92\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq"
Aug 05 17:33:47.909904 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:47.909876    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dndlr\" (UniqueName: \"kubernetes.io/projected/84b646e7-28f3-4a91-af8d-74a4305d8e92-kube-api-access-dndlr\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ks5mq\" (UID: \"84b646e7-28f3-4a91-af8d-74a4305d8e92\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq"
Aug 05 17:33:48.091970 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:48.091873    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq"
Aug 05 17:33:48.227847 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:48.227803    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq"]
Aug 05 17:33:48.230133 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:33:48.230108    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84b646e7_28f3_4a91_af8d_74a4305d8e92.slice/crio-2f41e00c640b0f67f09ca197cc86428f8b0901b90ad5b722f8a84fd5f19fd4f8 WatchSource:0}: Error finding container 2f41e00c640b0f67f09ca197cc86428f8b0901b90ad5b722f8a84fd5f19fd4f8: Status 404 returned error can't find the container with id 2f41e00c640b0f67f09ca197cc86428f8b0901b90ad5b722f8a84fd5f19fd4f8
Aug 05 17:33:48.231374 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:48.231358    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 17:33:48.732724 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:48.732679    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq" event={"ID":"84b646e7-28f3-4a91-af8d-74a4305d8e92","Type":"ContainerStarted","Data":"2f41e00c640b0f67f09ca197cc86428f8b0901b90ad5b722f8a84fd5f19fd4f8"}
Aug 05 17:33:49.739354 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:49.739315    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq" event={"ID":"84b646e7-28f3-4a91-af8d-74a4305d8e92","Type":"ContainerStarted","Data":"892c85572ae3774808da77483d3c821f41baf4ddbfa58302e39f948e13afc776"}
Aug 05 17:33:49.758370 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:49.758317    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq" podStartSLOduration=1.663022265 podStartE2EDuration="2.758300871s" totalImagesPullingTime="1.095278606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:33:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:33:49.757602228 +0000 UTC m=+736.739179657" watchObservedRunningTime="2026-08-05 17:33:49.758300871 +0000 UTC m=+736.739878300"
Aug 05 17:33:53.952986 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:53.952950    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds"]
Aug 05 17:33:53.957121 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:53.957096    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds"
Aug 05 17:33:53.959564 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:53.959539    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Aug 05 17:33:53.967252 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:53.967225    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds"]
Aug 05 17:33:54.057228 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:54.057184    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e287ca84-d2ca-4783-acc0-73229a00ff03-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds\" (UID: \"e287ca84-d2ca-4783-acc0-73229a00ff03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds"
Aug 05 17:33:54.057228 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:54.057228    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gjfc\" (UniqueName: \"kubernetes.io/projected/e287ca84-d2ca-4783-acc0-73229a00ff03-kube-api-access-8gjfc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds\" (UID: \"e287ca84-d2ca-4783-acc0-73229a00ff03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds"
Aug 05 17:33:54.057449 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:54.057256    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e287ca84-d2ca-4783-acc0-73229a00ff03-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds\" (UID: \"e287ca84-d2ca-4783-acc0-73229a00ff03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds"
Aug 05 17:33:54.057449 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:54.057295    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e287ca84-d2ca-4783-acc0-73229a00ff03-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds\" (UID: \"e287ca84-d2ca-4783-acc0-73229a00ff03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds"
Aug 05 17:33:54.057449 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:54.057310    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e287ca84-d2ca-4783-acc0-73229a00ff03-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds\" (UID: \"e287ca84-d2ca-4783-acc0-73229a00ff03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds"
Aug 05 17:33:54.057449 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:54.057368    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e287ca84-d2ca-4783-acc0-73229a00ff03-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds\" (UID: \"e287ca84-d2ca-4783-acc0-73229a00ff03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds"
Aug 05 17:33:54.158036 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:54.157990    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e287ca84-d2ca-4783-acc0-73229a00ff03-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds\" (UID: \"e287ca84-d2ca-4783-acc0-73229a00ff03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds"
Aug 05 17:33:54.158036 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:54.158032    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gjfc\" (UniqueName: \"kubernetes.io/projected/e287ca84-d2ca-4783-acc0-73229a00ff03-kube-api-access-8gjfc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds\" (UID: \"e287ca84-d2ca-4783-acc0-73229a00ff03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds"
Aug 05 17:33:54.158279 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:54.158064    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e287ca84-d2ca-4783-acc0-73229a00ff03-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds\" (UID: \"e287ca84-d2ca-4783-acc0-73229a00ff03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds"
Aug 05 17:33:54.158279 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:54.158088    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e287ca84-d2ca-4783-acc0-73229a00ff03-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds\" (UID: \"e287ca84-d2ca-4783-acc0-73229a00ff03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds"
Aug 05 17:33:54.158279 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:54.158113    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e287ca84-d2ca-4783-acc0-73229a00ff03-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds\" (UID: \"e287ca84-d2ca-4783-acc0-73229a00ff03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds"
Aug 05 17:33:54.158279 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:54.158149    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e287ca84-d2ca-4783-acc0-73229a00ff03-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds\" (UID: \"e287ca84-d2ca-4783-acc0-73229a00ff03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds"
Aug 05 17:33:54.158497 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:54.158432    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e287ca84-d2ca-4783-acc0-73229a00ff03-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds\" (UID: \"e287ca84-d2ca-4783-acc0-73229a00ff03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds"
Aug 05 17:33:54.158497 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:54.158452    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e287ca84-d2ca-4783-acc0-73229a00ff03-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds\" (UID: \"e287ca84-d2ca-4783-acc0-73229a00ff03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds"
Aug 05 17:33:54.158603 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:54.158502    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e287ca84-d2ca-4783-acc0-73229a00ff03-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds\" (UID: \"e287ca84-d2ca-4783-acc0-73229a00ff03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds"
Aug 05 17:33:54.160438 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:54.160407    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e287ca84-d2ca-4783-acc0-73229a00ff03-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds\" (UID: \"e287ca84-d2ca-4783-acc0-73229a00ff03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds"
Aug 05 17:33:54.160757 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:54.160739    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e287ca84-d2ca-4783-acc0-73229a00ff03-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds\" (UID: \"e287ca84-d2ca-4783-acc0-73229a00ff03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds"
Aug 05 17:33:54.169985 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:54.169956    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gjfc\" (UniqueName: \"kubernetes.io/projected/e287ca84-d2ca-4783-acc0-73229a00ff03-kube-api-access-8gjfc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds\" (UID: \"e287ca84-d2ca-4783-acc0-73229a00ff03\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds"
Aug 05 17:33:54.268998 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:54.268898    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds"
Aug 05 17:33:54.416016 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:54.415982    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds"]
Aug 05 17:33:54.418765 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:33:54.418736    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode287ca84_d2ca_4783_acc0_73229a00ff03.slice/crio-a31099bc33212f286cc5b609e528c6ab4cff0c1d884fab1edd264f1603a69f61 WatchSource:0}: Error finding container a31099bc33212f286cc5b609e528c6ab4cff0c1d884fab1edd264f1603a69f61: Status 404 returned error can't find the container with id a31099bc33212f286cc5b609e528c6ab4cff0c1d884fab1edd264f1603a69f61
Aug 05 17:33:54.763498 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:54.763462    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds" event={"ID":"e287ca84-d2ca-4783-acc0-73229a00ff03","Type":"ContainerStarted","Data":"bb6c9dd992c6192504a450599e11927c9803ee3641e91d4ab5a9604335ff67c8"}
Aug 05 17:33:54.763665 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:54.763528    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds" event={"ID":"e287ca84-d2ca-4783-acc0-73229a00ff03","Type":"ContainerStarted","Data":"a31099bc33212f286cc5b609e528c6ab4cff0c1d884fab1edd264f1603a69f61"}
Aug 05 17:33:54.782503 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:54.782451    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds" podStartSLOduration=1.514291372 podStartE2EDuration="1.782432327s" totalImagesPullingTime="268.140955ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:33:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:33:54.779994981 +0000 UTC m=+741.761572409" watchObservedRunningTime="2026-08-05 17:33:54.782432327 +0000 UTC m=+741.764009756"
Aug 05 17:33:58.092612 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:58.092565    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq"
Aug 05 17:33:58.093091 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:58.092672    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq"
Aug 05 17:33:58.105319 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:58.105290    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq"
Aug 05 17:33:58.792970 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:33:58.792942    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ks5mq"
Aug 05 17:34:04.269750 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:34:04.269690    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds"
Aug 05 17:34:04.270206 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:34:04.269785    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds"
Aug 05 17:34:04.283824 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:34:04.283795    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds"
Aug 05 17:34:04.819500 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:34:04.819469    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds"
Aug 05 17:35:01.718636 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:01.718555    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75db7cc4bd-5qrzc"]
Aug 05 17:35:01.721982 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:01.721964    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75db7cc4bd-5qrzc"
Aug 05 17:35:01.732775 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:01.732747    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75db7cc4bd-5qrzc"]
Aug 05 17:35:01.843448 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:01.843408    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr7hx\" (UniqueName: \"kubernetes.io/projected/92543e2a-899a-40b4-8c2f-5a25127e378f-kube-api-access-cr7hx\") pod \"authorino-75db7cc4bd-5qrzc\" (UID: \"92543e2a-899a-40b4-8c2f-5a25127e378f\") " pod="kuadrant-system/authorino-75db7cc4bd-5qrzc"
Aug 05 17:35:01.843618 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:01.843533    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/92543e2a-899a-40b4-8c2f-5a25127e378f-tls-cert\") pod \"authorino-75db7cc4bd-5qrzc\" (UID: \"92543e2a-899a-40b4-8c2f-5a25127e378f\") " pod="kuadrant-system/authorino-75db7cc4bd-5qrzc"
Aug 05 17:35:01.944230 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:01.944189    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr7hx\" (UniqueName: \"kubernetes.io/projected/92543e2a-899a-40b4-8c2f-5a25127e378f-kube-api-access-cr7hx\") pod \"authorino-75db7cc4bd-5qrzc\" (UID: \"92543e2a-899a-40b4-8c2f-5a25127e378f\") " pod="kuadrant-system/authorino-75db7cc4bd-5qrzc"
Aug 05 17:35:01.944420 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:01.944335    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/92543e2a-899a-40b4-8c2f-5a25127e378f-tls-cert\") pod \"authorino-75db7cc4bd-5qrzc\" (UID: \"92543e2a-899a-40b4-8c2f-5a25127e378f\") " pod="kuadrant-system/authorino-75db7cc4bd-5qrzc"
Aug 05 17:35:01.946766 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:01.946743    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/92543e2a-899a-40b4-8c2f-5a25127e378f-tls-cert\") pod \"authorino-75db7cc4bd-5qrzc\" (UID: \"92543e2a-899a-40b4-8c2f-5a25127e378f\") " pod="kuadrant-system/authorino-75db7cc4bd-5qrzc"
Aug 05 17:35:01.953124 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:01.953100    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr7hx\" (UniqueName: \"kubernetes.io/projected/92543e2a-899a-40b4-8c2f-5a25127e378f-kube-api-access-cr7hx\") pod \"authorino-75db7cc4bd-5qrzc\" (UID: \"92543e2a-899a-40b4-8c2f-5a25127e378f\") " pod="kuadrant-system/authorino-75db7cc4bd-5qrzc"
Aug 05 17:35:02.031198 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:02.031106    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75db7cc4bd-5qrzc"
Aug 05 17:35:02.166528 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:02.166501    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75db7cc4bd-5qrzc"]
Aug 05 17:35:02.169818 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:35:02.168685    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod92543e2a_899a_40b4_8c2f_5a25127e378f.slice/crio-6484608f14db14d95813fc8c36caabd6f15da9045a8ff34f83237c191b63f136 WatchSource:0}: Error finding container 6484608f14db14d95813fc8c36caabd6f15da9045a8ff34f83237c191b63f136: Status 404 returned error can't find the container with id 6484608f14db14d95813fc8c36caabd6f15da9045a8ff34f83237c191b63f136
Aug 05 17:35:03.077099 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:03.077069    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75db7cc4bd-5qrzc" event={"ID":"92543e2a-899a-40b4-8c2f-5a25127e378f","Type":"ContainerStarted","Data":"6484608f14db14d95813fc8c36caabd6f15da9045a8ff34f83237c191b63f136"}
Aug 05 17:35:04.083432 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:04.083388    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75db7cc4bd-5qrzc" event={"ID":"92543e2a-899a-40b4-8c2f-5a25127e378f","Type":"ContainerStarted","Data":"5b640e4d2b526aa0151a3b319cf2357d8c35febb4318daa0e21d5f71c471689d"}
Aug 05 17:35:04.108196 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:04.108135    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75db7cc4bd-5qrzc" podStartSLOduration=1.9937839830000001 podStartE2EDuration="3.108117798s" totalImagesPullingTime="1.114333815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:35:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:35:04.105992558 +0000 UTC m=+811.087570006" watchObservedRunningTime="2026-08-05 17:35:04.108117798 +0000 UTC m=+811.089695226"
Aug 05 17:35:04.137539 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:04.137503    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-cdd8d8969-b7k2d"]
Aug 05 17:35:04.137779 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:04.137749    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-cdd8d8969-b7k2d" podUID="d7d8973b-00ff-4516-9add-786c207b6d53" containerName="authorino" containerID="cri-o://bd7be98aa90c74a7f3360b469268c74a2f42023dba2038a30176c330aee9daae" gracePeriod=30
Aug 05 17:35:04.386812 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:04.386787    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cdd8d8969-b7k2d"
Aug 05 17:35:04.466989 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:04.466942    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d7d8973b-00ff-4516-9add-786c207b6d53-tls-cert\") pod \"d7d8973b-00ff-4516-9add-786c207b6d53\" (UID: \"d7d8973b-00ff-4516-9add-786c207b6d53\") "
Aug 05 17:35:04.467175 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:04.467026    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c8g9q\" (UniqueName: \"kubernetes.io/projected/d7d8973b-00ff-4516-9add-786c207b6d53-kube-api-access-c8g9q\") pod \"d7d8973b-00ff-4516-9add-786c207b6d53\" (UID: \"d7d8973b-00ff-4516-9add-786c207b6d53\") "
Aug 05 17:35:04.469085 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:04.469043    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7d8973b-00ff-4516-9add-786c207b6d53-kube-api-access-c8g9q" (OuterVolumeSpecName: "kube-api-access-c8g9q") pod "d7d8973b-00ff-4516-9add-786c207b6d53" (UID: "d7d8973b-00ff-4516-9add-786c207b6d53"). InnerVolumeSpecName "kube-api-access-c8g9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:35:04.477322 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:04.477292    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7d8973b-00ff-4516-9add-786c207b6d53-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "d7d8973b-00ff-4516-9add-786c207b6d53" (UID: "d7d8973b-00ff-4516-9add-786c207b6d53"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:35:04.567953 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:04.567914    2550 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d7d8973b-00ff-4516-9add-786c207b6d53-tls-cert\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:35:04.567953 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:04.567944    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8g9q\" (UniqueName: \"kubernetes.io/projected/d7d8973b-00ff-4516-9add-786c207b6d53-kube-api-access-c8g9q\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:35:05.089298 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:05.089264    2550 generic.go:356] "Generic (PLEG): container finished" podID="d7d8973b-00ff-4516-9add-786c207b6d53" containerID="bd7be98aa90c74a7f3360b469268c74a2f42023dba2038a30176c330aee9daae" exitCode=0
Aug 05 17:35:05.089752 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:05.089321    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cdd8d8969-b7k2d"
Aug 05 17:35:05.089752 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:05.089359    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cdd8d8969-b7k2d" event={"ID":"d7d8973b-00ff-4516-9add-786c207b6d53","Type":"ContainerDied","Data":"bd7be98aa90c74a7f3360b469268c74a2f42023dba2038a30176c330aee9daae"}
Aug 05 17:35:05.089752 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:05.089403    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cdd8d8969-b7k2d" event={"ID":"d7d8973b-00ff-4516-9add-786c207b6d53","Type":"ContainerDied","Data":"f00f719304c59b940e2bd79d4ad2d5f6a8b0b98224f87b96bd51d3b2f10c45ad"}
Aug 05 17:35:05.089752 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:05.089420    2550 scope.go:117] "RemoveContainer" containerID="bd7be98aa90c74a7f3360b469268c74a2f42023dba2038a30176c330aee9daae"
Aug 05 17:35:05.097914 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:05.097889    2550 scope.go:117] "RemoveContainer" containerID="bd7be98aa90c74a7f3360b469268c74a2f42023dba2038a30176c330aee9daae"
Aug 05 17:35:05.098190 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:35:05.098169    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd7be98aa90c74a7f3360b469268c74a2f42023dba2038a30176c330aee9daae\": container with ID starting with bd7be98aa90c74a7f3360b469268c74a2f42023dba2038a30176c330aee9daae not found: ID does not exist" containerID="bd7be98aa90c74a7f3360b469268c74a2f42023dba2038a30176c330aee9daae"
Aug 05 17:35:05.098273 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:05.098198    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd7be98aa90c74a7f3360b469268c74a2f42023dba2038a30176c330aee9daae"} err="failed to get container status \"bd7be98aa90c74a7f3360b469268c74a2f42023dba2038a30176c330aee9daae\": rpc error: code = NotFound desc = could not find container \"bd7be98aa90c74a7f3360b469268c74a2f42023dba2038a30176c330aee9daae\": container with ID starting with bd7be98aa90c74a7f3360b469268c74a2f42023dba2038a30176c330aee9daae not found: ID does not exist"
Aug 05 17:35:05.115283 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:05.115250    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-cdd8d8969-b7k2d"]
Aug 05 17:35:05.118641 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:05.118616    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-cdd8d8969-b7k2d"]
Aug 05 17:35:05.700655 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:05.700622    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7d8973b-00ff-4516-9add-786c207b6d53" path="/var/lib/kubelet/pods/d7d8973b-00ff-4516-9add-786c207b6d53/volumes"
Aug 05 17:35:58.162472 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:58.162431    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6bb68d8457-dbh88"]
Aug 05 17:35:58.163055 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:58.162731    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6bb68d8457-dbh88" podUID="d2fd17de-f4fc-4330-8f80-604352d6395e" containerName="manager" containerID="cri-o://8d0db0cb4a884094356fbe5aaa9fb1650e6fcc1558d5d9ee29b7ae3a69da9008" gracePeriod=30
Aug 05 17:35:58.342522 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:58.342489    2550 generic.go:356] "Generic (PLEG): container finished" podID="d2fd17de-f4fc-4330-8f80-604352d6395e" containerID="8d0db0cb4a884094356fbe5aaa9fb1650e6fcc1558d5d9ee29b7ae3a69da9008" exitCode=0
Aug 05 17:35:58.342675 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:58.342562    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bb68d8457-dbh88" event={"ID":"d2fd17de-f4fc-4330-8f80-604352d6395e","Type":"ContainerDied","Data":"8d0db0cb4a884094356fbe5aaa9fb1650e6fcc1558d5d9ee29b7ae3a69da9008"}
Aug 05 17:35:58.411118 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:58.411093    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bb68d8457-dbh88"
Aug 05 17:35:58.534106 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:58.534014    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2fd17de-f4fc-4330-8f80-604352d6395e-webhook-cert\") pod \"d2fd17de-f4fc-4330-8f80-604352d6395e\" (UID: \"d2fd17de-f4fc-4330-8f80-604352d6395e\") "
Aug 05 17:35:58.534106 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:58.534061    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9qpx\" (UniqueName: \"kubernetes.io/projected/d2fd17de-f4fc-4330-8f80-604352d6395e-kube-api-access-g9qpx\") pod \"d2fd17de-f4fc-4330-8f80-604352d6395e\" (UID: \"d2fd17de-f4fc-4330-8f80-604352d6395e\") "
Aug 05 17:35:58.536350 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:58.536317    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2fd17de-f4fc-4330-8f80-604352d6395e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d2fd17de-f4fc-4330-8f80-604352d6395e" (UID: "d2fd17de-f4fc-4330-8f80-604352d6395e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:35:58.536350 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:58.536335    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2fd17de-f4fc-4330-8f80-604352d6395e-kube-api-access-g9qpx" (OuterVolumeSpecName: "kube-api-access-g9qpx") pod "d2fd17de-f4fc-4330-8f80-604352d6395e" (UID: "d2fd17de-f4fc-4330-8f80-604352d6395e"). InnerVolumeSpecName "kube-api-access-g9qpx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:35:58.634856 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:58.634821    2550 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2fd17de-f4fc-4330-8f80-604352d6395e-webhook-cert\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:35:58.634856 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:58.634851    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9qpx\" (UniqueName: \"kubernetes.io/projected/d2fd17de-f4fc-4330-8f80-604352d6395e-kube-api-access-g9qpx\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:35:59.348328 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:59.348302    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bb68d8457-dbh88"
Aug 05 17:35:59.348770 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:59.348301    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bb68d8457-dbh88" event={"ID":"d2fd17de-f4fc-4330-8f80-604352d6395e","Type":"ContainerDied","Data":"325b3af4ea0d5a1d2a347a2e1aaf51836c4f049f4f35a1cad35fa31535f1590a"}
Aug 05 17:35:59.348770 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:59.348436    2550 scope.go:117] "RemoveContainer" containerID="8d0db0cb4a884094356fbe5aaa9fb1650e6fcc1558d5d9ee29b7ae3a69da9008"
Aug 05 17:35:59.371340 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:59.371301    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6bb68d8457-dbh88"]
Aug 05 17:35:59.377422 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:59.377384    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6bb68d8457-dbh88"]
Aug 05 17:35:59.702347 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:35:59.702307    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2fd17de-f4fc-4330-8f80-604352d6395e" path="/var/lib/kubelet/pods/d2fd17de-f4fc-4330-8f80-604352d6395e/volumes"
Aug 05 17:36:33.500570 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:36:33.500544    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 17:36:33.653804 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:36:33.653780    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 17:37:00.150397 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:00.150363    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6bb68d8457-zm8n9"]
Aug 05 17:37:00.150891 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:00.150744    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7d8973b-00ff-4516-9add-786c207b6d53" containerName="authorino"
Aug 05 17:37:00.150891 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:00.150756    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7d8973b-00ff-4516-9add-786c207b6d53" containerName="authorino"
Aug 05 17:37:00.150891 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:00.150767    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2fd17de-f4fc-4330-8f80-604352d6395e" containerName="manager"
Aug 05 17:37:00.150891 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:00.150773    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2fd17de-f4fc-4330-8f80-604352d6395e" containerName="manager"
Aug 05 17:37:00.150891 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:00.150838    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7d8973b-00ff-4516-9add-786c207b6d53" containerName="authorino"
Aug 05 17:37:00.150891 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:00.150848    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2fd17de-f4fc-4330-8f80-604352d6395e" containerName="manager"
Aug 05 17:37:00.153650 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:00.153635    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bb68d8457-zm8n9"
Aug 05 17:37:00.157176 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:00.157146    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 05 17:37:00.157293 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:00.157154    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 05 17:37:00.157293 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:00.157154    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-567ln\""
Aug 05 17:37:00.165106 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:00.165084    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bb68d8457-zm8n9"]
Aug 05 17:37:00.264765 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:00.264734    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1e4ee900-df98-41d0-aad2-bb27328ddc50-webhook-cert\") pod \"maas-controller-6bb68d8457-zm8n9\" (UID: \"1e4ee900-df98-41d0-aad2-bb27328ddc50\") " pod="opendatahub/maas-controller-6bb68d8457-zm8n9"
Aug 05 17:37:00.264944 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:00.264810    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lc97\" (UniqueName: \"kubernetes.io/projected/1e4ee900-df98-41d0-aad2-bb27328ddc50-kube-api-access-9lc97\") pod \"maas-controller-6bb68d8457-zm8n9\" (UID: \"1e4ee900-df98-41d0-aad2-bb27328ddc50\") " pod="opendatahub/maas-controller-6bb68d8457-zm8n9"
Aug 05 17:37:00.366172 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:00.366139    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lc97\" (UniqueName: \"kubernetes.io/projected/1e4ee900-df98-41d0-aad2-bb27328ddc50-kube-api-access-9lc97\") pod \"maas-controller-6bb68d8457-zm8n9\" (UID: \"1e4ee900-df98-41d0-aad2-bb27328ddc50\") " pod="opendatahub/maas-controller-6bb68d8457-zm8n9"
Aug 05 17:37:00.366320 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:00.366206    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1e4ee900-df98-41d0-aad2-bb27328ddc50-webhook-cert\") pod \"maas-controller-6bb68d8457-zm8n9\" (UID: \"1e4ee900-df98-41d0-aad2-bb27328ddc50\") " pod="opendatahub/maas-controller-6bb68d8457-zm8n9"
Aug 05 17:37:00.368588 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:00.368565    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1e4ee900-df98-41d0-aad2-bb27328ddc50-webhook-cert\") pod \"maas-controller-6bb68d8457-zm8n9\" (UID: \"1e4ee900-df98-41d0-aad2-bb27328ddc50\") " pod="opendatahub/maas-controller-6bb68d8457-zm8n9"
Aug 05 17:37:00.375801 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:00.375765    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lc97\" (UniqueName: \"kubernetes.io/projected/1e4ee900-df98-41d0-aad2-bb27328ddc50-kube-api-access-9lc97\") pod \"maas-controller-6bb68d8457-zm8n9\" (UID: \"1e4ee900-df98-41d0-aad2-bb27328ddc50\") " pod="opendatahub/maas-controller-6bb68d8457-zm8n9"
Aug 05 17:37:00.464451 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:00.464414    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bb68d8457-zm8n9"
Aug 05 17:37:00.595502 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:00.595476    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bb68d8457-zm8n9"]
Aug 05 17:37:00.597927 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:37:00.597889    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e4ee900_df98_41d0_aad2_bb27328ddc50.slice/crio-9197fe7d5145c6eb501cd588f7930065cc04d1b1cf7f9b56c73f3d11b360f5f9 WatchSource:0}: Error finding container 9197fe7d5145c6eb501cd588f7930065cc04d1b1cf7f9b56c73f3d11b360f5f9: Status 404 returned error can't find the container with id 9197fe7d5145c6eb501cd588f7930065cc04d1b1cf7f9b56c73f3d11b360f5f9
Aug 05 17:37:00.617928 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:00.617895    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bb68d8457-zm8n9" event={"ID":"1e4ee900-df98-41d0-aad2-bb27328ddc50","Type":"ContainerStarted","Data":"9197fe7d5145c6eb501cd588f7930065cc04d1b1cf7f9b56c73f3d11b360f5f9"}
Aug 05 17:37:01.624740 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:01.624649    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bb68d8457-zm8n9" event={"ID":"1e4ee900-df98-41d0-aad2-bb27328ddc50","Type":"ContainerStarted","Data":"8f19fa910747b7d861ef58c145502a2d8e34f61eb92d9e699d1356e8f27abc97"}
Aug 05 17:37:01.625092 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:01.624759    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6bb68d8457-zm8n9"
Aug 05 17:37:01.640616 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:01.640575    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6bb68d8457-zm8n9" podStartSLOduration=1.014807754 podStartE2EDuration="1.640560218s" totalImagesPullingTime="625.752464ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:37:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:37:01.639585065 +0000 UTC m=+928.621162493" watchObservedRunningTime="2026-08-05 17:37:01.640560218 +0000 UTC m=+928.622137646"
Aug 05 17:37:12.632095 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:12.632061    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6bb68d8457-zm8n9"
Aug 05 17:37:17.258695 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:17.258656    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6bb68d8457-zm8n9"]
Aug 05 17:37:17.259141 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:17.258864    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6bb68d8457-zm8n9" podUID="1e4ee900-df98-41d0-aad2-bb27328ddc50" containerName="manager" containerID="cri-o://8f19fa910747b7d861ef58c145502a2d8e34f61eb92d9e699d1356e8f27abc97" gracePeriod=30
Aug 05 17:37:17.510404 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:17.510334    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bb68d8457-zm8n9"
Aug 05 17:37:17.619330 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:17.619297    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lc97\" (UniqueName: \"kubernetes.io/projected/1e4ee900-df98-41d0-aad2-bb27328ddc50-kube-api-access-9lc97\") pod \"1e4ee900-df98-41d0-aad2-bb27328ddc50\" (UID: \"1e4ee900-df98-41d0-aad2-bb27328ddc50\") "
Aug 05 17:37:17.619536 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:17.619349    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1e4ee900-df98-41d0-aad2-bb27328ddc50-webhook-cert\") pod \"1e4ee900-df98-41d0-aad2-bb27328ddc50\" (UID: \"1e4ee900-df98-41d0-aad2-bb27328ddc50\") "
Aug 05 17:37:17.621335 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:17.621294    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e4ee900-df98-41d0-aad2-bb27328ddc50-kube-api-access-9lc97" (OuterVolumeSpecName: "kube-api-access-9lc97") pod "1e4ee900-df98-41d0-aad2-bb27328ddc50" (UID: "1e4ee900-df98-41d0-aad2-bb27328ddc50"). InnerVolumeSpecName "kube-api-access-9lc97". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:37:17.621335 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:17.621320    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e4ee900-df98-41d0-aad2-bb27328ddc50-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "1e4ee900-df98-41d0-aad2-bb27328ddc50" (UID: "1e4ee900-df98-41d0-aad2-bb27328ddc50"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:37:17.699968 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:17.699934    2550 generic.go:356] "Generic (PLEG): container finished" podID="1e4ee900-df98-41d0-aad2-bb27328ddc50" containerID="8f19fa910747b7d861ef58c145502a2d8e34f61eb92d9e699d1356e8f27abc97" exitCode=0
Aug 05 17:37:17.700135 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:17.700022    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bb68d8457-zm8n9"
Aug 05 17:37:17.700452 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:17.700426    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bb68d8457-zm8n9" event={"ID":"1e4ee900-df98-41d0-aad2-bb27328ddc50","Type":"ContainerDied","Data":"8f19fa910747b7d861ef58c145502a2d8e34f61eb92d9e699d1356e8f27abc97"}
Aug 05 17:37:17.700541 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:17.700481    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bb68d8457-zm8n9" event={"ID":"1e4ee900-df98-41d0-aad2-bb27328ddc50","Type":"ContainerDied","Data":"9197fe7d5145c6eb501cd588f7930065cc04d1b1cf7f9b56c73f3d11b360f5f9"}
Aug 05 17:37:17.700541 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:17.700512    2550 scope.go:117] "RemoveContainer" containerID="8f19fa910747b7d861ef58c145502a2d8e34f61eb92d9e699d1356e8f27abc97"
Aug 05 17:37:17.710264 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:17.710245    2550 scope.go:117] "RemoveContainer" containerID="8f19fa910747b7d861ef58c145502a2d8e34f61eb92d9e699d1356e8f27abc97"
Aug 05 17:37:17.710592 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:37:17.710566    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f19fa910747b7d861ef58c145502a2d8e34f61eb92d9e699d1356e8f27abc97\": container with ID starting with 8f19fa910747b7d861ef58c145502a2d8e34f61eb92d9e699d1356e8f27abc97 not found: ID does not exist" containerID="8f19fa910747b7d861ef58c145502a2d8e34f61eb92d9e699d1356e8f27abc97"
Aug 05 17:37:17.710709 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:17.710605    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f19fa910747b7d861ef58c145502a2d8e34f61eb92d9e699d1356e8f27abc97"} err="failed to get container status \"8f19fa910747b7d861ef58c145502a2d8e34f61eb92d9e699d1356e8f27abc97\": rpc error: code = NotFound desc = could not find container \"8f19fa910747b7d861ef58c145502a2d8e34f61eb92d9e699d1356e8f27abc97\": container with ID starting with 8f19fa910747b7d861ef58c145502a2d8e34f61eb92d9e699d1356e8f27abc97 not found: ID does not exist"
Aug 05 17:37:17.720026 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:17.719999    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lc97\" (UniqueName: \"kubernetes.io/projected/1e4ee900-df98-41d0-aad2-bb27328ddc50-kube-api-access-9lc97\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:37:17.720026 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:17.720025    2550 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1e4ee900-df98-41d0-aad2-bb27328ddc50-webhook-cert\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:37:17.724712 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:17.724672    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6bb68d8457-zm8n9"]
Aug 05 17:37:17.728376 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:17.728353    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6bb68d8457-zm8n9"]
Aug 05 17:37:18.473565 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:18.473533    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6bb68d8457-cqsfl"]
Aug 05 17:37:18.474033 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:18.473873    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e4ee900-df98-41d0-aad2-bb27328ddc50" containerName="manager"
Aug 05 17:37:18.474033 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:18.473884    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e4ee900-df98-41d0-aad2-bb27328ddc50" containerName="manager"
Aug 05 17:37:18.474033 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:18.473951    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e4ee900-df98-41d0-aad2-bb27328ddc50" containerName="manager"
Aug 05 17:37:18.478140 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:18.478120    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bb68d8457-cqsfl"
Aug 05 17:37:18.480476 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:18.480459    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 05 17:37:18.481436 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:18.481420    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-567ln\""
Aug 05 17:37:18.481436 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:18.481427    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 05 17:37:18.487533 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:18.487501    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bb68d8457-cqsfl"]
Aug 05 17:37:18.626207 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:18.626171    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf4d0eeb-01e0-4740-b068-1b1a838dec25-webhook-cert\") pod \"maas-controller-6bb68d8457-cqsfl\" (UID: \"cf4d0eeb-01e0-4740-b068-1b1a838dec25\") " pod="opendatahub/maas-controller-6bb68d8457-cqsfl"
Aug 05 17:37:18.626207 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:18.626212    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lt7h\" (UniqueName: \"kubernetes.io/projected/cf4d0eeb-01e0-4740-b068-1b1a838dec25-kube-api-access-2lt7h\") pod \"maas-controller-6bb68d8457-cqsfl\" (UID: \"cf4d0eeb-01e0-4740-b068-1b1a838dec25\") " pod="opendatahub/maas-controller-6bb68d8457-cqsfl"
Aug 05 17:37:18.727010 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:18.726914    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf4d0eeb-01e0-4740-b068-1b1a838dec25-webhook-cert\") pod \"maas-controller-6bb68d8457-cqsfl\" (UID: \"cf4d0eeb-01e0-4740-b068-1b1a838dec25\") " pod="opendatahub/maas-controller-6bb68d8457-cqsfl"
Aug 05 17:37:18.727010 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:18.726957    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lt7h\" (UniqueName: \"kubernetes.io/projected/cf4d0eeb-01e0-4740-b068-1b1a838dec25-kube-api-access-2lt7h\") pod \"maas-controller-6bb68d8457-cqsfl\" (UID: \"cf4d0eeb-01e0-4740-b068-1b1a838dec25\") " pod="opendatahub/maas-controller-6bb68d8457-cqsfl"
Aug 05 17:37:18.729513 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:18.729478    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf4d0eeb-01e0-4740-b068-1b1a838dec25-webhook-cert\") pod \"maas-controller-6bb68d8457-cqsfl\" (UID: \"cf4d0eeb-01e0-4740-b068-1b1a838dec25\") " pod="opendatahub/maas-controller-6bb68d8457-cqsfl"
Aug 05 17:37:18.735247 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:18.735222    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lt7h\" (UniqueName: \"kubernetes.io/projected/cf4d0eeb-01e0-4740-b068-1b1a838dec25-kube-api-access-2lt7h\") pod \"maas-controller-6bb68d8457-cqsfl\" (UID: \"cf4d0eeb-01e0-4740-b068-1b1a838dec25\") " pod="opendatahub/maas-controller-6bb68d8457-cqsfl"
Aug 05 17:37:18.788785 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:18.788748    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bb68d8457-cqsfl"
Aug 05 17:37:18.923027 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:18.922998    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bb68d8457-cqsfl"]
Aug 05 17:37:18.925564 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:37:18.925539    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf4d0eeb_01e0_4740_b068_1b1a838dec25.slice/crio-9d251d6b107c51a3211cc14fbffce23a9f0a050c0712487dc2fc18a98686a95d WatchSource:0}: Error finding container 9d251d6b107c51a3211cc14fbffce23a9f0a050c0712487dc2fc18a98686a95d: Status 404 returned error can't find the container with id 9d251d6b107c51a3211cc14fbffce23a9f0a050c0712487dc2fc18a98686a95d
Aug 05 17:37:19.700597 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:19.700563    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e4ee900-df98-41d0-aad2-bb27328ddc50" path="/var/lib/kubelet/pods/1e4ee900-df98-41d0-aad2-bb27328ddc50/volumes"
Aug 05 17:37:19.712515 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:19.712487    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bb68d8457-cqsfl" event={"ID":"cf4d0eeb-01e0-4740-b068-1b1a838dec25","Type":"ContainerStarted","Data":"8df9386236da2174645ba0696ee3266c13ec6b99b1306abb8e366c34de870e56"}
Aug 05 17:37:19.712735 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:19.712544    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bb68d8457-cqsfl" event={"ID":"cf4d0eeb-01e0-4740-b068-1b1a838dec25","Type":"ContainerStarted","Data":"9d251d6b107c51a3211cc14fbffce23a9f0a050c0712487dc2fc18a98686a95d"}
Aug 05 17:37:19.712735 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:19.712603    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6bb68d8457-cqsfl"
Aug 05 17:37:19.732409 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:19.732356    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6bb68d8457-cqsfl" podStartSLOduration=1.309148178 podStartE2EDuration="1.732337123s" totalImagesPullingTime="423.188945ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:37:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:37:19.731171587 +0000 UTC m=+946.712749016" watchObservedRunningTime="2026-08-05 17:37:19.732337123 +0000 UTC m=+946.713914552"
Aug 05 17:37:30.720472 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:37:30.720442    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6bb68d8457-cqsfl"
Aug 05 17:41:33.528419 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:41:33.528394    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 17:41:33.680522 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:41:33.680497    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 17:46:33.554518 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:46:33.554494    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 17:46:33.705049 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:46:33.705024    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 17:48:05.982684 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:05.982647    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"]
Aug 05 17:48:05.986047 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:05.986031    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"
Aug 05 17:48:05.988515 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:05.988491    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-836d574c-kserve-self-signed-certs\""
Aug 05 17:48:06.006093 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:06.006071    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"]
Aug 05 17:48:06.075014 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:06.074979    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-tls-certs\") pod \"e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk\" (UID: \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\") " pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"
Aug 05 17:48:06.075199 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:06.075046    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-home\") pod \"e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk\" (UID: \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\") " pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"
Aug 05 17:48:06.075199 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:06.075082    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-dshm\") pod \"e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk\" (UID: \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\") " pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"
Aug 05 17:48:06.075199 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:06.075126    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-model-cache\") pod \"e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk\" (UID: \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\") " pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"
Aug 05 17:48:06.075368 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:06.075213    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pgvd\" (UniqueName: \"kubernetes.io/projected/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-kube-api-access-6pgvd\") pod \"e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk\" (UID: \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\") " pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"
Aug 05 17:48:06.075368 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:06.075254    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-tmp-dir\") pod \"e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk\" (UID: \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\") " pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"
Aug 05 17:48:06.176246 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:06.176209    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-tmp-dir\") pod \"e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk\" (UID: \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\") " pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"
Aug 05 17:48:06.176432 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:06.176280    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-tls-certs\") pod \"e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk\" (UID: \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\") " pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"
Aug 05 17:48:06.176432 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:06.176321    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-home\") pod \"e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk\" (UID: \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\") " pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"
Aug 05 17:48:06.176432 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:06.176348    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-dshm\") pod \"e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk\" (UID: \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\") " pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"
Aug 05 17:48:06.176432 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:06.176386    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-model-cache\") pod \"e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk\" (UID: \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\") " pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"
Aug 05 17:48:06.176432 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:06.176426    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pgvd\" (UniqueName: \"kubernetes.io/projected/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-kube-api-access-6pgvd\") pod \"e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk\" (UID: \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\") " pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"
Aug 05 17:48:06.176718 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:06.176677    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-tmp-dir\") pod \"e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk\" (UID: \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\") " pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"
Aug 05 17:48:06.176786 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:06.176756    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-home\") pod \"e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk\" (UID: \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\") " pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"
Aug 05 17:48:06.176926 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:06.176898    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-model-cache\") pod \"e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk\" (UID: \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\") " pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"
Aug 05 17:48:06.178565 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:06.178536    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-dshm\") pod \"e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk\" (UID: \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\") " pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"
Aug 05 17:48:06.178882 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:06.178861    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-tls-certs\") pod \"e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk\" (UID: \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\") " pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"
Aug 05 17:48:06.185292 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:06.185262    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pgvd\" (UniqueName: \"kubernetes.io/projected/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-kube-api-access-6pgvd\") pod \"e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk\" (UID: \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\") " pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"
Aug 05 17:48:06.295557 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:06.295469    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"
Aug 05 17:48:06.444344 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:06.444315    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"]
Aug 05 17:48:06.446520 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:48:06.446492    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6375a3e4_ec06_4fa3_976e_e2b5fa7f5ee5.slice/crio-fe319ba0c9de622add6e1409eee49e09957be52ca4cdd652487364ce0606a7a8 WatchSource:0}: Error finding container fe319ba0c9de622add6e1409eee49e09957be52ca4cdd652487364ce0606a7a8: Status 404 returned error can't find the container with id fe319ba0c9de622add6e1409eee49e09957be52ca4cdd652487364ce0606a7a8
Aug 05 17:48:06.658187 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:06.658153    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk" event={"ID":"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5","Type":"ContainerStarted","Data":"e17138ed7b1c73127f56f593f5beb3750522eda1e2f2ff7ac2f601814275c746"}
Aug 05 17:48:06.658187 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:06.658195    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk" event={"ID":"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5","Type":"ContainerStarted","Data":"fe319ba0c9de622add6e1409eee49e09957be52ca4cdd652487364ce0606a7a8"}
Aug 05 17:48:06.678580 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:06.678539    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk" podStartSLOduration=1.678525388 podStartE2EDuration="1.678525388s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:48:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:48:06.677038929 +0000 UTC m=+1593.658616369" watchObservedRunningTime="2026-08-05 17:48:06.678525388 +0000 UTC m=+1593.660102858"
Aug 05 17:48:16.296335 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:16.296297    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"
Aug 05 17:48:16.296814 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:16.296347    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"
Aug 05 17:48:16.309227 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:16.309201    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"
Aug 05 17:48:16.713821 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:16.713793    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"
Aug 05 17:48:50.317980 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:48:50.317946    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"]
Aug 05 17:49:05.337642 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.337598    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk" podUID="6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5" containerName="main" containerID="cri-o://e17138ed7b1c73127f56f593f5beb3750522eda1e2f2ff7ac2f601814275c746" gracePeriod=45
Aug 05 17:49:05.592140 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.592091    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"
Aug 05 17:49:05.697073 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.697049    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-tmp-dir\") pod \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\" (UID: \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\") "
Aug 05 17:49:05.697237 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.697122    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-model-cache\") pod \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\" (UID: \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\") "
Aug 05 17:49:05.697237 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.697141    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6pgvd\" (UniqueName: \"kubernetes.io/projected/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-kube-api-access-6pgvd\") pod \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\" (UID: \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\") "
Aug 05 17:49:05.697237 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.697164    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-dshm\") pod \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\" (UID: \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\") "
Aug 05 17:49:05.697237 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.697193    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-tls-certs\") pod \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\" (UID: \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\") "
Aug 05 17:49:05.697237 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.697209    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-home\") pod \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\" (UID: \"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5\") "
Aug 05 17:49:05.697493 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.697314    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5" (UID: "6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:49:05.697493 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.697481    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-tmp-dir\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:49:05.697587 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.697499    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-model-cache" (OuterVolumeSpecName: "model-cache") pod "6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5" (UID: "6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:49:05.697587 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.697511    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-home" (OuterVolumeSpecName: "home") pod "6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5" (UID: "6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:49:05.699515 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.699480    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-dshm" (OuterVolumeSpecName: "dshm") pod "6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5" (UID: "6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:49:05.699515 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.699504    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-kube-api-access-6pgvd" (OuterVolumeSpecName: "kube-api-access-6pgvd") pod "6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5" (UID: "6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5"). InnerVolumeSpecName "kube-api-access-6pgvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:49:05.699648 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.699568    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5" (UID: "6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:49:05.798899 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.798870    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-model-cache\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:49:05.798899 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.798896    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6pgvd\" (UniqueName: \"kubernetes.io/projected/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-kube-api-access-6pgvd\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:49:05.799056 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.798906    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-dshm\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:49:05.799056 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.798916    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-tls-certs\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:49:05.799056 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.798924    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5-home\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:49:05.931397 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.931363    2550 generic.go:356] "Generic (PLEG): container finished" podID="6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5" containerID="e17138ed7b1c73127f56f593f5beb3750522eda1e2f2ff7ac2f601814275c746" exitCode=0
Aug 05 17:49:05.931550 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.931409    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"
Aug 05 17:49:05.931550 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.931449    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk" event={"ID":"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5","Type":"ContainerDied","Data":"e17138ed7b1c73127f56f593f5beb3750522eda1e2f2ff7ac2f601814275c746"}
Aug 05 17:49:05.931550 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.931500    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk" event={"ID":"6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5","Type":"ContainerDied","Data":"fe319ba0c9de622add6e1409eee49e09957be52ca4cdd652487364ce0606a7a8"}
Aug 05 17:49:05.931550 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.931517    2550 scope.go:117] "RemoveContainer" containerID="e17138ed7b1c73127f56f593f5beb3750522eda1e2f2ff7ac2f601814275c746"
Aug 05 17:49:05.939682 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.939664    2550 scope.go:117] "RemoveContainer" containerID="e17138ed7b1c73127f56f593f5beb3750522eda1e2f2ff7ac2f601814275c746"
Aug 05 17:49:05.939956 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:49:05.939937    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e17138ed7b1c73127f56f593f5beb3750522eda1e2f2ff7ac2f601814275c746\": container with ID starting with e17138ed7b1c73127f56f593f5beb3750522eda1e2f2ff7ac2f601814275c746 not found: ID does not exist" containerID="e17138ed7b1c73127f56f593f5beb3750522eda1e2f2ff7ac2f601814275c746"
Aug 05 17:49:05.940023 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.939964    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e17138ed7b1c73127f56f593f5beb3750522eda1e2f2ff7ac2f601814275c746"} err="failed to get container status \"e17138ed7b1c73127f56f593f5beb3750522eda1e2f2ff7ac2f601814275c746\": rpc error: code = NotFound desc = could not find container \"e17138ed7b1c73127f56f593f5beb3750522eda1e2f2ff7ac2f601814275c746\": container with ID starting with e17138ed7b1c73127f56f593f5beb3750522eda1e2f2ff7ac2f601814275c746 not found: ID does not exist"
Aug 05 17:49:05.947951 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.947926    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"]
Aug 05 17:49:05.952819 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:05.952796    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-836d574c-kserve-66d9dcf78c-rxwpk"]
Aug 05 17:49:07.700436 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:49:07.700403    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5" path="/var/lib/kubelet/pods/6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5/volumes"
Aug 05 17:51:33.588388 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:51:33.588361    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 17:51:33.735315 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:51:33.735289    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 17:53:00.930259 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:00.930174    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"]
Aug 05 17:53:00.930742 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:00.930547    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5" containerName="main"
Aug 05 17:53:00.930742 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:00.930559    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5" containerName="main"
Aug 05 17:53:00.930742 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:00.930625    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="6375a3e4-ec06-4fa3-976e-e2b5fa7f5ee5" containerName="main"
Aug 05 17:53:00.933680 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:00.933664    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:00.936219 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:00.936199    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-b570cb7c-openshift-default-dockercfg-998rs\""
Aug 05 17:53:00.958642 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:00.958612    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"]
Aug 05 17:53:01.042906 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.042871    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-workload-certs\") pod \"e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") " pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:01.043084 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.042920    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xqv8\" (UniqueName: \"kubernetes.io/projected/57e78de5-8964-4e21-aa25-d6079c0bca5e-kube-api-access-8xqv8\") pod \"e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") " pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:01.043084 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.043002    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-workload-socket\") pod \"e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") " pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:01.043084 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.043042    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-istio-data\") pod \"e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") " pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:01.043084 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.043066    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-credential-socket\") pod \"e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") " pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:01.043265 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.043090    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57e78de5-8964-4e21-aa25-d6079c0bca5e-istio-token\") pod \"e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") " pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:01.043265 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.043113    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-istio-envoy\") pod \"e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") " pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:01.043265 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.043150    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57e78de5-8964-4e21-aa25-d6079c0bca5e-istiod-ca-cert\") pod \"e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") " pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:01.043265 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.043176    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57e78de5-8964-4e21-aa25-d6079c0bca5e-istio-podinfo\") pod \"e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") " pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:01.144202 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.144167    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57e78de5-8964-4e21-aa25-d6079c0bca5e-istiod-ca-cert\") pod \"e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") " pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:01.144380 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.144208    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57e78de5-8964-4e21-aa25-d6079c0bca5e-istio-podinfo\") pod \"e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") " pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:01.144380 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.144238    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-workload-certs\") pod \"e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") " pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:01.144380 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.144280    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xqv8\" (UniqueName: \"kubernetes.io/projected/57e78de5-8964-4e21-aa25-d6079c0bca5e-kube-api-access-8xqv8\") pod \"e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") " pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:01.144380 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.144325    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-workload-socket\") pod \"e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") " pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:01.144380 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.144353    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-istio-data\") pod \"e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") " pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:01.144380 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.144377    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-credential-socket\") pod \"e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") " pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:01.144759 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.144399    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57e78de5-8964-4e21-aa25-d6079c0bca5e-istio-token\") pod \"e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") " pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:01.144759 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.144420    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-istio-envoy\") pod \"e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") " pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:01.144877 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.144762    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-workload-certs\") pod \"e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") " pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:01.144877 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.144803    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-workload-socket\") pod \"e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") " pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:01.144877 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.144866    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-credential-socket\") pod \"e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") " pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:01.145030 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.145012    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57e78de5-8964-4e21-aa25-d6079c0bca5e-istiod-ca-cert\") pod \"e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") " pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:01.145100 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.145080    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-istio-data\") pod \"e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") " pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:01.146597 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.146577    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-istio-envoy\") pod \"e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") " pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:01.146850 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.146832    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57e78de5-8964-4e21-aa25-d6079c0bca5e-istio-podinfo\") pod \"e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") " pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:01.153181 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.153159    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57e78de5-8964-4e21-aa25-d6079c0bca5e-istio-token\") pod \"e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") " pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:01.153271 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.153160    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xqv8\" (UniqueName: \"kubernetes.io/projected/57e78de5-8964-4e21-aa25-d6079c0bca5e-kube-api-access-8xqv8\") pod \"e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") " pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:01.244284 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.244199    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:01.390341 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.390314    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"]
Aug 05 17:53:01.392756 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:53:01.392724    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57e78de5_8964_4e21_aa25_d6079c0bca5e.slice/crio-df9171ab2372ec7606c5e7dfbd86c9dee20582ff8a1b70114298909593793296 WatchSource:0}: Error finding container df9171ab2372ec7606c5e7dfbd86c9dee20582ff8a1b70114298909593793296: Status 404 returned error can't find the container with id df9171ab2372ec7606c5e7dfbd86c9dee20582ff8a1b70114298909593793296
Aug 05 17:53:01.394896 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.394866    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 17:53:01.394984 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.394929    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 17:53:01.394984 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:01.394962    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 17:53:02.015737 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:02.015684    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn" event={"ID":"57e78de5-8964-4e21-aa25-d6079c0bca5e","Type":"ContainerStarted","Data":"e1c9ac34bc42d81f7db7cbefefd04dd30c3e39d1a22536e4e0d4c2794043c69e"}
Aug 05 17:53:02.016127 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:02.015761    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn" event={"ID":"57e78de5-8964-4e21-aa25-d6079c0bca5e","Type":"ContainerStarted","Data":"df9171ab2372ec7606c5e7dfbd86c9dee20582ff8a1b70114298909593793296"}
Aug 05 17:53:02.052087 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:02.052031    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn" podStartSLOduration=2.052015506 podStartE2EDuration="2.052015506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:53:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:53:02.049224829 +0000 UTC m=+1889.030802258" watchObservedRunningTime="2026-08-05 17:53:02.052015506 +0000 UTC m=+1889.033592935"
Aug 05 17:53:02.244505 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:02.244473    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:02.250725 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:02.250676    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:03.020083 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:03.020049    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:03.021111 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:03.021091    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:04.562950 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:04.562917    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm"]
Aug 05 17:53:04.566291 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:04.566274    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm"
Aug 05 17:53:04.569801 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:04.569783    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-b570cb7c-serving-cert\""
Aug 05 17:53:04.584935 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:04.584909    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm"]
Aug 05 17:53:04.675297 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:04.675264    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zst4z\" (UniqueName: \"kubernetes.io/projected/dcd77a0b-8be1-4b39-9fef-b290e13b0a61-kube-api-access-zst4z\") pod \"maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm\" (UID: \"dcd77a0b-8be1-4b39-9fef-b290e13b0a61\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm"
Aug 05 17:53:04.675467 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:04.675315    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcd77a0b-8be1-4b39-9fef-b290e13b0a61-openshift-service-ca\") pod \"maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm\" (UID: \"dcd77a0b-8be1-4b39-9fef-b290e13b0a61\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm"
Aug 05 17:53:04.675509 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:04.675455    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcd77a0b-8be1-4b39-9fef-b290e13b0a61-maas-api-tls\") pod \"maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm\" (UID: \"dcd77a0b-8be1-4b39-9fef-b290e13b0a61\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm"
Aug 05 17:53:04.776977 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:04.776934    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zst4z\" (UniqueName: \"kubernetes.io/projected/dcd77a0b-8be1-4b39-9fef-b290e13b0a61-kube-api-access-zst4z\") pod \"maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm\" (UID: \"dcd77a0b-8be1-4b39-9fef-b290e13b0a61\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm"
Aug 05 17:53:04.777191 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:04.777092    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcd77a0b-8be1-4b39-9fef-b290e13b0a61-openshift-service-ca\") pod \"maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm\" (UID: \"dcd77a0b-8be1-4b39-9fef-b290e13b0a61\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm"
Aug 05 17:53:04.777259 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:04.777205    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcd77a0b-8be1-4b39-9fef-b290e13b0a61-maas-api-tls\") pod \"maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm\" (UID: \"dcd77a0b-8be1-4b39-9fef-b290e13b0a61\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm"
Aug 05 17:53:04.777411 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:53:04.777386    2550 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-serving-cert: secret "maas-api-e2e-ait-b570cb7c-serving-cert" not found
Aug 05 17:53:04.777492 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:53:04.777480    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcd77a0b-8be1-4b39-9fef-b290e13b0a61-maas-api-tls podName:dcd77a0b-8be1-4b39-9fef-b290e13b0a61 nodeName:}" failed. No retries permitted until 2026-08-05 17:53:05.277456114 +0000 UTC m=+1892.259033524 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dcd77a0b-8be1-4b39-9fef-b290e13b0a61-maas-api-tls") pod "maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm" (UID: "dcd77a0b-8be1-4b39-9fef-b290e13b0a61") : secret "maas-api-e2e-ait-b570cb7c-serving-cert" not found
Aug 05 17:53:04.777880 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:04.777864    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcd77a0b-8be1-4b39-9fef-b290e13b0a61-openshift-service-ca\") pod \"maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm\" (UID: \"dcd77a0b-8be1-4b39-9fef-b290e13b0a61\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm"
Aug 05 17:53:04.788340 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:04.788317    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zst4z\" (UniqueName: \"kubernetes.io/projected/dcd77a0b-8be1-4b39-9fef-b290e13b0a61-kube-api-access-zst4z\") pod \"maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm\" (UID: \"dcd77a0b-8be1-4b39-9fef-b290e13b0a61\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm"
Aug 05 17:53:05.281803 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:05.281767    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcd77a0b-8be1-4b39-9fef-b290e13b0a61-maas-api-tls\") pod \"maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm\" (UID: \"dcd77a0b-8be1-4b39-9fef-b290e13b0a61\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm"
Aug 05 17:53:05.284185 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:05.284154    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcd77a0b-8be1-4b39-9fef-b290e13b0a61-maas-api-tls\") pod \"maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm\" (UID: \"dcd77a0b-8be1-4b39-9fef-b290e13b0a61\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm"
Aug 05 17:53:05.476756 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:05.476691    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm"
Aug 05 17:53:05.651738 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:05.651691    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm"]
Aug 05 17:53:05.655505 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:53:05.655480    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcd77a0b_8be1_4b39_9fef_b290e13b0a61.slice/crio-4798ad86c5709b10d476bdea9d8614aeba77e5ab9715cf646571fe4e44091efb WatchSource:0}: Error finding container 4798ad86c5709b10d476bdea9d8614aeba77e5ab9715cf646571fe4e44091efb: Status 404 returned error can't find the container with id 4798ad86c5709b10d476bdea9d8614aeba77e5ab9715cf646571fe4e44091efb
Aug 05 17:53:05.657462 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:05.657446    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 17:53:06.034633 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:06.034598    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm" event={"ID":"dcd77a0b-8be1-4b39-9fef-b290e13b0a61","Type":"ContainerStarted","Data":"4798ad86c5709b10d476bdea9d8614aeba77e5ab9715cf646571fe4e44091efb"}
Aug 05 17:53:07.046428 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:07.046390    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm" event={"ID":"dcd77a0b-8be1-4b39-9fef-b290e13b0a61","Type":"ContainerStarted","Data":"338fae5a163d74d8d296882945411e633e18f29fd8acc9c32de84cb40bbdb9f7"}
Aug 05 17:53:07.046917 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:07.046526    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm"
Aug 05 17:53:13.060875 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:13.060846    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm"
Aug 05 17:53:13.081662 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:13.081614    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm" podStartSLOduration=8.69263423 podStartE2EDuration="9.081597662s" totalImagesPullingTime="388.963432ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:53:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:53:07.066096306 +0000 UTC m=+1894.047673734" watchObservedRunningTime="2026-08-05 17:53:13.081597662 +0000 UTC m=+1900.063175090"
Aug 05 17:53:18.047363 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:18.046827    2550 patch_prober.go:28] interesting pod/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": read tcp 10.132.0.2:55070->10.132.0.45:15021: read: connection reset by peer" start-of-body=
Aug 05 17:53:18.047363 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:18.046888    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn" podUID="57e78de5-8964-4e21-aa25-d6079c0bca5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": read tcp 10.132.0.2:55070->10.132.0.45:15021: read: connection reset by peer"
Aug 05 17:53:18.101692 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:18.101664    2550 generic.go:356] "Generic (PLEG): container finished" podID="57e78de5-8964-4e21-aa25-d6079c0bca5e" containerID="e1c9ac34bc42d81f7db7cbefefd04dd30c3e39d1a22536e4e0d4c2794043c69e" exitCode=0
Aug 05 17:53:18.101843 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:18.101743    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn" event={"ID":"57e78de5-8964-4e21-aa25-d6079c0bca5e","Type":"ContainerDied","Data":"e1c9ac34bc42d81f7db7cbefefd04dd30c3e39d1a22536e4e0d4c2794043c69e"}
Aug 05 17:53:18.102155 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:18.102136    2550 scope.go:117] "RemoveContainer" containerID="e1c9ac34bc42d81f7db7cbefefd04dd30c3e39d1a22536e4e0d4c2794043c69e"
Aug 05 17:53:18.103142 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:18.103116    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 17:53:18.103212 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:18.103180    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 17:53:18.103267 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:18.103217    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 17:53:19.107805 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:19.107769    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn" event={"ID":"57e78de5-8964-4e21-aa25-d6079c0bca5e","Type":"ContainerStarted","Data":"c7de1908cb5c329637f49c99b2a8033c6490506febf89286a45abec891583a6a"}
Aug 05 17:53:19.244589 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:19.244549    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:19.244783 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:19.244761    2550 patch_prober.go:28] interesting pod/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Aug 05 17:53:19.244831 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:19.244815    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn" podUID="57e78de5-8964-4e21-aa25-d6079c0bca5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Aug 05 17:53:20.245504 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:20.245466    2550 patch_prober.go:28] interesting pod/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Aug 05 17:53:20.245924 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:20.245528    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn" podUID="57e78de5-8964-4e21-aa25-d6079c0bca5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Aug 05 17:53:21.245056 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:21.245019    2550 patch_prober.go:28] interesting pod/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Aug 05 17:53:21.245242 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:21.245107    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn" podUID="57e78de5-8964-4e21-aa25-d6079c0bca5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Aug 05 17:53:22.245244 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:22.245210    2550 patch_prober.go:28] interesting pod/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Aug 05 17:53:22.245638 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:22.245287    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn" podUID="57e78de5-8964-4e21-aa25-d6079c0bca5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Aug 05 17:53:23.244879 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:23.244843    2550 patch_prober.go:28] interesting pod/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Aug 05 17:53:23.245102 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:23.244911    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn" podUID="57e78de5-8964-4e21-aa25-d6079c0bca5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Aug 05 17:53:24.248191 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:24.248162    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:24.248567 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:24.248547    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:24.249289 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:24.249272    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:35.220042 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:35.220005    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm"]
Aug 05 17:53:35.220514 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:35.220254    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm" podUID="dcd77a0b-8be1-4b39-9fef-b290e13b0a61" containerName="maas-api" containerID="cri-o://338fae5a163d74d8d296882945411e633e18f29fd8acc9c32de84cb40bbdb9f7" gracePeriod=30
Aug 05 17:53:35.471896 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:35.471836    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm"
Aug 05 17:53:35.558250 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:35.558219    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zst4z\" (UniqueName: \"kubernetes.io/projected/dcd77a0b-8be1-4b39-9fef-b290e13b0a61-kube-api-access-zst4z\") pod \"dcd77a0b-8be1-4b39-9fef-b290e13b0a61\" (UID: \"dcd77a0b-8be1-4b39-9fef-b290e13b0a61\") "
Aug 05 17:53:35.558432 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:35.558275    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcd77a0b-8be1-4b39-9fef-b290e13b0a61-maas-api-tls\") pod \"dcd77a0b-8be1-4b39-9fef-b290e13b0a61\" (UID: \"dcd77a0b-8be1-4b39-9fef-b290e13b0a61\") "
Aug 05 17:53:35.558432 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:35.558328    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcd77a0b-8be1-4b39-9fef-b290e13b0a61-openshift-service-ca\") pod \"dcd77a0b-8be1-4b39-9fef-b290e13b0a61\" (UID: \"dcd77a0b-8be1-4b39-9fef-b290e13b0a61\") "
Aug 05 17:53:35.558767 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:35.558719    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcd77a0b-8be1-4b39-9fef-b290e13b0a61-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dcd77a0b-8be1-4b39-9fef-b290e13b0a61" (UID: "dcd77a0b-8be1-4b39-9fef-b290e13b0a61"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:53:35.560521 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:35.560490    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcd77a0b-8be1-4b39-9fef-b290e13b0a61-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dcd77a0b-8be1-4b39-9fef-b290e13b0a61" (UID: "dcd77a0b-8be1-4b39-9fef-b290e13b0a61"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:53:35.560629 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:35.560580    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcd77a0b-8be1-4b39-9fef-b290e13b0a61-kube-api-access-zst4z" (OuterVolumeSpecName: "kube-api-access-zst4z") pod "dcd77a0b-8be1-4b39-9fef-b290e13b0a61" (UID: "dcd77a0b-8be1-4b39-9fef-b290e13b0a61"). InnerVolumeSpecName "kube-api-access-zst4z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:53:35.631152 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:35.631116    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"]
Aug 05 17:53:35.631345 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:35.631321    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn" podUID="57e78de5-8964-4e21-aa25-d6079c0bca5e" containerName="istio-proxy" containerID="cri-o://c7de1908cb5c329637f49c99b2a8033c6490506febf89286a45abec891583a6a" gracePeriod=30
Aug 05 17:53:35.659406 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:35.659378    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zst4z\" (UniqueName: \"kubernetes.io/projected/dcd77a0b-8be1-4b39-9fef-b290e13b0a61-kube-api-access-zst4z\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:53:35.659406 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:35.659404    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcd77a0b-8be1-4b39-9fef-b290e13b0a61-maas-api-tls\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:53:35.659553 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:35.659413    2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcd77a0b-8be1-4b39-9fef-b290e13b0a61-openshift-service-ca\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:53:36.196469 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:36.196434    2550 generic.go:356] "Generic (PLEG): container finished" podID="dcd77a0b-8be1-4b39-9fef-b290e13b0a61" containerID="338fae5a163d74d8d296882945411e633e18f29fd8acc9c32de84cb40bbdb9f7" exitCode=0
Aug 05 17:53:36.196636 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:36.196475    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm"
Aug 05 17:53:36.196636 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:36.196523    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm" event={"ID":"dcd77a0b-8be1-4b39-9fef-b290e13b0a61","Type":"ContainerDied","Data":"338fae5a163d74d8d296882945411e633e18f29fd8acc9c32de84cb40bbdb9f7"}
Aug 05 17:53:36.196636 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:36.196572    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm" event={"ID":"dcd77a0b-8be1-4b39-9fef-b290e13b0a61","Type":"ContainerDied","Data":"4798ad86c5709b10d476bdea9d8614aeba77e5ab9715cf646571fe4e44091efb"}
Aug 05 17:53:36.196636 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:36.196592    2550 scope.go:117] "RemoveContainer" containerID="338fae5a163d74d8d296882945411e633e18f29fd8acc9c32de84cb40bbdb9f7"
Aug 05 17:53:36.204759 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:36.204741    2550 scope.go:117] "RemoveContainer" containerID="338fae5a163d74d8d296882945411e633e18f29fd8acc9c32de84cb40bbdb9f7"
Aug 05 17:53:36.205003 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:53:36.204985    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"338fae5a163d74d8d296882945411e633e18f29fd8acc9c32de84cb40bbdb9f7\": container with ID starting with 338fae5a163d74d8d296882945411e633e18f29fd8acc9c32de84cb40bbdb9f7 not found: ID does not exist" containerID="338fae5a163d74d8d296882945411e633e18f29fd8acc9c32de84cb40bbdb9f7"
Aug 05 17:53:36.205084 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:36.205011    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"338fae5a163d74d8d296882945411e633e18f29fd8acc9c32de84cb40bbdb9f7"} err="failed to get container status \"338fae5a163d74d8d296882945411e633e18f29fd8acc9c32de84cb40bbdb9f7\": rpc error: code = NotFound desc = could not find container \"338fae5a163d74d8d296882945411e633e18f29fd8acc9c32de84cb40bbdb9f7\": container with ID starting with 338fae5a163d74d8d296882945411e633e18f29fd8acc9c32de84cb40bbdb9f7 not found: ID does not exist"
Aug 05 17:53:36.219993 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:36.219969    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm"]
Aug 05 17:53:36.225679 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:36.225658    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b570cb7c-54fbf86989-hhmnm"]
Aug 05 17:53:37.701835 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:37.701803    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcd77a0b-8be1-4b39-9fef-b290e13b0a61" path="/var/lib/kubelet/pods/dcd77a0b-8be1-4b39-9fef-b290e13b0a61/volumes"
Aug 05 17:53:39.250973 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:39.250932    2550 patch_prober.go:28] interesting pod/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Aug 05 17:53:39.251404 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:39.250997    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn" podUID="57e78de5-8964-4e21-aa25-d6079c0bca5e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 05 17:53:40.871663 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:40.871636    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:41.008766 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.008737    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-workload-socket\") pod \"57e78de5-8964-4e21-aa25-d6079c0bca5e\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") "
Aug 05 17:53:41.008946 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.008805    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57e78de5-8964-4e21-aa25-d6079c0bca5e-istiod-ca-cert\") pod \"57e78de5-8964-4e21-aa25-d6079c0bca5e\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") "
Aug 05 17:53:41.008946 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.008834    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57e78de5-8964-4e21-aa25-d6079c0bca5e-istio-token\") pod \"57e78de5-8964-4e21-aa25-d6079c0bca5e\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") "
Aug 05 17:53:41.008946 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.008869    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-credential-socket\") pod \"57e78de5-8964-4e21-aa25-d6079c0bca5e\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") "
Aug 05 17:53:41.008946 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.008922    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57e78de5-8964-4e21-aa25-d6079c0bca5e-istio-podinfo\") pod \"57e78de5-8964-4e21-aa25-d6079c0bca5e\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") "
Aug 05 17:53:41.009206 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.008987    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-istio-envoy\") pod \"57e78de5-8964-4e21-aa25-d6079c0bca5e\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") "
Aug 05 17:53:41.009206 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.009005    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "57e78de5-8964-4e21-aa25-d6079c0bca5e" (UID: "57e78de5-8964-4e21-aa25-d6079c0bca5e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:53:41.009206 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.009034    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xqv8\" (UniqueName: \"kubernetes.io/projected/57e78de5-8964-4e21-aa25-d6079c0bca5e-kube-api-access-8xqv8\") pod \"57e78de5-8964-4e21-aa25-d6079c0bca5e\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") "
Aug 05 17:53:41.009206 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.009071    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-workload-certs\") pod \"57e78de5-8964-4e21-aa25-d6079c0bca5e\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") "
Aug 05 17:53:41.009206 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.009114    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-istio-data\") pod \"57e78de5-8964-4e21-aa25-d6079c0bca5e\" (UID: \"57e78de5-8964-4e21-aa25-d6079c0bca5e\") "
Aug 05 17:53:41.009464 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.009213    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57e78de5-8964-4e21-aa25-d6079c0bca5e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "57e78de5-8964-4e21-aa25-d6079c0bca5e" (UID: "57e78de5-8964-4e21-aa25-d6079c0bca5e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:53:41.009464 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.009231    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "57e78de5-8964-4e21-aa25-d6079c0bca5e" (UID: "57e78de5-8964-4e21-aa25-d6079c0bca5e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:53:41.009464 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.009452    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "57e78de5-8964-4e21-aa25-d6079c0bca5e" (UID: "57e78de5-8964-4e21-aa25-d6079c0bca5e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:53:41.009626 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.009473    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-istio-data" (OuterVolumeSpecName: "istio-data") pod "57e78de5-8964-4e21-aa25-d6079c0bca5e" (UID: "57e78de5-8964-4e21-aa25-d6079c0bca5e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:53:41.009626 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.009527    2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-workload-socket\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:53:41.009626 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.009551    2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57e78de5-8964-4e21-aa25-d6079c0bca5e-istiod-ca-cert\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:53:41.009626 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.009567    2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-credential-socket\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:53:41.010985 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.010962    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57e78de5-8964-4e21-aa25-d6079c0bca5e-istio-token" (OuterVolumeSpecName: "istio-token") pod "57e78de5-8964-4e21-aa25-d6079c0bca5e" (UID: "57e78de5-8964-4e21-aa25-d6079c0bca5e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:53:41.011093 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.011075    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/57e78de5-8964-4e21-aa25-d6079c0bca5e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "57e78de5-8964-4e21-aa25-d6079c0bca5e" (UID: "57e78de5-8964-4e21-aa25-d6079c0bca5e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 05 17:53:41.011195 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.011176    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57e78de5-8964-4e21-aa25-d6079c0bca5e-kube-api-access-8xqv8" (OuterVolumeSpecName: "kube-api-access-8xqv8") pod "57e78de5-8964-4e21-aa25-d6079c0bca5e" (UID: "57e78de5-8964-4e21-aa25-d6079c0bca5e"). InnerVolumeSpecName "kube-api-access-8xqv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:53:41.011530 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.011509    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "57e78de5-8964-4e21-aa25-d6079c0bca5e" (UID: "57e78de5-8964-4e21-aa25-d6079c0bca5e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:53:41.110755 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.110734    2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-workload-certs\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:53:41.110755 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.110754    2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-istio-data\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:53:41.110896 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.110769    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57e78de5-8964-4e21-aa25-d6079c0bca5e-istio-token\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:53:41.110896 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.110778    2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57e78de5-8964-4e21-aa25-d6079c0bca5e-istio-podinfo\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:53:41.110896 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.110786    2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57e78de5-8964-4e21-aa25-d6079c0bca5e-istio-envoy\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:53:41.110896 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.110794    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xqv8\" (UniqueName: \"kubernetes.io/projected/57e78de5-8964-4e21-aa25-d6079c0bca5e-kube-api-access-8xqv8\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:53:41.223355 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.223328    2550 generic.go:356] "Generic (PLEG): container finished" podID="57e78de5-8964-4e21-aa25-d6079c0bca5e" containerID="c7de1908cb5c329637f49c99b2a8033c6490506febf89286a45abec891583a6a" exitCode=0
Aug 05 17:53:41.223472 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.223373    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"
Aug 05 17:53:41.223472 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.223416    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn" event={"ID":"57e78de5-8964-4e21-aa25-d6079c0bca5e","Type":"ContainerDied","Data":"c7de1908cb5c329637f49c99b2a8033c6490506febf89286a45abec891583a6a"}
Aug 05 17:53:41.223607 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.223479    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn" event={"ID":"57e78de5-8964-4e21-aa25-d6079c0bca5e","Type":"ContainerDied","Data":"df9171ab2372ec7606c5e7dfbd86c9dee20582ff8a1b70114298909593793296"}
Aug 05 17:53:41.223607 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.223512    2550 scope.go:117] "RemoveContainer" containerID="c7de1908cb5c329637f49c99b2a8033c6490506febf89286a45abec891583a6a"
Aug 05 17:53:41.231993 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.231980    2550 scope.go:117] "RemoveContainer" containerID="e1c9ac34bc42d81f7db7cbefefd04dd30c3e39d1a22536e4e0d4c2794043c69e"
Aug 05 17:53:41.239370 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.239352    2550 scope.go:117] "RemoveContainer" containerID="c7de1908cb5c329637f49c99b2a8033c6490506febf89286a45abec891583a6a"
Aug 05 17:53:41.239616 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:53:41.239600    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7de1908cb5c329637f49c99b2a8033c6490506febf89286a45abec891583a6a\": container with ID starting with c7de1908cb5c329637f49c99b2a8033c6490506febf89286a45abec891583a6a not found: ID does not exist" containerID="c7de1908cb5c329637f49c99b2a8033c6490506febf89286a45abec891583a6a"
Aug 05 17:53:41.239673 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.239624    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7de1908cb5c329637f49c99b2a8033c6490506febf89286a45abec891583a6a"} err="failed to get container status \"c7de1908cb5c329637f49c99b2a8033c6490506febf89286a45abec891583a6a\": rpc error: code = NotFound desc = could not find container \"c7de1908cb5c329637f49c99b2a8033c6490506febf89286a45abec891583a6a\": container with ID starting with c7de1908cb5c329637f49c99b2a8033c6490506febf89286a45abec891583a6a not found: ID does not exist"
Aug 05 17:53:41.239673 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.239641    2550 scope.go:117] "RemoveContainer" containerID="e1c9ac34bc42d81f7db7cbefefd04dd30c3e39d1a22536e4e0d4c2794043c69e"
Aug 05 17:53:41.239883 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:53:41.239865    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1c9ac34bc42d81f7db7cbefefd04dd30c3e39d1a22536e4e0d4c2794043c69e\": container with ID starting with e1c9ac34bc42d81f7db7cbefefd04dd30c3e39d1a22536e4e0d4c2794043c69e not found: ID does not exist" containerID="e1c9ac34bc42d81f7db7cbefefd04dd30c3e39d1a22536e4e0d4c2794043c69e"
Aug 05 17:53:41.239922 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.239887    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1c9ac34bc42d81f7db7cbefefd04dd30c3e39d1a22536e4e0d4c2794043c69e"} err="failed to get container status \"e1c9ac34bc42d81f7db7cbefefd04dd30c3e39d1a22536e4e0d4c2794043c69e\": rpc error: code = NotFound desc = could not find container \"e1c9ac34bc42d81f7db7cbefefd04dd30c3e39d1a22536e4e0d4c2794043c69e\": container with ID starting with e1c9ac34bc42d81f7db7cbefefd04dd30c3e39d1a22536e4e0d4c2794043c69e not found: ID does not exist"
Aug 05 17:53:41.246112 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.246094    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"]
Aug 05 17:53:41.250636 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.250617    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-b570cb7c-openshift-default-79c8d5fb66-bnbtn"]
Aug 05 17:53:41.700511 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:41.700482    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57e78de5-8964-4e21-aa25-d6079c0bca5e" path="/var/lib/kubelet/pods/57e78de5-8964-4e21-aa25-d6079c0bca5e/volumes"
Aug 05 17:53:43.700917 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:43.700889    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg"]
Aug 05 17:53:43.701291 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:43.701198    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcd77a0b-8be1-4b39-9fef-b290e13b0a61" containerName="maas-api"
Aug 05 17:53:43.701291 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:43.701208    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcd77a0b-8be1-4b39-9fef-b290e13b0a61" containerName="maas-api"
Aug 05 17:53:43.701291 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:43.701224    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57e78de5-8964-4e21-aa25-d6079c0bca5e" containerName="istio-proxy"
Aug 05 17:53:43.701291 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:43.701230    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="57e78de5-8964-4e21-aa25-d6079c0bca5e" containerName="istio-proxy"
Aug 05 17:53:43.701291 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:43.701240    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57e78de5-8964-4e21-aa25-d6079c0bca5e" containerName="istio-proxy"
Aug 05 17:53:43.701291 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:43.701246    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="57e78de5-8964-4e21-aa25-d6079c0bca5e" containerName="istio-proxy"
Aug 05 17:53:43.701533 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:43.701316    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="57e78de5-8964-4e21-aa25-d6079c0bca5e" containerName="istio-proxy"
Aug 05 17:53:43.701533 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:43.701324    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="57e78de5-8964-4e21-aa25-d6079c0bca5e" containerName="istio-proxy"
Aug 05 17:53:43.701533 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:43.701332    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcd77a0b-8be1-4b39-9fef-b290e13b0a61" containerName="maas-api"
Aug 05 17:53:43.705644 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:43.705627    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg"
Aug 05 17:53:43.708050 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:43.708032    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-d993b9ed-serving-cert\""
Aug 05 17:53:43.715859 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:43.715834    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg"]
Aug 05 17:53:43.733604 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:43.733573    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/985dedee-e4f2-44c7-a959-434eb1ea0d90-openshift-service-ca\") pod \"maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg\" (UID: \"985dedee-e4f2-44c7-a959-434eb1ea0d90\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg"
Aug 05 17:53:43.733849 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:43.733827    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z9fm\" (UniqueName: \"kubernetes.io/projected/985dedee-e4f2-44c7-a959-434eb1ea0d90-kube-api-access-6z9fm\") pod \"maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg\" (UID: \"985dedee-e4f2-44c7-a959-434eb1ea0d90\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg"
Aug 05 17:53:43.733947 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:43.733868    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/985dedee-e4f2-44c7-a959-434eb1ea0d90-maas-api-tls\") pod \"maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg\" (UID: \"985dedee-e4f2-44c7-a959-434eb1ea0d90\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg"
Aug 05 17:53:43.835028 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:43.834990    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/985dedee-e4f2-44c7-a959-434eb1ea0d90-openshift-service-ca\") pod \"maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg\" (UID: \"985dedee-e4f2-44c7-a959-434eb1ea0d90\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg"
Aug 05 17:53:43.835215 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:43.835055    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z9fm\" (UniqueName: \"kubernetes.io/projected/985dedee-e4f2-44c7-a959-434eb1ea0d90-kube-api-access-6z9fm\") pod \"maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg\" (UID: \"985dedee-e4f2-44c7-a959-434eb1ea0d90\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg"
Aug 05 17:53:43.835215 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:43.835077    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/985dedee-e4f2-44c7-a959-434eb1ea0d90-maas-api-tls\") pod \"maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg\" (UID: \"985dedee-e4f2-44c7-a959-434eb1ea0d90\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg"
Aug 05 17:53:43.835215 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:53:43.835204    2550 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-serving-cert: secret "maas-api-e2e-migrate-d993b9ed-serving-cert" not found
Aug 05 17:53:43.835352 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:53:43.835269    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/985dedee-e4f2-44c7-a959-434eb1ea0d90-maas-api-tls podName:985dedee-e4f2-44c7-a959-434eb1ea0d90 nodeName:}" failed. No retries permitted until 2026-08-05 17:53:44.335252026 +0000 UTC m=+1931.316829433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/985dedee-e4f2-44c7-a959-434eb1ea0d90-maas-api-tls") pod "maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg" (UID: "985dedee-e4f2-44c7-a959-434eb1ea0d90") : secret "maas-api-e2e-migrate-d993b9ed-serving-cert" not found
Aug 05 17:53:43.835666 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:43.835645    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/985dedee-e4f2-44c7-a959-434eb1ea0d90-openshift-service-ca\") pod \"maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg\" (UID: \"985dedee-e4f2-44c7-a959-434eb1ea0d90\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg"
Aug 05 17:53:43.845563 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:43.845532    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z9fm\" (UniqueName: \"kubernetes.io/projected/985dedee-e4f2-44c7-a959-434eb1ea0d90-kube-api-access-6z9fm\") pod \"maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg\" (UID: \"985dedee-e4f2-44c7-a959-434eb1ea0d90\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg"
Aug 05 17:53:44.338411 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:44.338372    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/985dedee-e4f2-44c7-a959-434eb1ea0d90-maas-api-tls\") pod \"maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg\" (UID: \"985dedee-e4f2-44c7-a959-434eb1ea0d90\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg"
Aug 05 17:53:44.340967 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:44.340944    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/985dedee-e4f2-44c7-a959-434eb1ea0d90-maas-api-tls\") pod \"maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg\" (UID: \"985dedee-e4f2-44c7-a959-434eb1ea0d90\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg"
Aug 05 17:53:44.419530 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:44.419497    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9"]
Aug 05 17:53:44.423129 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:44.423108    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9"
Aug 05 17:53:44.430096 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:44.430071    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-d993b9ed\""
Aug 05 17:53:44.430480 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:44.430447    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-d993b9ed-dockercfg-z8s45\""
Aug 05 17:53:44.434017 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:44.433992    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9"]
Aug 05 17:53:44.439319 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:44.439297    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b249d475-77ea-4b70-adbf-b28ea976649f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9\" (UID: \"b249d475-77ea-4b70-adbf-b28ea976649f\") " pod="openshift-ingress/payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9"
Aug 05 17:53:44.439415 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:44.439360    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg9c4\" (UniqueName: \"kubernetes.io/projected/b249d475-77ea-4b70-adbf-b28ea976649f-kube-api-access-qg9c4\") pod \"payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9\" (UID: \"b249d475-77ea-4b70-adbf-b28ea976649f\") " pod="openshift-ingress/payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9"
Aug 05 17:53:44.540105 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:44.540058    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qg9c4\" (UniqueName: \"kubernetes.io/projected/b249d475-77ea-4b70-adbf-b28ea976649f-kube-api-access-qg9c4\") pod \"payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9\" (UID: \"b249d475-77ea-4b70-adbf-b28ea976649f\") " pod="openshift-ingress/payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9"
Aug 05 17:53:44.540294 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:44.540175    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b249d475-77ea-4b70-adbf-b28ea976649f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9\" (UID: \"b249d475-77ea-4b70-adbf-b28ea976649f\") " pod="openshift-ingress/payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9"
Aug 05 17:53:44.540916 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:44.540894    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b249d475-77ea-4b70-adbf-b28ea976649f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9\" (UID: \"b249d475-77ea-4b70-adbf-b28ea976649f\") " pod="openshift-ingress/payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9"
Aug 05 17:53:44.550198 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:44.550174    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg9c4\" (UniqueName: \"kubernetes.io/projected/b249d475-77ea-4b70-adbf-b28ea976649f-kube-api-access-qg9c4\") pod \"payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9\" (UID: \"b249d475-77ea-4b70-adbf-b28ea976649f\") " pod="openshift-ingress/payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9"
Aug 05 17:53:44.616148 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:44.616064    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg"
Aug 05 17:53:44.735145 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:44.735118    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9"
Aug 05 17:53:44.758287 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:44.758257    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg"]
Aug 05 17:53:44.759962 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:53:44.759933    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod985dedee_e4f2_44c7_a959_434eb1ea0d90.slice/crio-24c1a67740073cf701c125dc7efe9fe112c0be2cc6ae9cf95d6ddaa4ee7741d4 WatchSource:0}: Error finding container 24c1a67740073cf701c125dc7efe9fe112c0be2cc6ae9cf95d6ddaa4ee7741d4: Status 404 returned error can't find the container with id 24c1a67740073cf701c125dc7efe9fe112c0be2cc6ae9cf95d6ddaa4ee7741d4
Aug 05 17:53:44.883142 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:44.883118    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9"]
Aug 05 17:53:44.884523 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:53:44.884494    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb249d475_77ea_4b70_adbf_b28ea976649f.slice/crio-bf858ada7b39a97c1eda1bb5a67ac7ac0d316216c10ff0138df747458e9f943e WatchSource:0}: Error finding container bf858ada7b39a97c1eda1bb5a67ac7ac0d316216c10ff0138df747458e9f943e: Status 404 returned error can't find the container with id bf858ada7b39a97c1eda1bb5a67ac7ac0d316216c10ff0138df747458e9f943e
Aug 05 17:53:45.244711 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:45.244674    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg" event={"ID":"985dedee-e4f2-44c7-a959-434eb1ea0d90","Type":"ContainerStarted","Data":"f3b888578e82d579f1b1740439685dcc584650ce1fde17023697ca4305237892"}
Aug 05 17:53:45.244917 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:45.244894    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg"
Aug 05 17:53:45.244992 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:45.244940    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg" event={"ID":"985dedee-e4f2-44c7-a959-434eb1ea0d90","Type":"ContainerStarted","Data":"24c1a67740073cf701c125dc7efe9fe112c0be2cc6ae9cf95d6ddaa4ee7741d4"}
Aug 05 17:53:45.246223 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:45.246199    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9" event={"ID":"b249d475-77ea-4b70-adbf-b28ea976649f","Type":"ContainerStarted","Data":"68ddb81ccff9bbcb025844e8c18607f2e4992a4230250ab5e3a50d3079a4284d"}
Aug 05 17:53:45.246362 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:45.246235    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9" event={"ID":"b249d475-77ea-4b70-adbf-b28ea976649f","Type":"ContainerStarted","Data":"bf858ada7b39a97c1eda1bb5a67ac7ac0d316216c10ff0138df747458e9f943e"}
Aug 05 17:53:45.246362 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:45.246320    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9"
Aug 05 17:53:45.266074 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:45.266029    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg" podStartSLOduration=1.914680398 podStartE2EDuration="2.266016006s" totalImagesPullingTime="351.335608ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:53:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:53:45.265833642 +0000 UTC m=+1932.247411071" watchObservedRunningTime="2026-08-05 17:53:45.266016006 +0000 UTC m=+1932.247593434"
Aug 05 17:53:45.286741 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:45.286684    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9" podStartSLOduration=1.286671022 podStartE2EDuration="1.286671022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:53:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:53:45.283765162 +0000 UTC m=+1932.265342591" watchObservedRunningTime="2026-08-05 17:53:45.286671022 +0000 UTC m=+1932.268248450"
Aug 05 17:53:51.256467 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:51.256435    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg"
Aug 05 17:53:56.253317 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:53:56.253282    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9"
Aug 05 17:54:14.958837 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:14.958798    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg"]
Aug 05 17:54:14.959389 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:14.959079    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg" podUID="985dedee-e4f2-44c7-a959-434eb1ea0d90" containerName="maas-api" containerID="cri-o://f3b888578e82d579f1b1740439685dcc584650ce1fde17023697ca4305237892" gracePeriod=30
Aug 05 17:54:15.018328 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.018291    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9"]
Aug 05 17:54:15.018605 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.018573    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9" podUID="b249d475-77ea-4b70-adbf-b28ea976649f" containerName="payload-processing" containerID="cri-o://68ddb81ccff9bbcb025844e8c18607f2e4992a4230250ab5e3a50d3079a4284d" gracePeriod=30
Aug 05 17:54:15.303132 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.303104    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg"
Aug 05 17:54:15.306261 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.306241    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9"
Aug 05 17:54:15.397084 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.397051    2550 generic.go:356] "Generic (PLEG): container finished" podID="985dedee-e4f2-44c7-a959-434eb1ea0d90" containerID="f3b888578e82d579f1b1740439685dcc584650ce1fde17023697ca4305237892" exitCode=0
Aug 05 17:54:15.397297 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.397094    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg"
Aug 05 17:54:15.397297 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.397133    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg" event={"ID":"985dedee-e4f2-44c7-a959-434eb1ea0d90","Type":"ContainerDied","Data":"f3b888578e82d579f1b1740439685dcc584650ce1fde17023697ca4305237892"}
Aug 05 17:54:15.397297 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.397219    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg" event={"ID":"985dedee-e4f2-44c7-a959-434eb1ea0d90","Type":"ContainerDied","Data":"24c1a67740073cf701c125dc7efe9fe112c0be2cc6ae9cf95d6ddaa4ee7741d4"}
Aug 05 17:54:15.397297 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.397249    2550 scope.go:117] "RemoveContainer" containerID="f3b888578e82d579f1b1740439685dcc584650ce1fde17023697ca4305237892"
Aug 05 17:54:15.398790 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.398757    2550 generic.go:356] "Generic (PLEG): container finished" podID="b249d475-77ea-4b70-adbf-b28ea976649f" containerID="68ddb81ccff9bbcb025844e8c18607f2e4992a4230250ab5e3a50d3079a4284d" exitCode=0
Aug 05 17:54:15.398899 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.398824    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9"
Aug 05 17:54:15.398899 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.398838    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9" event={"ID":"b249d475-77ea-4b70-adbf-b28ea976649f","Type":"ContainerDied","Data":"68ddb81ccff9bbcb025844e8c18607f2e4992a4230250ab5e3a50d3079a4284d"}
Aug 05 17:54:15.398899 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.398894    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9" event={"ID":"b249d475-77ea-4b70-adbf-b28ea976649f","Type":"ContainerDied","Data":"bf858ada7b39a97c1eda1bb5a67ac7ac0d316216c10ff0138df747458e9f943e"}
Aug 05 17:54:15.407801 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.407781    2550 scope.go:117] "RemoveContainer" containerID="f3b888578e82d579f1b1740439685dcc584650ce1fde17023697ca4305237892"
Aug 05 17:54:15.408107 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:54:15.408077    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3b888578e82d579f1b1740439685dcc584650ce1fde17023697ca4305237892\": container with ID starting with f3b888578e82d579f1b1740439685dcc584650ce1fde17023697ca4305237892 not found: ID does not exist" containerID="f3b888578e82d579f1b1740439685dcc584650ce1fde17023697ca4305237892"
Aug 05 17:54:15.408204 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.408111    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3b888578e82d579f1b1740439685dcc584650ce1fde17023697ca4305237892"} err="failed to get container status \"f3b888578e82d579f1b1740439685dcc584650ce1fde17023697ca4305237892\": rpc error: code = NotFound desc = could not find container \"f3b888578e82d579f1b1740439685dcc584650ce1fde17023697ca4305237892\": container with ID starting with f3b888578e82d579f1b1740439685dcc584650ce1fde17023697ca4305237892 not found: ID does not exist"
Aug 05 17:54:15.408204 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.408126    2550 scope.go:117] "RemoveContainer" containerID="68ddb81ccff9bbcb025844e8c18607f2e4992a4230250ab5e3a50d3079a4284d"
Aug 05 17:54:15.417819 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.417799    2550 scope.go:117] "RemoveContainer" containerID="68ddb81ccff9bbcb025844e8c18607f2e4992a4230250ab5e3a50d3079a4284d"
Aug 05 17:54:15.418098 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:54:15.418081    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68ddb81ccff9bbcb025844e8c18607f2e4992a4230250ab5e3a50d3079a4284d\": container with ID starting with 68ddb81ccff9bbcb025844e8c18607f2e4992a4230250ab5e3a50d3079a4284d not found: ID does not exist" containerID="68ddb81ccff9bbcb025844e8c18607f2e4992a4230250ab5e3a50d3079a4284d"
Aug 05 17:54:15.418220 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.418107    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68ddb81ccff9bbcb025844e8c18607f2e4992a4230250ab5e3a50d3079a4284d"} err="failed to get container status \"68ddb81ccff9bbcb025844e8c18607f2e4992a4230250ab5e3a50d3079a4284d\": rpc error: code = NotFound desc = could not find container \"68ddb81ccff9bbcb025844e8c18607f2e4992a4230250ab5e3a50d3079a4284d\": container with ID starting with 68ddb81ccff9bbcb025844e8c18607f2e4992a4230250ab5e3a50d3079a4284d not found: ID does not exist"
Aug 05 17:54:15.430445 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.430419    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6z9fm\" (UniqueName: \"kubernetes.io/projected/985dedee-e4f2-44c7-a959-434eb1ea0d90-kube-api-access-6z9fm\") pod \"985dedee-e4f2-44c7-a959-434eb1ea0d90\" (UID: \"985dedee-e4f2-44c7-a959-434eb1ea0d90\") "
Aug 05 17:54:15.430528 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.430456    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/985dedee-e4f2-44c7-a959-434eb1ea0d90-openshift-service-ca\") pod \"985dedee-e4f2-44c7-a959-434eb1ea0d90\" (UID: \"985dedee-e4f2-44c7-a959-434eb1ea0d90\") "
Aug 05 17:54:15.430528 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.430480    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/985dedee-e4f2-44c7-a959-434eb1ea0d90-maas-api-tls\") pod \"985dedee-e4f2-44c7-a959-434eb1ea0d90\" (UID: \"985dedee-e4f2-44c7-a959-434eb1ea0d90\") "
Aug 05 17:54:15.430528 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.430497    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b249d475-77ea-4b70-adbf-b28ea976649f-plugins-config-volume\") pod \"b249d475-77ea-4b70-adbf-b28ea976649f\" (UID: \"b249d475-77ea-4b70-adbf-b28ea976649f\") "
Aug 05 17:54:15.430644 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.430575    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qg9c4\" (UniqueName: \"kubernetes.io/projected/b249d475-77ea-4b70-adbf-b28ea976649f-kube-api-access-qg9c4\") pod \"b249d475-77ea-4b70-adbf-b28ea976649f\" (UID: \"b249d475-77ea-4b70-adbf-b28ea976649f\") "
Aug 05 17:54:15.430869 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.430841    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/985dedee-e4f2-44c7-a959-434eb1ea0d90-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "985dedee-e4f2-44c7-a959-434eb1ea0d90" (UID: "985dedee-e4f2-44c7-a959-434eb1ea0d90"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:54:15.430960 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.430934    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b249d475-77ea-4b70-adbf-b28ea976649f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b249d475-77ea-4b70-adbf-b28ea976649f" (UID: "b249d475-77ea-4b70-adbf-b28ea976649f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:54:15.432634 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.432609    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/985dedee-e4f2-44c7-a959-434eb1ea0d90-kube-api-access-6z9fm" (OuterVolumeSpecName: "kube-api-access-6z9fm") pod "985dedee-e4f2-44c7-a959-434eb1ea0d90" (UID: "985dedee-e4f2-44c7-a959-434eb1ea0d90"). InnerVolumeSpecName "kube-api-access-6z9fm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:54:15.432721 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.432639    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/985dedee-e4f2-44c7-a959-434eb1ea0d90-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "985dedee-e4f2-44c7-a959-434eb1ea0d90" (UID: "985dedee-e4f2-44c7-a959-434eb1ea0d90"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:54:15.432721 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.432632    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b249d475-77ea-4b70-adbf-b28ea976649f-kube-api-access-qg9c4" (OuterVolumeSpecName: "kube-api-access-qg9c4") pod "b249d475-77ea-4b70-adbf-b28ea976649f" (UID: "b249d475-77ea-4b70-adbf-b28ea976649f"). InnerVolumeSpecName "kube-api-access-qg9c4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:54:15.532021 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.531936    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qg9c4\" (UniqueName: \"kubernetes.io/projected/b249d475-77ea-4b70-adbf-b28ea976649f-kube-api-access-qg9c4\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:54:15.532021 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.531968    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6z9fm\" (UniqueName: \"kubernetes.io/projected/985dedee-e4f2-44c7-a959-434eb1ea0d90-kube-api-access-6z9fm\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:54:15.532021 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.531979    2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/985dedee-e4f2-44c7-a959-434eb1ea0d90-openshift-service-ca\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:54:15.532021 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.531990    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/985dedee-e4f2-44c7-a959-434eb1ea0d90-maas-api-tls\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:54:15.532021 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.531998    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b249d475-77ea-4b70-adbf-b28ea976649f-plugins-config-volume\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:54:15.724588 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.724557    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg"]
Aug 05 17:54:15.731653 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.731624    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d993b9ed-95754b676-6qvxg"]
Aug 05 17:54:15.742518 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.742494    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9"]
Aug 05 17:54:15.746168 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:15.746148    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d993b9ed-b76959db5-xxgm9"]
Aug 05 17:54:16.027336 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.027301    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"]
Aug 05 17:54:16.027797 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.027644    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b249d475-77ea-4b70-adbf-b28ea976649f" containerName="payload-processing"
Aug 05 17:54:16.027797 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.027654    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="b249d475-77ea-4b70-adbf-b28ea976649f" containerName="payload-processing"
Aug 05 17:54:16.027797 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.027667    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="985dedee-e4f2-44c7-a959-434eb1ea0d90" containerName="maas-api"
Aug 05 17:54:16.027797 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.027672    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="985dedee-e4f2-44c7-a959-434eb1ea0d90" containerName="maas-api"
Aug 05 17:54:16.027797 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.027738    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="985dedee-e4f2-44c7-a959-434eb1ea0d90" containerName="maas-api"
Aug 05 17:54:16.027797 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.027749    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="b249d475-77ea-4b70-adbf-b28ea976649f" containerName="payload-processing"
Aug 05 17:54:16.032092 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.032073    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.034644 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.034624    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-9759ef96-openshift-default-dockercfg-vhnvn\""
Aug 05 17:54:16.040693 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.040669    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"]
Aug 05 17:54:16.135691 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.135656    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istio-token\") pod \"e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") " pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.135691 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.135728    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istio-data\") pod \"e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") " pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.136032 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.135796    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7vvx\" (UniqueName: \"kubernetes.io/projected/3b7f68b9-5556-47e0-80bb-ef08da8252ca-kube-api-access-f7vvx\") pod \"e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") " pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.136032 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.135826    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-workload-socket\") pod \"e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") " pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.136032 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.135856    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-credential-socket\") pod \"e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") " pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.136032 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.135878    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-workload-certs\") pod \"e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") " pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.136032 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.135924    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istio-podinfo\") pod \"e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") " pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.136032 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.135946    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istio-envoy\") pod \"e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") " pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.136032 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.135984    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istiod-ca-cert\") pod \"e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") " pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.237090 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.236996    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istio-podinfo\") pod \"e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") " pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.237090 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.237037    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istio-envoy\") pod \"e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") " pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.237090 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.237061    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istiod-ca-cert\") pod \"e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") " pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.237090 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.237081    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istio-token\") pod \"e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") " pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.237412 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.237101    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istio-data\") pod \"e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") " pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.237412 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.237144    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7vvx\" (UniqueName: \"kubernetes.io/projected/3b7f68b9-5556-47e0-80bb-ef08da8252ca-kube-api-access-f7vvx\") pod \"e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") " pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.237412 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.237163    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-workload-socket\") pod \"e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") " pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.237412 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.237190    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-credential-socket\") pod \"e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") " pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.237412 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.237224    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-workload-certs\") pod \"e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") " pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.237660 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.237553    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istio-data\") pod \"e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") " pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.237660 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.237581    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-workload-certs\") pod \"e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") " pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.237811 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.237662    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-workload-socket\") pod \"e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") " pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.237811 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.237782    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-credential-socket\") pod \"e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") " pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.238392 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.238367    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istiod-ca-cert\") pod \"e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") " pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.239590 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.239561    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istio-envoy\") pod \"e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") " pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.239685 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.239673    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istio-podinfo\") pod \"e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") " pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.245234 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.245204    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istio-token\") pod \"e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") " pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.245477 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.245460    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7vvx\" (UniqueName: \"kubernetes.io/projected/3b7f68b9-5556-47e0-80bb-ef08da8252ca-kube-api-access-f7vvx\") pod \"e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") " pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.343483 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.343440    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:16.506463 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.506439    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"]
Aug 05 17:54:16.509048 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:54:16.509021    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b7f68b9_5556_47e0_80bb_ef08da8252ca.slice/crio-a6cd8194c9d1a1549b376b62397ff4b206337d3d3758c48b6151924f01ad9a46 WatchSource:0}: Error finding container a6cd8194c9d1a1549b376b62397ff4b206337d3d3758c48b6151924f01ad9a46: Status 404 returned error can't find the container with id a6cd8194c9d1a1549b376b62397ff4b206337d3d3758c48b6151924f01ad9a46
Aug 05 17:54:16.511407 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.511378    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 17:54:16.511495 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.511439    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 17:54:16.511495 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:16.511470    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 17:54:17.414904 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:17.414874    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp" event={"ID":"3b7f68b9-5556-47e0-80bb-ef08da8252ca","Type":"ContainerStarted","Data":"280d98e501c5d5df27efefd99cb0ccb0d56d9366f1da2f4cf8a392e81417a0d9"}
Aug 05 17:54:17.415314 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:17.414926    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp" event={"ID":"3b7f68b9-5556-47e0-80bb-ef08da8252ca","Type":"ContainerStarted","Data":"a6cd8194c9d1a1549b376b62397ff4b206337d3d3758c48b6151924f01ad9a46"}
Aug 05 17:54:17.447360 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:17.447297    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp" podStartSLOduration=1.447273271 podStartE2EDuration="1.447273271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:54:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:54:17.444660208 +0000 UTC m=+1964.426237638" watchObservedRunningTime="2026-08-05 17:54:17.447273271 +0000 UTC m=+1964.428850702"
Aug 05 17:54:17.700788 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:17.700710    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="985dedee-e4f2-44c7-a959-434eb1ea0d90" path="/var/lib/kubelet/pods/985dedee-e4f2-44c7-a959-434eb1ea0d90/volumes"
Aug 05 17:54:17.701126 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:17.701112    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b249d475-77ea-4b70-adbf-b28ea976649f" path="/var/lib/kubelet/pods/b249d475-77ea-4b70-adbf-b28ea976649f/volumes"
Aug 05 17:54:18.344295 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:18.344254    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:18.349481 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:18.349448    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:18.419663 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:18.419632    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:18.420778 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:18.420758    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:19.973316 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:19.973285    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw"]
Aug 05 17:54:19.978325 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:19.978303    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw"
Aug 05 17:54:19.980826 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:19.980808    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-9759ef96-serving-cert\""
Aug 05 17:54:19.987138 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:19.987116    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw"]
Aug 05 17:54:20.073238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:20.073208    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbf4107c-fce6-4269-9f4b-7b49c0eaef70-openshift-service-ca\") pod \"maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw\" (UID: \"fbf4107c-fce6-4269-9f4b-7b49c0eaef70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw"
Aug 05 17:54:20.073238 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:20.073250    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hn4j\" (UniqueName: \"kubernetes.io/projected/fbf4107c-fce6-4269-9f4b-7b49c0eaef70-kube-api-access-2hn4j\") pod \"maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw\" (UID: \"fbf4107c-fce6-4269-9f4b-7b49c0eaef70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw"
Aug 05 17:54:20.073481 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:20.073356    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbf4107c-fce6-4269-9f4b-7b49c0eaef70-maas-api-tls\") pod \"maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw\" (UID: \"fbf4107c-fce6-4269-9f4b-7b49c0eaef70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw"
Aug 05 17:54:20.174116 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:20.174080    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbf4107c-fce6-4269-9f4b-7b49c0eaef70-maas-api-tls\") pod \"maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw\" (UID: \"fbf4107c-fce6-4269-9f4b-7b49c0eaef70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw"
Aug 05 17:54:20.174286 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:20.174134    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbf4107c-fce6-4269-9f4b-7b49c0eaef70-openshift-service-ca\") pod \"maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw\" (UID: \"fbf4107c-fce6-4269-9f4b-7b49c0eaef70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw"
Aug 05 17:54:20.174286 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:20.174170    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hn4j\" (UniqueName: \"kubernetes.io/projected/fbf4107c-fce6-4269-9f4b-7b49c0eaef70-kube-api-access-2hn4j\") pod \"maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw\" (UID: \"fbf4107c-fce6-4269-9f4b-7b49c0eaef70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw"
Aug 05 17:54:20.174286 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:54:20.174263    2550 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-serving-cert: secret "maas-api-e2e-ait-9759ef96-serving-cert" not found
Aug 05 17:54:20.174419 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:54:20.174355    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbf4107c-fce6-4269-9f4b-7b49c0eaef70-maas-api-tls podName:fbf4107c-fce6-4269-9f4b-7b49c0eaef70 nodeName:}" failed. No retries permitted until 2026-08-05 17:54:20.674338397 +0000 UTC m=+1967.655915804 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fbf4107c-fce6-4269-9f4b-7b49c0eaef70-maas-api-tls") pod "maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw" (UID: "fbf4107c-fce6-4269-9f4b-7b49c0eaef70") : secret "maas-api-e2e-ait-9759ef96-serving-cert" not found
Aug 05 17:54:20.174875 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:20.174858    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbf4107c-fce6-4269-9f4b-7b49c0eaef70-openshift-service-ca\") pod \"maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw\" (UID: \"fbf4107c-fce6-4269-9f4b-7b49c0eaef70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw"
Aug 05 17:54:20.182835 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:20.182812    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hn4j\" (UniqueName: \"kubernetes.io/projected/fbf4107c-fce6-4269-9f4b-7b49c0eaef70-kube-api-access-2hn4j\") pod \"maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw\" (UID: \"fbf4107c-fce6-4269-9f4b-7b49c0eaef70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw"
Aug 05 17:54:20.678351 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:20.678307    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbf4107c-fce6-4269-9f4b-7b49c0eaef70-maas-api-tls\") pod \"maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw\" (UID: \"fbf4107c-fce6-4269-9f4b-7b49c0eaef70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw"
Aug 05 17:54:20.680686 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:20.680659    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbf4107c-fce6-4269-9f4b-7b49c0eaef70-maas-api-tls\") pod \"maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw\" (UID: \"fbf4107c-fce6-4269-9f4b-7b49c0eaef70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw"
Aug 05 17:54:20.728916 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:20.728879    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9759ef96-687746f584-5tclw"]
Aug 05 17:54:20.732386 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:20.732369    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9759ef96-687746f584-5tclw"
Aug 05 17:54:20.735157 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:20.735131    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-9759ef96\""
Aug 05 17:54:20.735264 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:20.735228    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-9759ef96-dockercfg-ff8bc\""
Aug 05 17:54:20.742545 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:20.742517    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9759ef96-687746f584-5tclw"]
Aug 05 17:54:20.879765 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:20.879720    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b58c6d47-2621-4142-97b9-7ec5bb073c66-plugins-config-volume\") pod \"payload-processing-e2e-ait-9759ef96-687746f584-5tclw\" (UID: \"b58c6d47-2621-4142-97b9-7ec5bb073c66\") " pod="openshift-ingress/payload-processing-e2e-ait-9759ef96-687746f584-5tclw"
Aug 05 17:54:20.879765 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:20.879772    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sbvg\" (UniqueName: \"kubernetes.io/projected/b58c6d47-2621-4142-97b9-7ec5bb073c66-kube-api-access-9sbvg\") pod \"payload-processing-e2e-ait-9759ef96-687746f584-5tclw\" (UID: \"b58c6d47-2621-4142-97b9-7ec5bb073c66\") " pod="openshift-ingress/payload-processing-e2e-ait-9759ef96-687746f584-5tclw"
Aug 05 17:54:20.890633 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:20.890603    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw"
Aug 05 17:54:20.980898 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:20.980825    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b58c6d47-2621-4142-97b9-7ec5bb073c66-plugins-config-volume\") pod \"payload-processing-e2e-ait-9759ef96-687746f584-5tclw\" (UID: \"b58c6d47-2621-4142-97b9-7ec5bb073c66\") " pod="openshift-ingress/payload-processing-e2e-ait-9759ef96-687746f584-5tclw"
Aug 05 17:54:20.980898 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:20.980863    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sbvg\" (UniqueName: \"kubernetes.io/projected/b58c6d47-2621-4142-97b9-7ec5bb073c66-kube-api-access-9sbvg\") pod \"payload-processing-e2e-ait-9759ef96-687746f584-5tclw\" (UID: \"b58c6d47-2621-4142-97b9-7ec5bb073c66\") " pod="openshift-ingress/payload-processing-e2e-ait-9759ef96-687746f584-5tclw"
Aug 05 17:54:20.981783 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:20.981724    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b58c6d47-2621-4142-97b9-7ec5bb073c66-plugins-config-volume\") pod \"payload-processing-e2e-ait-9759ef96-687746f584-5tclw\" (UID: \"b58c6d47-2621-4142-97b9-7ec5bb073c66\") " pod="openshift-ingress/payload-processing-e2e-ait-9759ef96-687746f584-5tclw"
Aug 05 17:54:20.991397 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:20.991366    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sbvg\" (UniqueName: \"kubernetes.io/projected/b58c6d47-2621-4142-97b9-7ec5bb073c66-kube-api-access-9sbvg\") pod \"payload-processing-e2e-ait-9759ef96-687746f584-5tclw\" (UID: \"b58c6d47-2621-4142-97b9-7ec5bb073c66\") " pod="openshift-ingress/payload-processing-e2e-ait-9759ef96-687746f584-5tclw"
Aug 05 17:54:21.032350 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:21.032318    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw"]
Aug 05 17:54:21.034516 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:54:21.034482    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbf4107c_fce6_4269_9f4b_7b49c0eaef70.slice/crio-2bb3fb29526cbdb71ad24aa0043f76dd52c823f5a3d0fb08cbf7ef76e2465014 WatchSource:0}: Error finding container 2bb3fb29526cbdb71ad24aa0043f76dd52c823f5a3d0fb08cbf7ef76e2465014: Status 404 returned error can't find the container with id 2bb3fb29526cbdb71ad24aa0043f76dd52c823f5a3d0fb08cbf7ef76e2465014
Aug 05 17:54:21.044520 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:21.044495    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9759ef96-687746f584-5tclw"
Aug 05 17:54:21.182298 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:21.182259    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9759ef96-687746f584-5tclw"]
Aug 05 17:54:21.185584 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:54:21.185539    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb58c6d47_2621_4142_97b9_7ec5bb073c66.slice/crio-68dfe18aafe125a33514baeef347a813b29f0bfd26b5f953d0f127c20d001280 WatchSource:0}: Error finding container 68dfe18aafe125a33514baeef347a813b29f0bfd26b5f953d0f127c20d001280: Status 404 returned error can't find the container with id 68dfe18aafe125a33514baeef347a813b29f0bfd26b5f953d0f127c20d001280
Aug 05 17:54:21.435138 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:21.435105    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw" event={"ID":"fbf4107c-fce6-4269-9f4b-7b49c0eaef70","Type":"ContainerStarted","Data":"2bb3fb29526cbdb71ad24aa0043f76dd52c823f5a3d0fb08cbf7ef76e2465014"}
Aug 05 17:54:21.436901 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:21.436875    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9759ef96-687746f584-5tclw" event={"ID":"b58c6d47-2621-4142-97b9-7ec5bb073c66","Type":"ContainerStarted","Data":"66aee1e168d2ad7c00fc6adf3a28b1c247d848571ee98c394e795522887b76c0"}
Aug 05 17:54:21.437007 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:21.436926    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9759ef96-687746f584-5tclw" event={"ID":"b58c6d47-2621-4142-97b9-7ec5bb073c66","Type":"ContainerStarted","Data":"68dfe18aafe125a33514baeef347a813b29f0bfd26b5f953d0f127c20d001280"}
Aug 05 17:54:21.437007 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:21.436993    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-9759ef96-687746f584-5tclw"
Aug 05 17:54:21.452681 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:21.452627    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-9759ef96-687746f584-5tclw" podStartSLOduration=1.452610527 podStartE2EDuration="1.452610527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:54:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:54:21.451691876 +0000 UTC m=+1968.433269305" watchObservedRunningTime="2026-08-05 17:54:21.452610527 +0000 UTC m=+1968.434187956"
Aug 05 17:54:22.443746 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:22.443679    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw" event={"ID":"fbf4107c-fce6-4269-9f4b-7b49c0eaef70","Type":"ContainerStarted","Data":"122358cc201f75e06348f0d21eca95aa3e1d5fd9e2f8069942a2a0ffb93d4aa1"}
Aug 05 17:54:22.444260 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:22.443804    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw"
Aug 05 17:54:22.461213 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:22.461169    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw" podStartSLOduration=3.111899287 podStartE2EDuration="3.461154468s" totalImagesPullingTime="349.255181ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:54:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:54:22.460046056 +0000 UTC m=+1969.441623484" watchObservedRunningTime="2026-08-05 17:54:22.461154468 +0000 UTC m=+1969.442731908"
Aug 05 17:54:28.455649 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:28.455569    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw"
Aug 05 17:54:32.445835 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:32.445808    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-9759ef96-687746f584-5tclw"
Aug 05 17:54:32.495125 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:32.495099    2550 generic.go:356] "Generic (PLEG): container finished" podID="3b7f68b9-5556-47e0-80bb-ef08da8252ca" containerID="280d98e501c5d5df27efefd99cb0ccb0d56d9366f1da2f4cf8a392e81417a0d9" exitCode=0
Aug 05 17:54:32.495268 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:32.495167    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp" event={"ID":"3b7f68b9-5556-47e0-80bb-ef08da8252ca","Type":"ContainerDied","Data":"280d98e501c5d5df27efefd99cb0ccb0d56d9366f1da2f4cf8a392e81417a0d9"}
Aug 05 17:54:32.495504 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:32.495488    2550 scope.go:117] "RemoveContainer" containerID="280d98e501c5d5df27efefd99cb0ccb0d56d9366f1da2f4cf8a392e81417a0d9"
Aug 05 17:54:32.496403 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:32.496377    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 17:54:32.496487 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:32.496438    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 17:54:32.496487 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:32.496470    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 17:54:33.419821 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:33.419776    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:33.501673 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:33.501633    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp" event={"ID":"3b7f68b9-5556-47e0-80bb-ef08da8252ca","Type":"ContainerStarted","Data":"823afa8054ba570e46ab1a2c8902611259364efc11476b83538bbe36ac9a22c3"}
Aug 05 17:54:34.343856 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:34.343824    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:34.344795 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:34.344773    2550 patch_prober.go:28] interesting pod/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Aug 05 17:54:34.344907 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:34.344826    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp" podUID="3b7f68b9-5556-47e0-80bb-ef08da8252ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Aug 05 17:54:35.344054 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:35.344017    2550 patch_prober.go:28] interesting pod/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Aug 05 17:54:35.344421 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:35.344069    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp" podUID="3b7f68b9-5556-47e0-80bb-ef08da8252ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Aug 05 17:54:36.344792 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:36.344745    2550 patch_prober.go:28] interesting pod/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Aug 05 17:54:36.345187 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:36.344814    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp" podUID="3b7f68b9-5556-47e0-80bb-ef08da8252ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Aug 05 17:54:37.344058 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:37.344024    2550 patch_prober.go:28] interesting pod/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Aug 05 17:54:37.344245 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:37.344077    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp" podUID="3b7f68b9-5556-47e0-80bb-ef08da8252ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Aug 05 17:54:38.344415 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:38.344379    2550 patch_prober.go:28] interesting pod/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Aug 05 17:54:38.344846 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:38.344432    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp" podUID="3b7f68b9-5556-47e0-80bb-ef08da8252ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Aug 05 17:54:39.347629 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:39.347598    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:39.348087 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:39.347883    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:39.348501 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:39.348486    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:50.411752 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:50.411712    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw"]
Aug 05 17:54:50.412329 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:50.411968    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw" podUID="fbf4107c-fce6-4269-9f4b-7b49c0eaef70" containerName="maas-api" containerID="cri-o://122358cc201f75e06348f0d21eca95aa3e1d5fd9e2f8069942a2a0ffb93d4aa1" gracePeriod=30
Aug 05 17:54:50.468777 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:50.468745    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9759ef96-687746f584-5tclw"]
Aug 05 17:54:50.469002 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:50.468955    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-9759ef96-687746f584-5tclw" podUID="b58c6d47-2621-4142-97b9-7ec5bb073c66" containerName="payload-processing" containerID="cri-o://66aee1e168d2ad7c00fc6adf3a28b1c247d848571ee98c394e795522887b76c0" gracePeriod=30
Aug 05 17:54:50.591580 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:50.591549    2550 generic.go:356] "Generic (PLEG): container finished" podID="fbf4107c-fce6-4269-9f4b-7b49c0eaef70" containerID="122358cc201f75e06348f0d21eca95aa3e1d5fd9e2f8069942a2a0ffb93d4aa1" exitCode=0
Aug 05 17:54:50.591711 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:50.591622    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw" event={"ID":"fbf4107c-fce6-4269-9f4b-7b49c0eaef70","Type":"ContainerDied","Data":"122358cc201f75e06348f0d21eca95aa3e1d5fd9e2f8069942a2a0ffb93d4aa1"}
Aug 05 17:54:50.593653 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:50.593630    2550 generic.go:356] "Generic (PLEG): container finished" podID="b58c6d47-2621-4142-97b9-7ec5bb073c66" containerID="66aee1e168d2ad7c00fc6adf3a28b1c247d848571ee98c394e795522887b76c0" exitCode=0
Aug 05 17:54:50.593812 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:50.593678    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9759ef96-687746f584-5tclw" event={"ID":"b58c6d47-2621-4142-97b9-7ec5bb073c66","Type":"ContainerDied","Data":"66aee1e168d2ad7c00fc6adf3a28b1c247d848571ee98c394e795522887b76c0"}
Aug 05 17:54:50.730695 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:50.730671    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw"
Aug 05 17:54:50.757477 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:50.757204    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9759ef96-687746f584-5tclw"
Aug 05 17:54:50.852379 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:50.852344    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9sbvg\" (UniqueName: \"kubernetes.io/projected/b58c6d47-2621-4142-97b9-7ec5bb073c66-kube-api-access-9sbvg\") pod \"b58c6d47-2621-4142-97b9-7ec5bb073c66\" (UID: \"b58c6d47-2621-4142-97b9-7ec5bb073c66\") "
Aug 05 17:54:50.852567 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:50.852438    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbf4107c-fce6-4269-9f4b-7b49c0eaef70-maas-api-tls\") pod \"fbf4107c-fce6-4269-9f4b-7b49c0eaef70\" (UID: \"fbf4107c-fce6-4269-9f4b-7b49c0eaef70\") "
Aug 05 17:54:50.852567 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:50.852496    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hn4j\" (UniqueName: \"kubernetes.io/projected/fbf4107c-fce6-4269-9f4b-7b49c0eaef70-kube-api-access-2hn4j\") pod \"fbf4107c-fce6-4269-9f4b-7b49c0eaef70\" (UID: \"fbf4107c-fce6-4269-9f4b-7b49c0eaef70\") "
Aug 05 17:54:50.852737 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:50.852617    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b58c6d47-2621-4142-97b9-7ec5bb073c66-plugins-config-volume\") pod \"b58c6d47-2621-4142-97b9-7ec5bb073c66\" (UID: \"b58c6d47-2621-4142-97b9-7ec5bb073c66\") "
Aug 05 17:54:50.852737 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:50.852672    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbf4107c-fce6-4269-9f4b-7b49c0eaef70-openshift-service-ca\") pod \"fbf4107c-fce6-4269-9f4b-7b49c0eaef70\" (UID: \"fbf4107c-fce6-4269-9f4b-7b49c0eaef70\") "
Aug 05 17:54:50.853016 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:50.852986    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b58c6d47-2621-4142-97b9-7ec5bb073c66-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b58c6d47-2621-4142-97b9-7ec5bb073c66" (UID: "b58c6d47-2621-4142-97b9-7ec5bb073c66"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:54:50.853102 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:50.853020    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbf4107c-fce6-4269-9f4b-7b49c0eaef70-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fbf4107c-fce6-4269-9f4b-7b49c0eaef70" (UID: "fbf4107c-fce6-4269-9f4b-7b49c0eaef70"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:54:50.854498 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:50.854473    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b58c6d47-2621-4142-97b9-7ec5bb073c66-kube-api-access-9sbvg" (OuterVolumeSpecName: "kube-api-access-9sbvg") pod "b58c6d47-2621-4142-97b9-7ec5bb073c66" (UID: "b58c6d47-2621-4142-97b9-7ec5bb073c66"). InnerVolumeSpecName "kube-api-access-9sbvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:54:50.854593 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:50.854527    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbf4107c-fce6-4269-9f4b-7b49c0eaef70-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fbf4107c-fce6-4269-9f4b-7b49c0eaef70" (UID: "fbf4107c-fce6-4269-9f4b-7b49c0eaef70"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:54:50.854593 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:50.854529    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbf4107c-fce6-4269-9f4b-7b49c0eaef70-kube-api-access-2hn4j" (OuterVolumeSpecName: "kube-api-access-2hn4j") pod "fbf4107c-fce6-4269-9f4b-7b49c0eaef70" (UID: "fbf4107c-fce6-4269-9f4b-7b49c0eaef70"). InnerVolumeSpecName "kube-api-access-2hn4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:54:50.954056 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:50.953991    2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbf4107c-fce6-4269-9f4b-7b49c0eaef70-openshift-service-ca\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:54:50.954056 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:50.954015    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9sbvg\" (UniqueName: \"kubernetes.io/projected/b58c6d47-2621-4142-97b9-7ec5bb073c66-kube-api-access-9sbvg\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:54:50.954056 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:50.954025    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbf4107c-fce6-4269-9f4b-7b49c0eaef70-maas-api-tls\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:54:50.954056 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:50.954033    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hn4j\" (UniqueName: \"kubernetes.io/projected/fbf4107c-fce6-4269-9f4b-7b49c0eaef70-kube-api-access-2hn4j\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:54:50.954056 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:50.954043    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b58c6d47-2621-4142-97b9-7ec5bb073c66-plugins-config-volume\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:54:51.604829 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:51.604797    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9759ef96-687746f584-5tclw"
Aug 05 17:54:51.605297 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:51.604826    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9759ef96-687746f584-5tclw" event={"ID":"b58c6d47-2621-4142-97b9-7ec5bb073c66","Type":"ContainerDied","Data":"68dfe18aafe125a33514baeef347a813b29f0bfd26b5f953d0f127c20d001280"}
Aug 05 17:54:51.605297 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:51.604912    2550 scope.go:117] "RemoveContainer" containerID="66aee1e168d2ad7c00fc6adf3a28b1c247d848571ee98c394e795522887b76c0"
Aug 05 17:54:51.606146 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:51.605977    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw" event={"ID":"fbf4107c-fce6-4269-9f4b-7b49c0eaef70","Type":"ContainerDied","Data":"2bb3fb29526cbdb71ad24aa0043f76dd52c823f5a3d0fb08cbf7ef76e2465014"}
Aug 05 17:54:51.606146 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:51.606009    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw"
Aug 05 17:54:51.614828 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:51.614813    2550 scope.go:117] "RemoveContainer" containerID="122358cc201f75e06348f0d21eca95aa3e1d5fd9e2f8069942a2a0ffb93d4aa1"
Aug 05 17:54:51.626708 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:51.626679    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9759ef96-687746f584-5tclw"]
Aug 05 17:54:51.629817 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:51.629797    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9759ef96-687746f584-5tclw"]
Aug 05 17:54:51.639405 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:51.639383    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw"]
Aug 05 17:54:51.641773 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:51.641755    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9759ef96-5578c5b84b-snrgw"]
Aug 05 17:54:51.703247 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:51.703165    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b58c6d47-2621-4142-97b9-7ec5bb073c66" path="/var/lib/kubelet/pods/b58c6d47-2621-4142-97b9-7ec5bb073c66/volumes"
Aug 05 17:54:51.703723 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:51.703690    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbf4107c-fce6-4269-9f4b-7b49c0eaef70" path="/var/lib/kubelet/pods/fbf4107c-fce6-4269-9f4b-7b49c0eaef70/volumes"
Aug 05 17:54:51.704053 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:51.704036    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"]
Aug 05 17:54:51.704298 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:51.704275    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp" podUID="3b7f68b9-5556-47e0-80bb-ef08da8252ca" containerName="istio-proxy" containerID="cri-o://823afa8054ba570e46ab1a2c8902611259364efc11476b83538bbe36ac9a22c3" gracePeriod=30
Aug 05 17:54:54.350099 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:54.350061    2550 patch_prober.go:28] interesting pod/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Aug 05 17:54:54.350476 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:54.350126    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp" podUID="3b7f68b9-5556-47e0-80bb-ef08da8252ca" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 05 17:54:55.890338 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:55.890308    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f"]
Aug 05 17:54:55.890824 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:55.890645    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbf4107c-fce6-4269-9f4b-7b49c0eaef70" containerName="maas-api"
Aug 05 17:54:55.890824 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:55.890657    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbf4107c-fce6-4269-9f4b-7b49c0eaef70" containerName="maas-api"
Aug 05 17:54:55.890824 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:55.890678    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b58c6d47-2621-4142-97b9-7ec5bb073c66" containerName="payload-processing"
Aug 05 17:54:55.890824 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:55.890684    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="b58c6d47-2621-4142-97b9-7ec5bb073c66" containerName="payload-processing"
Aug 05 17:54:55.890824 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:55.890763    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="b58c6d47-2621-4142-97b9-7ec5bb073c66" containerName="payload-processing"
Aug 05 17:54:55.890824 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:55.890774    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbf4107c-fce6-4269-9f4b-7b49c0eaef70" containerName="maas-api"
Aug 05 17:54:55.895204 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:55.895180    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f"
Aug 05 17:54:55.897606 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:55.897578    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-0d1b0907-serving-cert\""
Aug 05 17:54:55.903065 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:55.902779    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f"]
Aug 05 17:54:55.994335 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:55.994301    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72x88\" (UniqueName: \"kubernetes.io/projected/29cc806f-0e5e-41a8-a86f-6ad718f0b82a-kube-api-access-72x88\") pod \"maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f\" (UID: \"29cc806f-0e5e-41a8-a86f-6ad718f0b82a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f"
Aug 05 17:54:55.994528 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:55.994340    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29cc806f-0e5e-41a8-a86f-6ad718f0b82a-openshift-service-ca\") pod \"maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f\" (UID: \"29cc806f-0e5e-41a8-a86f-6ad718f0b82a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f"
Aug 05 17:54:55.994528 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:55.994459    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29cc806f-0e5e-41a8-a86f-6ad718f0b82a-maas-api-tls\") pod \"maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f\" (UID: \"29cc806f-0e5e-41a8-a86f-6ad718f0b82a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f"
Aug 05 17:54:56.095436 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:56.095403    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72x88\" (UniqueName: \"kubernetes.io/projected/29cc806f-0e5e-41a8-a86f-6ad718f0b82a-kube-api-access-72x88\") pod \"maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f\" (UID: \"29cc806f-0e5e-41a8-a86f-6ad718f0b82a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f"
Aug 05 17:54:56.095436 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:56.095440    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29cc806f-0e5e-41a8-a86f-6ad718f0b82a-openshift-service-ca\") pod \"maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f\" (UID: \"29cc806f-0e5e-41a8-a86f-6ad718f0b82a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f"
Aug 05 17:54:56.095603 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:56.095485    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29cc806f-0e5e-41a8-a86f-6ad718f0b82a-maas-api-tls\") pod \"maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f\" (UID: \"29cc806f-0e5e-41a8-a86f-6ad718f0b82a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f"
Aug 05 17:54:56.095641 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:54:56.095631    2550 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-serving-cert: secret "maas-api-e2e-derive-0d1b0907-serving-cert" not found
Aug 05 17:54:56.095732 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:54:56.095695    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29cc806f-0e5e-41a8-a86f-6ad718f0b82a-maas-api-tls podName:29cc806f-0e5e-41a8-a86f-6ad718f0b82a nodeName:}" failed. No retries permitted until 2026-08-05 17:54:56.595676497 +0000 UTC m=+2003.577253903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/29cc806f-0e5e-41a8-a86f-6ad718f0b82a-maas-api-tls") pod "maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f" (UID: "29cc806f-0e5e-41a8-a86f-6ad718f0b82a") : secret "maas-api-e2e-derive-0d1b0907-serving-cert" not found
Aug 05 17:54:56.096125 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:56.096108    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29cc806f-0e5e-41a8-a86f-6ad718f0b82a-openshift-service-ca\") pod \"maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f\" (UID: \"29cc806f-0e5e-41a8-a86f-6ad718f0b82a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f"
Aug 05 17:54:56.104107 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:56.104080    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72x88\" (UniqueName: \"kubernetes.io/projected/29cc806f-0e5e-41a8-a86f-6ad718f0b82a-kube-api-access-72x88\") pod \"maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f\" (UID: \"29cc806f-0e5e-41a8-a86f-6ad718f0b82a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f"
Aug 05 17:54:56.541898 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:56.541866    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs"]
Aug 05 17:54:56.545291 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:56.545275    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs"
Aug 05 17:54:56.547573 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:56.547553    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-0d1b0907-dockercfg-tcvr9\""
Aug 05 17:54:56.547662 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:56.547626    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-0d1b0907\""
Aug 05 17:54:56.555868 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:56.555842    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs"]
Aug 05 17:54:56.599755 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:56.599722    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29cc806f-0e5e-41a8-a86f-6ad718f0b82a-maas-api-tls\") pod \"maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f\" (UID: \"29cc806f-0e5e-41a8-a86f-6ad718f0b82a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f"
Aug 05 17:54:56.599927 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:56.599835    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhhk4\" (UniqueName: \"kubernetes.io/projected/9f8976ca-9b13-444d-a730-dc0124d98945-kube-api-access-mhhk4\") pod \"payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs\" (UID: \"9f8976ca-9b13-444d-a730-dc0124d98945\") " pod="openshift-ingress/payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs"
Aug 05 17:54:56.599927 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:56.599866    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f8976ca-9b13-444d-a730-dc0124d98945-plugins-config-volume\") pod \"payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs\" (UID: \"9f8976ca-9b13-444d-a730-dc0124d98945\") " pod="openshift-ingress/payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs"
Aug 05 17:54:56.602051 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:56.602021    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29cc806f-0e5e-41a8-a86f-6ad718f0b82a-maas-api-tls\") pod \"maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f\" (UID: \"29cc806f-0e5e-41a8-a86f-6ad718f0b82a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f"
Aug 05 17:54:56.700948 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:56.700914    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhhk4\" (UniqueName: \"kubernetes.io/projected/9f8976ca-9b13-444d-a730-dc0124d98945-kube-api-access-mhhk4\") pod \"payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs\" (UID: \"9f8976ca-9b13-444d-a730-dc0124d98945\") " pod="openshift-ingress/payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs"
Aug 05 17:54:56.700948 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:56.700955    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f8976ca-9b13-444d-a730-dc0124d98945-plugins-config-volume\") pod \"payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs\" (UID: \"9f8976ca-9b13-444d-a730-dc0124d98945\") " pod="openshift-ingress/payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs"
Aug 05 17:54:56.701644 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:56.701625    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f8976ca-9b13-444d-a730-dc0124d98945-plugins-config-volume\") pod \"payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs\" (UID: \"9f8976ca-9b13-444d-a730-dc0124d98945\") " pod="openshift-ingress/payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs"
Aug 05 17:54:56.714326 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:56.714297    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhhk4\" (UniqueName: \"kubernetes.io/projected/9f8976ca-9b13-444d-a730-dc0124d98945-kube-api-access-mhhk4\") pod \"payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs\" (UID: \"9f8976ca-9b13-444d-a730-dc0124d98945\") " pod="openshift-ingress/payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs"
Aug 05 17:54:56.807303 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:56.807242    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f"
Aug 05 17:54:56.856807 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:56.856645    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs"
Aug 05 17:54:56.969299 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:56.969232    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f"]
Aug 05 17:54:56.970875 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:56.970845    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:56.971505 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:54:56.971475    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29cc806f_0e5e_41a8_a86f_6ad718f0b82a.slice/crio-5666dfc95229f972337bc29cdc4452731aab8ea1fb8b9067f50e8f4f8161391e WatchSource:0}: Error finding container 5666dfc95229f972337bc29cdc4452731aab8ea1fb8b9067f50e8f4f8161391e: Status 404 returned error can't find the container with id 5666dfc95229f972337bc29cdc4452731aab8ea1fb8b9067f50e8f4f8161391e
Aug 05 17:54:57.014875 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.014721    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs"]
Aug 05 17:54:57.017388 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:54:57.017352    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8976ca_9b13_444d_a730_dc0124d98945.slice/crio-ce8bf87e195ff0c199d9c5eeeec4751a154ed0f43d230a5e1e5d1989f1a3152f WatchSource:0}: Error finding container ce8bf87e195ff0c199d9c5eeeec4751a154ed0f43d230a5e1e5d1989f1a3152f: Status 404 returned error can't find the container with id ce8bf87e195ff0c199d9c5eeeec4751a154ed0f43d230a5e1e5d1989f1a3152f
Aug 05 17:54:57.105058 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.105016    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7vvx\" (UniqueName: \"kubernetes.io/projected/3b7f68b9-5556-47e0-80bb-ef08da8252ca-kube-api-access-f7vvx\") pod \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") "
Aug 05 17:54:57.105199 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.105099    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istio-podinfo\") pod \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") "
Aug 05 17:54:57.105199 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.105129    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istiod-ca-cert\") pod \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") "
Aug 05 17:54:57.105199 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.105167    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-workload-socket\") pod \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") "
Aug 05 17:54:57.105199 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.105186    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istio-envoy\") pod \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") "
Aug 05 17:54:57.105420 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.105251    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istio-token\") pod \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") "
Aug 05 17:54:57.105420 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.105280    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-credential-socket\") pod \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") "
Aug 05 17:54:57.105420 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.105331    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-workload-certs\") pod \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") "
Aug 05 17:54:57.105420 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.105371    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istio-data\") pod \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\" (UID: \"3b7f68b9-5556-47e0-80bb-ef08da8252ca\") "
Aug 05 17:54:57.105631 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.105418    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3b7f68b9-5556-47e0-80bb-ef08da8252ca" (UID: "3b7f68b9-5556-47e0-80bb-ef08da8252ca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:54:57.105631 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.105512    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3b7f68b9-5556-47e0-80bb-ef08da8252ca" (UID: "3b7f68b9-5556-47e0-80bb-ef08da8252ca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:54:57.105631 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.105593    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3b7f68b9-5556-47e0-80bb-ef08da8252ca" (UID: "3b7f68b9-5556-47e0-80bb-ef08da8252ca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:54:57.105832 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.105686    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istio-data" (OuterVolumeSpecName: "istio-data") pod "3b7f68b9-5556-47e0-80bb-ef08da8252ca" (UID: "3b7f68b9-5556-47e0-80bb-ef08da8252ca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:54:57.105832 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.105766    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3b7f68b9-5556-47e0-80bb-ef08da8252ca" (UID: "3b7f68b9-5556-47e0-80bb-ef08da8252ca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:54:57.106349 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.106328    2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-credential-socket\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:54:57.106502 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.106354    2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-workload-certs\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:54:57.106502 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.106368    2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istio-data\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:54:57.106502 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.106397    2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istiod-ca-cert\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:54:57.106502 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.106409    2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-workload-socket\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:54:57.107921 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.107883    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3b7f68b9-5556-47e0-80bb-ef08da8252ca" (UID: "3b7f68b9-5556-47e0-80bb-ef08da8252ca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 05 17:54:57.107921 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.107893    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istio-token" (OuterVolumeSpecName: "istio-token") pod "3b7f68b9-5556-47e0-80bb-ef08da8252ca" (UID: "3b7f68b9-5556-47e0-80bb-ef08da8252ca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:54:57.108089 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.108016    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3b7f68b9-5556-47e0-80bb-ef08da8252ca" (UID: "3b7f68b9-5556-47e0-80bb-ef08da8252ca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:54:57.108089 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.108060    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b7f68b9-5556-47e0-80bb-ef08da8252ca-kube-api-access-f7vvx" (OuterVolumeSpecName: "kube-api-access-f7vvx") pod "3b7f68b9-5556-47e0-80bb-ef08da8252ca" (UID: "3b7f68b9-5556-47e0-80bb-ef08da8252ca"). InnerVolumeSpecName "kube-api-access-f7vvx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:54:57.207001 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.206968    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7vvx\" (UniqueName: \"kubernetes.io/projected/3b7f68b9-5556-47e0-80bb-ef08da8252ca-kube-api-access-f7vvx\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:54:57.207001 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.206995    2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istio-podinfo\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:54:57.207001 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.207005    2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istio-envoy\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:54:57.207226 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.207013    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b7f68b9-5556-47e0-80bb-ef08da8252ca-istio-token\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:54:57.638504 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.638371    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f" event={"ID":"29cc806f-0e5e-41a8-a86f-6ad718f0b82a","Type":"ContainerStarted","Data":"0d6e8f0ef0868dc498cfb0fdae7a2c0b268834f351e66d5b52ee22dde130928f"}
Aug 05 17:54:57.638504 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.638428    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f" event={"ID":"29cc806f-0e5e-41a8-a86f-6ad718f0b82a","Type":"ContainerStarted","Data":"5666dfc95229f972337bc29cdc4452731aab8ea1fb8b9067f50e8f4f8161391e"}
Aug 05 17:54:57.638504 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.638458    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f"
Aug 05 17:54:57.640102 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.640071    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs" event={"ID":"9f8976ca-9b13-444d-a730-dc0124d98945","Type":"ContainerStarted","Data":"e34261a01eb156becca91987ef1dfd98052bac4052fc11add0a5b2318d317a75"}
Aug 05 17:54:57.640362 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.640130    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs" event={"ID":"9f8976ca-9b13-444d-a730-dc0124d98945","Type":"ContainerStarted","Data":"ce8bf87e195ff0c199d9c5eeeec4751a154ed0f43d230a5e1e5d1989f1a3152f"}
Aug 05 17:54:57.640362 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.640164    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs"
Aug 05 17:54:57.641812 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.641787    2550 generic.go:356] "Generic (PLEG): container finished" podID="3b7f68b9-5556-47e0-80bb-ef08da8252ca" containerID="823afa8054ba570e46ab1a2c8902611259364efc11476b83538bbe36ac9a22c3" exitCode=0
Aug 05 17:54:57.641922 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.641832    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"
Aug 05 17:54:57.641922 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.641847    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp" event={"ID":"3b7f68b9-5556-47e0-80bb-ef08da8252ca","Type":"ContainerDied","Data":"823afa8054ba570e46ab1a2c8902611259364efc11476b83538bbe36ac9a22c3"}
Aug 05 17:54:57.641922 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.641902    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp" event={"ID":"3b7f68b9-5556-47e0-80bb-ef08da8252ca","Type":"ContainerDied","Data":"a6cd8194c9d1a1549b376b62397ff4b206337d3d3758c48b6151924f01ad9a46"}
Aug 05 17:54:57.642046 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.641937    2550 scope.go:117] "RemoveContainer" containerID="823afa8054ba570e46ab1a2c8902611259364efc11476b83538bbe36ac9a22c3"
Aug 05 17:54:57.651563 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.651542    2550 scope.go:117] "RemoveContainer" containerID="280d98e501c5d5df27efefd99cb0ccb0d56d9366f1da2f4cf8a392e81417a0d9"
Aug 05 17:54:57.659969 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.659952    2550 scope.go:117] "RemoveContainer" containerID="823afa8054ba570e46ab1a2c8902611259364efc11476b83538bbe36ac9a22c3"
Aug 05 17:54:57.660219 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.660184    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f" podStartSLOduration=2.351969341 podStartE2EDuration="2.660171452s" totalImagesPullingTime="308.202111ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:54:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:54:57.657993423 +0000 UTC m=+2004.639570853" watchObservedRunningTime="2026-08-05 17:54:57.660171452 +0000 UTC m=+2004.641748881"
Aug 05 17:54:57.660317 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:54:57.660221    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"823afa8054ba570e46ab1a2c8902611259364efc11476b83538bbe36ac9a22c3\": container with ID starting with 823afa8054ba570e46ab1a2c8902611259364efc11476b83538bbe36ac9a22c3 not found: ID does not exist" containerID="823afa8054ba570e46ab1a2c8902611259364efc11476b83538bbe36ac9a22c3"
Aug 05 17:54:57.660317 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.660259    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"823afa8054ba570e46ab1a2c8902611259364efc11476b83538bbe36ac9a22c3"} err="failed to get container status \"823afa8054ba570e46ab1a2c8902611259364efc11476b83538bbe36ac9a22c3\": rpc error: code = NotFound desc = could not find container \"823afa8054ba570e46ab1a2c8902611259364efc11476b83538bbe36ac9a22c3\": container with ID starting with 823afa8054ba570e46ab1a2c8902611259364efc11476b83538bbe36ac9a22c3 not found: ID does not exist"
Aug 05 17:54:57.660317 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.660282    2550 scope.go:117] "RemoveContainer" containerID="280d98e501c5d5df27efefd99cb0ccb0d56d9366f1da2f4cf8a392e81417a0d9"
Aug 05 17:54:57.660543 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:54:57.660524    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"280d98e501c5d5df27efefd99cb0ccb0d56d9366f1da2f4cf8a392e81417a0d9\": container with ID starting with 280d98e501c5d5df27efefd99cb0ccb0d56d9366f1da2f4cf8a392e81417a0d9 not found: ID does not exist" containerID="280d98e501c5d5df27efefd99cb0ccb0d56d9366f1da2f4cf8a392e81417a0d9"
Aug 05 17:54:57.660599 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.660551    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"280d98e501c5d5df27efefd99cb0ccb0d56d9366f1da2f4cf8a392e81417a0d9"} err="failed to get container status \"280d98e501c5d5df27efefd99cb0ccb0d56d9366f1da2f4cf8a392e81417a0d9\": rpc error: code = NotFound desc = could not find container \"280d98e501c5d5df27efefd99cb0ccb0d56d9366f1da2f4cf8a392e81417a0d9\": container with ID starting with 280d98e501c5d5df27efefd99cb0ccb0d56d9366f1da2f4cf8a392e81417a0d9 not found: ID does not exist"
Aug 05 17:54:57.673317 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.673296    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"]
Aug 05 17:54:57.677852 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.677824    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-9759ef96-openshift-default-6b85696db4-44pnp"]
Aug 05 17:54:57.692829 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.692785    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs" podStartSLOduration=1.692770934 podStartE2EDuration="1.692770934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:54:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:54:57.691967499 +0000 UTC m=+2004.673544930" watchObservedRunningTime="2026-08-05 17:54:57.692770934 +0000 UTC m=+2004.674348362"
Aug 05 17:54:57.699845 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:54:57.699813    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b7f68b9-5556-47e0-80bb-ef08da8252ca" path="/var/lib/kubelet/pods/3b7f68b9-5556-47e0-80bb-ef08da8252ca/volumes"
Aug 05 17:55:03.653522 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:03.653492    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f"
Aug 05 17:55:08.650596 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:08.650559    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs"
Aug 05 17:55:26.475398 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:26.475363    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f"]
Aug 05 17:55:26.476352 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:26.475623    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f" podUID="29cc806f-0e5e-41a8-a86f-6ad718f0b82a" containerName="maas-api" containerID="cri-o://0d6e8f0ef0868dc498cfb0fdae7a2c0b268834f351e66d5b52ee22dde130928f" gracePeriod=30
Aug 05 17:55:26.535191 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:26.535153    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs"]
Aug 05 17:55:26.535449 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:26.535422    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs" podUID="9f8976ca-9b13-444d-a730-dc0124d98945" containerName="payload-processing" containerID="cri-o://e34261a01eb156becca91987ef1dfd98052bac4052fc11add0a5b2318d317a75" gracePeriod=30
Aug 05 17:55:26.611885 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:55:26.611846    2550 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8976ca_9b13_444d_a730_dc0124d98945.slice/crio-conmon-e34261a01eb156becca91987ef1dfd98052bac4052fc11add0a5b2318d317a75.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8976ca_9b13_444d_a730_dc0124d98945.slice/crio-e34261a01eb156becca91987ef1dfd98052bac4052fc11add0a5b2318d317a75.scope\": RecentStats: unable to find data in memory cache]"
Aug 05 17:55:26.782227 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:26.782198    2550 generic.go:356] "Generic (PLEG): container finished" podID="29cc806f-0e5e-41a8-a86f-6ad718f0b82a" containerID="0d6e8f0ef0868dc498cfb0fdae7a2c0b268834f351e66d5b52ee22dde130928f" exitCode=0
Aug 05 17:55:26.782391 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:26.782269    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f" event={"ID":"29cc806f-0e5e-41a8-a86f-6ad718f0b82a","Type":"ContainerDied","Data":"0d6e8f0ef0868dc498cfb0fdae7a2c0b268834f351e66d5b52ee22dde130928f"}
Aug 05 17:55:26.782391 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:26.782325    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f" event={"ID":"29cc806f-0e5e-41a8-a86f-6ad718f0b82a","Type":"ContainerDied","Data":"5666dfc95229f972337bc29cdc4452731aab8ea1fb8b9067f50e8f4f8161391e"}
Aug 05 17:55:26.782391 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:26.782345    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5666dfc95229f972337bc29cdc4452731aab8ea1fb8b9067f50e8f4f8161391e"
Aug 05 17:55:26.783906 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:26.783884    2550 generic.go:356] "Generic (PLEG): container finished" podID="9f8976ca-9b13-444d-a730-dc0124d98945" containerID="e34261a01eb156becca91987ef1dfd98052bac4052fc11add0a5b2318d317a75" exitCode=0
Aug 05 17:55:26.784014 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:26.783922    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs" event={"ID":"9f8976ca-9b13-444d-a730-dc0124d98945","Type":"ContainerDied","Data":"e34261a01eb156becca91987ef1dfd98052bac4052fc11add0a5b2318d317a75"}
Aug 05 17:55:26.792745 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:26.792715    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f"
Aug 05 17:55:26.856442 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:26.856408    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs"
Aug 05 17:55:26.865888 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:26.865862    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29cc806f-0e5e-41a8-a86f-6ad718f0b82a-openshift-service-ca\") pod \"29cc806f-0e5e-41a8-a86f-6ad718f0b82a\" (UID: \"29cc806f-0e5e-41a8-a86f-6ad718f0b82a\") "
Aug 05 17:55:26.866032 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:26.865903    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29cc806f-0e5e-41a8-a86f-6ad718f0b82a-maas-api-tls\") pod \"29cc806f-0e5e-41a8-a86f-6ad718f0b82a\" (UID: \"29cc806f-0e5e-41a8-a86f-6ad718f0b82a\") "
Aug 05 17:55:26.866032 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:26.865946    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72x88\" (UniqueName: \"kubernetes.io/projected/29cc806f-0e5e-41a8-a86f-6ad718f0b82a-kube-api-access-72x88\") pod \"29cc806f-0e5e-41a8-a86f-6ad718f0b82a\" (UID: \"29cc806f-0e5e-41a8-a86f-6ad718f0b82a\") "
Aug 05 17:55:26.866289 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:26.866263    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29cc806f-0e5e-41a8-a86f-6ad718f0b82a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "29cc806f-0e5e-41a8-a86f-6ad718f0b82a" (UID: "29cc806f-0e5e-41a8-a86f-6ad718f0b82a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:55:26.868107 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:26.868078    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29cc806f-0e5e-41a8-a86f-6ad718f0b82a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "29cc806f-0e5e-41a8-a86f-6ad718f0b82a" (UID: "29cc806f-0e5e-41a8-a86f-6ad718f0b82a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:55:26.868235 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:26.868138    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29cc806f-0e5e-41a8-a86f-6ad718f0b82a-kube-api-access-72x88" (OuterVolumeSpecName: "kube-api-access-72x88") pod "29cc806f-0e5e-41a8-a86f-6ad718f0b82a" (UID: "29cc806f-0e5e-41a8-a86f-6ad718f0b82a"). InnerVolumeSpecName "kube-api-access-72x88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:55:26.967440 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:26.967403    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f8976ca-9b13-444d-a730-dc0124d98945-plugins-config-volume\") pod \"9f8976ca-9b13-444d-a730-dc0124d98945\" (UID: \"9f8976ca-9b13-444d-a730-dc0124d98945\") "
Aug 05 17:55:26.967639 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:26.967494    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhhk4\" (UniqueName: \"kubernetes.io/projected/9f8976ca-9b13-444d-a730-dc0124d98945-kube-api-access-mhhk4\") pod \"9f8976ca-9b13-444d-a730-dc0124d98945\" (UID: \"9f8976ca-9b13-444d-a730-dc0124d98945\") "
Aug 05 17:55:26.967639 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:26.967621    2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29cc806f-0e5e-41a8-a86f-6ad718f0b82a-openshift-service-ca\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:55:26.967639 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:26.967631    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29cc806f-0e5e-41a8-a86f-6ad718f0b82a-maas-api-tls\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:55:26.967639 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:26.967641    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72x88\" (UniqueName: \"kubernetes.io/projected/29cc806f-0e5e-41a8-a86f-6ad718f0b82a-kube-api-access-72x88\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:55:26.967930 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:26.967901    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f8976ca-9b13-444d-a730-dc0124d98945-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9f8976ca-9b13-444d-a730-dc0124d98945" (UID: "9f8976ca-9b13-444d-a730-dc0124d98945"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:55:26.969496 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:26.969474    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f8976ca-9b13-444d-a730-dc0124d98945-kube-api-access-mhhk4" (OuterVolumeSpecName: "kube-api-access-mhhk4") pod "9f8976ca-9b13-444d-a730-dc0124d98945" (UID: "9f8976ca-9b13-444d-a730-dc0124d98945"). InnerVolumeSpecName "kube-api-access-mhhk4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:55:27.068835 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:27.068740    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhhk4\" (UniqueName: \"kubernetes.io/projected/9f8976ca-9b13-444d-a730-dc0124d98945-kube-api-access-mhhk4\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:55:27.068835 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:27.068775    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f8976ca-9b13-444d-a730-dc0124d98945-plugins-config-volume\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:55:27.790565 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:27.790482    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f"
Aug 05 17:55:27.791072 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:27.790481    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs"
Aug 05 17:55:27.791072 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:27.790487    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs" event={"ID":"9f8976ca-9b13-444d-a730-dc0124d98945","Type":"ContainerDied","Data":"ce8bf87e195ff0c199d9c5eeeec4751a154ed0f43d230a5e1e5d1989f1a3152f"}
Aug 05 17:55:27.791072 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:27.790720    2550 scope.go:117] "RemoveContainer" containerID="e34261a01eb156becca91987ef1dfd98052bac4052fc11add0a5b2318d317a75"
Aug 05 17:55:27.815042 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:27.815016    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs"]
Aug 05 17:55:27.820928 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:27.820905    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0d1b0907-58fb984b98-swzvs"]
Aug 05 17:55:27.831989 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:27.831954    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f"]
Aug 05 17:55:27.833828 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:27.833803    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0d1b0907-657c8bff44-mh92f"]
Aug 05 17:55:29.700470 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:29.700427    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29cc806f-0e5e-41a8-a86f-6ad718f0b82a" path="/var/lib/kubelet/pods/29cc806f-0e5e-41a8-a86f-6ad718f0b82a/volumes"
Aug 05 17:55:29.700985 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:55:29.700790    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f8976ca-9b13-444d-a730-dc0124d98945" path="/var/lib/kubelet/pods/9f8976ca-9b13-444d-a730-dc0124d98945/volumes"
Aug 05 17:56:33.617517 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:56:33.617486    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 17:56:33.760428 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:56:33.760401    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 17:57:34.495002 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:34.494969    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj"]
Aug 05 17:57:34.495605 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:34.495347    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f8976ca-9b13-444d-a730-dc0124d98945" containerName="payload-processing"
Aug 05 17:57:34.495605 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:34.495363    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f8976ca-9b13-444d-a730-dc0124d98945" containerName="payload-processing"
Aug 05 17:57:34.495605 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:34.495378    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b7f68b9-5556-47e0-80bb-ef08da8252ca" containerName="istio-proxy"
Aug 05 17:57:34.495605 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:34.495384    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b7f68b9-5556-47e0-80bb-ef08da8252ca" containerName="istio-proxy"
Aug 05 17:57:34.495605 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:34.495389    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b7f68b9-5556-47e0-80bb-ef08da8252ca" containerName="istio-proxy"
Aug 05 17:57:34.495605 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:34.495395    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b7f68b9-5556-47e0-80bb-ef08da8252ca" containerName="istio-proxy"
Aug 05 17:57:34.495605 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:34.495404    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29cc806f-0e5e-41a8-a86f-6ad718f0b82a" containerName="maas-api"
Aug 05 17:57:34.495605 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:34.495409    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="29cc806f-0e5e-41a8-a86f-6ad718f0b82a" containerName="maas-api"
Aug 05 17:57:34.495605 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:34.495464    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="29cc806f-0e5e-41a8-a86f-6ad718f0b82a" containerName="maas-api"
Aug 05 17:57:34.495605 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:34.495472    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f8976ca-9b13-444d-a730-dc0124d98945" containerName="payload-processing"
Aug 05 17:57:34.495605 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:34.495481    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b7f68b9-5556-47e0-80bb-ef08da8252ca" containerName="istio-proxy"
Aug 05 17:57:34.495605 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:34.495486    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b7f68b9-5556-47e0-80bb-ef08da8252ca" containerName="istio-proxy"
Aug 05 17:57:34.498489 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:34.498470    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj"
Aug 05 17:57:34.501172 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:34.501154    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-cc16b618-serving-cert\""
Aug 05 17:57:34.509440 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:34.509415    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj"]
Aug 05 17:57:34.681632 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:34.681605    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64f73236-a562-4e3d-a22d-c0d72f5182f0-maas-api-tls\") pod \"maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj\" (UID: \"64f73236-a562-4e3d-a22d-c0d72f5182f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj"
Aug 05 17:57:34.681833 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:34.681651    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdln9\" (UniqueName: \"kubernetes.io/projected/64f73236-a562-4e3d-a22d-c0d72f5182f0-kube-api-access-cdln9\") pod \"maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj\" (UID: \"64f73236-a562-4e3d-a22d-c0d72f5182f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj"
Aug 05 17:57:34.681833 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:34.681721    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64f73236-a562-4e3d-a22d-c0d72f5182f0-openshift-service-ca\") pod \"maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj\" (UID: \"64f73236-a562-4e3d-a22d-c0d72f5182f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj"
Aug 05 17:57:34.783145 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:34.783060    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdln9\" (UniqueName: \"kubernetes.io/projected/64f73236-a562-4e3d-a22d-c0d72f5182f0-kube-api-access-cdln9\") pod \"maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj\" (UID: \"64f73236-a562-4e3d-a22d-c0d72f5182f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj"
Aug 05 17:57:34.783145 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:34.783106    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64f73236-a562-4e3d-a22d-c0d72f5182f0-openshift-service-ca\") pod \"maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj\" (UID: \"64f73236-a562-4e3d-a22d-c0d72f5182f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj"
Aug 05 17:57:34.783352 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:34.783185    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64f73236-a562-4e3d-a22d-c0d72f5182f0-maas-api-tls\") pod \"maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj\" (UID: \"64f73236-a562-4e3d-a22d-c0d72f5182f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj"
Aug 05 17:57:34.783352 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:57:34.783288    2550 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-serving-cert: secret "maas-api-e2e-mt-cc16b618-serving-cert" not found
Aug 05 17:57:34.783352 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:57:34.783347    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64f73236-a562-4e3d-a22d-c0d72f5182f0-maas-api-tls podName:64f73236-a562-4e3d-a22d-c0d72f5182f0 nodeName:}" failed. No retries permitted until 2026-08-05 17:57:35.283327908 +0000 UTC m=+2162.264905329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/64f73236-a562-4e3d-a22d-c0d72f5182f0-maas-api-tls") pod "maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj" (UID: "64f73236-a562-4e3d-a22d-c0d72f5182f0") : secret "maas-api-e2e-mt-cc16b618-serving-cert" not found
Aug 05 17:57:34.783760 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:34.783741    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64f73236-a562-4e3d-a22d-c0d72f5182f0-openshift-service-ca\") pod \"maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj\" (UID: \"64f73236-a562-4e3d-a22d-c0d72f5182f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj"
Aug 05 17:57:34.791618 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:34.791594    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdln9\" (UniqueName: \"kubernetes.io/projected/64f73236-a562-4e3d-a22d-c0d72f5182f0-kube-api-access-cdln9\") pod \"maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj\" (UID: \"64f73236-a562-4e3d-a22d-c0d72f5182f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj"
Aug 05 17:57:35.287451 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:35.287423    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64f73236-a562-4e3d-a22d-c0d72f5182f0-maas-api-tls\") pod \"maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj\" (UID: \"64f73236-a562-4e3d-a22d-c0d72f5182f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj"
Aug 05 17:57:35.289754 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:35.289733    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64f73236-a562-4e3d-a22d-c0d72f5182f0-maas-api-tls\") pod \"maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj\" (UID: \"64f73236-a562-4e3d-a22d-c0d72f5182f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj"
Aug 05 17:57:35.317166 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:35.317138    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86"]
Aug 05 17:57:35.320759 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:35.320741    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86"
Aug 05 17:57:35.323105 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:35.323086    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-cc16b618\""
Aug 05 17:57:35.323173 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:35.323103    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-cc16b618-dockercfg-kpr7b\""
Aug 05 17:57:35.331415 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:35.331392    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86"]
Aug 05 17:57:35.346828 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:35.346804    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv"]
Aug 05 17:57:35.350727 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:35.350691    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv"
Aug 05 17:57:35.356798 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:35.356776    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv"]
Aug 05 17:57:35.408521 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:35.408494    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj"
Aug 05 17:57:35.489084 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:35.489054    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85545eca-7966-41aa-ae89-db4bd19feb60-plugins-config-volume\") pod \"payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv\" (UID: \"85545eca-7966-41aa-ae89-db4bd19feb60\") " pod="openshift-ingress/payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv"
Aug 05 17:57:35.489228 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:35.489100    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmpbp\" (UniqueName: \"kubernetes.io/projected/85545eca-7966-41aa-ae89-db4bd19feb60-kube-api-access-kmpbp\") pod \"payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv\" (UID: \"85545eca-7966-41aa-ae89-db4bd19feb60\") " pod="openshift-ingress/payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv"
Aug 05 17:57:35.489228 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:35.489138    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8d9ee98-7202-42b7-86c8-bc8675d7e41b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86\" (UID: \"c8d9ee98-7202-42b7-86c8-bc8675d7e41b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86"
Aug 05 17:57:35.489228 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:35.489163    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4jnw\" (UniqueName: \"kubernetes.io/projected/c8d9ee98-7202-42b7-86c8-bc8675d7e41b-kube-api-access-l4jnw\") pod \"payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86\" (UID: \"c8d9ee98-7202-42b7-86c8-bc8675d7e41b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86"
Aug 05 17:57:35.551934 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:35.551910    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj"]
Aug 05 17:57:35.554728 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:57:35.554678    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64f73236_a562_4e3d_a22d_c0d72f5182f0.slice/crio-9181783784d0d4e5adb02bf16f5e38c88feed9b15f9bbd53b77e2207885af9a9 WatchSource:0}: Error finding container 9181783784d0d4e5adb02bf16f5e38c88feed9b15f9bbd53b77e2207885af9a9: Status 404 returned error can't find the container with id 9181783784d0d4e5adb02bf16f5e38c88feed9b15f9bbd53b77e2207885af9a9
Aug 05 17:57:35.590390 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:35.590365    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4jnw\" (UniqueName: \"kubernetes.io/projected/c8d9ee98-7202-42b7-86c8-bc8675d7e41b-kube-api-access-l4jnw\") pod \"payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86\" (UID: \"c8d9ee98-7202-42b7-86c8-bc8675d7e41b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86"
Aug 05 17:57:35.590499 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:35.590420    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85545eca-7966-41aa-ae89-db4bd19feb60-plugins-config-volume\") pod \"payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv\" (UID: \"85545eca-7966-41aa-ae89-db4bd19feb60\") " pod="openshift-ingress/payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv"
Aug 05 17:57:35.590499 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:35.590445    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmpbp\" (UniqueName: \"kubernetes.io/projected/85545eca-7966-41aa-ae89-db4bd19feb60-kube-api-access-kmpbp\") pod \"payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv\" (UID: \"85545eca-7966-41aa-ae89-db4bd19feb60\") " pod="openshift-ingress/payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv"
Aug 05 17:57:35.590499 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:35.590471    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8d9ee98-7202-42b7-86c8-bc8675d7e41b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86\" (UID: \"c8d9ee98-7202-42b7-86c8-bc8675d7e41b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86"
Aug 05 17:57:35.591110 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:35.591090    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85545eca-7966-41aa-ae89-db4bd19feb60-plugins-config-volume\") pod \"payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv\" (UID: \"85545eca-7966-41aa-ae89-db4bd19feb60\") " pod="openshift-ingress/payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv"
Aug 05 17:57:35.591199 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:35.591183    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8d9ee98-7202-42b7-86c8-bc8675d7e41b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86\" (UID: \"c8d9ee98-7202-42b7-86c8-bc8675d7e41b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86"
Aug 05 17:57:35.599085 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:35.599058    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmpbp\" (UniqueName: \"kubernetes.io/projected/85545eca-7966-41aa-ae89-db4bd19feb60-kube-api-access-kmpbp\") pod \"payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv\" (UID: \"85545eca-7966-41aa-ae89-db4bd19feb60\") " pod="openshift-ingress/payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv"
Aug 05 17:57:35.599266 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:35.599249    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4jnw\" (UniqueName: \"kubernetes.io/projected/c8d9ee98-7202-42b7-86c8-bc8675d7e41b-kube-api-access-l4jnw\") pod \"payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86\" (UID: \"c8d9ee98-7202-42b7-86c8-bc8675d7e41b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86"
Aug 05 17:57:35.631763 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:35.631739    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86"
Aug 05 17:57:35.662006 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:35.661982    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv"
Aug 05 17:57:35.776990 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:35.776959    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86"]
Aug 05 17:57:35.779039 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:57:35.779006    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8d9ee98_7202_42b7_86c8_bc8675d7e41b.slice/crio-8dc1334c7455c8ebe52ce51f61395d8e236f6e0bd7755c1b912acd6a9528e5ee WatchSource:0}: Error finding container 8dc1334c7455c8ebe52ce51f61395d8e236f6e0bd7755c1b912acd6a9528e5ee: Status 404 returned error can't find the container with id 8dc1334c7455c8ebe52ce51f61395d8e236f6e0bd7755c1b912acd6a9528e5ee
Aug 05 17:57:35.807373 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:35.807336    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv"]
Aug 05 17:57:35.809694 ip-10-0-134-46 kubenswrapper[2550]: W0805 17:57:35.809662    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85545eca_7966_41aa_ae89_db4bd19feb60.slice/crio-9ca47ae869ab3fa817a72f06ce5f8f372c71477271be603e901dd11471dd0ff1 WatchSource:0}: Error finding container 9ca47ae869ab3fa817a72f06ce5f8f372c71477271be603e901dd11471dd0ff1: Status 404 returned error can't find the container with id 9ca47ae869ab3fa817a72f06ce5f8f372c71477271be603e901dd11471dd0ff1
Aug 05 17:57:36.386317 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:36.386223    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86" event={"ID":"c8d9ee98-7202-42b7-86c8-bc8675d7e41b","Type":"ContainerStarted","Data":"f946f815244c4dc72eb0d0f39e69b5df9c66dad13246e4f72e51803a530460b1"}
Aug 05 17:57:36.386317 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:36.386283    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86" event={"ID":"c8d9ee98-7202-42b7-86c8-bc8675d7e41b","Type":"ContainerStarted","Data":"8dc1334c7455c8ebe52ce51f61395d8e236f6e0bd7755c1b912acd6a9528e5ee"}
Aug 05 17:57:36.386317 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:36.386314    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86"
Aug 05 17:57:36.387872 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:36.387847    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv" event={"ID":"85545eca-7966-41aa-ae89-db4bd19feb60","Type":"ContainerStarted","Data":"a370c3947bcfbcadc96be11e5e186cbedd49dcae2dd40d6796594dd6b2ff475a"}
Aug 05 17:57:36.394079 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:36.388202    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv"
Aug 05 17:57:36.394079 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:36.388268    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv" event={"ID":"85545eca-7966-41aa-ae89-db4bd19feb60","Type":"ContainerStarted","Data":"9ca47ae869ab3fa817a72f06ce5f8f372c71477271be603e901dd11471dd0ff1"}
Aug 05 17:57:36.395690 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:36.395664    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj" event={"ID":"64f73236-a562-4e3d-a22d-c0d72f5182f0","Type":"ContainerStarted","Data":"b550d221048a30f82a03e67d178df61454640da024fbfb7ce87b25feafedc80d"}
Aug 05 17:57:36.395836 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:36.395749    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj" event={"ID":"64f73236-a562-4e3d-a22d-c0d72f5182f0","Type":"ContainerStarted","Data":"9181783784d0d4e5adb02bf16f5e38c88feed9b15f9bbd53b77e2207885af9a9"}
Aug 05 17:57:36.395836 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:36.395769    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj"
Aug 05 17:57:36.403954 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:36.403914    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86" podStartSLOduration=1.403900782 podStartE2EDuration="1.403900782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:57:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:57:36.403494604 +0000 UTC m=+2163.385072033" watchObservedRunningTime="2026-08-05 17:57:36.403900782 +0000 UTC m=+2163.385478211"
Aug 05 17:57:36.421214 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:36.421174    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj" podStartSLOduration=1.8457853050000002 podStartE2EDuration="2.42116137s" totalImagesPullingTime="575.376065ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:57:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:57:36.419841999 +0000 UTC m=+2163.401419429" watchObservedRunningTime="2026-08-05 17:57:36.42116137 +0000 UTC m=+2163.402738800"
Aug 05 17:57:36.437960 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:36.437920    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv" podStartSLOduration=1.4379069979999999 podStartE2EDuration="1.437906998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:57:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:57:36.43666656 +0000 UTC m=+2163.418243989" watchObservedRunningTime="2026-08-05 17:57:36.437906998 +0000 UTC m=+2163.419484426"
Aug 05 17:57:42.406398 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:42.406366    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj"
Aug 05 17:57:47.403213 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:47.403172    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86"
Aug 05 17:57:47.403640 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:57:47.403312    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv"
Aug 05 17:58:16.105920 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.105881    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj"]
Aug 05 17:58:16.106560 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.106168    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj" podUID="64f73236-a562-4e3d-a22d-c0d72f5182f0" containerName="maas-api" containerID="cri-o://b550d221048a30f82a03e67d178df61454640da024fbfb7ce87b25feafedc80d" gracePeriod=30
Aug 05 17:58:16.162918 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.162880    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv"]
Aug 05 17:58:16.163409 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.163143    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv" podUID="85545eca-7966-41aa-ae89-db4bd19feb60" containerName="payload-processing" containerID="cri-o://a370c3947bcfbcadc96be11e5e186cbedd49dcae2dd40d6796594dd6b2ff475a" gracePeriod=30
Aug 05 17:58:16.170911 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.170880    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86"]
Aug 05 17:58:16.171152 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.171125    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86" podUID="c8d9ee98-7202-42b7-86c8-bc8675d7e41b" containerName="payload-pre-processing" containerID="cri-o://f946f815244c4dc72eb0d0f39e69b5df9c66dad13246e4f72e51803a530460b1" gracePeriod=30
Aug 05 17:58:16.433782 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.433753    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj"
Aug 05 17:58:16.449314 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.449289    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv"
Aug 05 17:58:16.479366 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.479341    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86"
Aug 05 17:58:16.546950 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.546921    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64f73236-a562-4e3d-a22d-c0d72f5182f0-maas-api-tls\") pod \"64f73236-a562-4e3d-a22d-c0d72f5182f0\" (UID: \"64f73236-a562-4e3d-a22d-c0d72f5182f0\") "
Aug 05 17:58:16.547151 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.546986    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmpbp\" (UniqueName: \"kubernetes.io/projected/85545eca-7966-41aa-ae89-db4bd19feb60-kube-api-access-kmpbp\") pod \"85545eca-7966-41aa-ae89-db4bd19feb60\" (UID: \"85545eca-7966-41aa-ae89-db4bd19feb60\") "
Aug 05 17:58:16.547151 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.547025    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64f73236-a562-4e3d-a22d-c0d72f5182f0-openshift-service-ca\") pod \"64f73236-a562-4e3d-a22d-c0d72f5182f0\" (UID: \"64f73236-a562-4e3d-a22d-c0d72f5182f0\") "
Aug 05 17:58:16.547151 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.547041    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85545eca-7966-41aa-ae89-db4bd19feb60-plugins-config-volume\") pod \"85545eca-7966-41aa-ae89-db4bd19feb60\" (UID: \"85545eca-7966-41aa-ae89-db4bd19feb60\") "
Aug 05 17:58:16.547151 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.547082    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdln9\" (UniqueName: \"kubernetes.io/projected/64f73236-a562-4e3d-a22d-c0d72f5182f0-kube-api-access-cdln9\") pod \"64f73236-a562-4e3d-a22d-c0d72f5182f0\" (UID: \"64f73236-a562-4e3d-a22d-c0d72f5182f0\") "
Aug 05 17:58:16.547440 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.547415    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64f73236-a562-4e3d-a22d-c0d72f5182f0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "64f73236-a562-4e3d-a22d-c0d72f5182f0" (UID: "64f73236-a562-4e3d-a22d-c0d72f5182f0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:58:16.547494 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.547442    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85545eca-7966-41aa-ae89-db4bd19feb60-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "85545eca-7966-41aa-ae89-db4bd19feb60" (UID: "85545eca-7966-41aa-ae89-db4bd19feb60"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:58:16.549086 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.549054    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64f73236-a562-4e3d-a22d-c0d72f5182f0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "64f73236-a562-4e3d-a22d-c0d72f5182f0" (UID: "64f73236-a562-4e3d-a22d-c0d72f5182f0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:58:16.549191 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.549134    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85545eca-7966-41aa-ae89-db4bd19feb60-kube-api-access-kmpbp" (OuterVolumeSpecName: "kube-api-access-kmpbp") pod "85545eca-7966-41aa-ae89-db4bd19feb60" (UID: "85545eca-7966-41aa-ae89-db4bd19feb60"). InnerVolumeSpecName "kube-api-access-kmpbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:58:16.549191 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.549161    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64f73236-a562-4e3d-a22d-c0d72f5182f0-kube-api-access-cdln9" (OuterVolumeSpecName: "kube-api-access-cdln9") pod "64f73236-a562-4e3d-a22d-c0d72f5182f0" (UID: "64f73236-a562-4e3d-a22d-c0d72f5182f0"). InnerVolumeSpecName "kube-api-access-cdln9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:58:16.587151 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.587115    2550 generic.go:356] "Generic (PLEG): container finished" podID="c8d9ee98-7202-42b7-86c8-bc8675d7e41b" containerID="f946f815244c4dc72eb0d0f39e69b5df9c66dad13246e4f72e51803a530460b1" exitCode=0
Aug 05 17:58:16.587336 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.587162    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86"
Aug 05 17:58:16.587336 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.587208    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86" event={"ID":"c8d9ee98-7202-42b7-86c8-bc8675d7e41b","Type":"ContainerDied","Data":"f946f815244c4dc72eb0d0f39e69b5df9c66dad13246e4f72e51803a530460b1"}
Aug 05 17:58:16.587336 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.587268    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86" event={"ID":"c8d9ee98-7202-42b7-86c8-bc8675d7e41b","Type":"ContainerDied","Data":"8dc1334c7455c8ebe52ce51f61395d8e236f6e0bd7755c1b912acd6a9528e5ee"}
Aug 05 17:58:16.587336 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.587299    2550 scope.go:117] "RemoveContainer" containerID="f946f815244c4dc72eb0d0f39e69b5df9c66dad13246e4f72e51803a530460b1"
Aug 05 17:58:16.588422 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.588378    2550 generic.go:356] "Generic (PLEG): container finished" podID="85545eca-7966-41aa-ae89-db4bd19feb60" containerID="a370c3947bcfbcadc96be11e5e186cbedd49dcae2dd40d6796594dd6b2ff475a" exitCode=0
Aug 05 17:58:16.588492 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.588428    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv"
Aug 05 17:58:16.588492 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.588465    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv" event={"ID":"85545eca-7966-41aa-ae89-db4bd19feb60","Type":"ContainerDied","Data":"a370c3947bcfbcadc96be11e5e186cbedd49dcae2dd40d6796594dd6b2ff475a"}
Aug 05 17:58:16.588597 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.588517    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv" event={"ID":"85545eca-7966-41aa-ae89-db4bd19feb60","Type":"ContainerDied","Data":"9ca47ae869ab3fa817a72f06ce5f8f372c71477271be603e901dd11471dd0ff1"}
Aug 05 17:58:16.589833 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.589813    2550 generic.go:356] "Generic (PLEG): container finished" podID="64f73236-a562-4e3d-a22d-c0d72f5182f0" containerID="b550d221048a30f82a03e67d178df61454640da024fbfb7ce87b25feafedc80d" exitCode=0
Aug 05 17:58:16.589936 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.589862    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj"
Aug 05 17:58:16.589936 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.589903    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj" event={"ID":"64f73236-a562-4e3d-a22d-c0d72f5182f0","Type":"ContainerDied","Data":"b550d221048a30f82a03e67d178df61454640da024fbfb7ce87b25feafedc80d"}
Aug 05 17:58:16.590032 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.589947    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj" event={"ID":"64f73236-a562-4e3d-a22d-c0d72f5182f0","Type":"ContainerDied","Data":"9181783784d0d4e5adb02bf16f5e38c88feed9b15f9bbd53b77e2207885af9a9"}
Aug 05 17:58:16.597783 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.597734    2550 scope.go:117] "RemoveContainer" containerID="f946f815244c4dc72eb0d0f39e69b5df9c66dad13246e4f72e51803a530460b1"
Aug 05 17:58:16.598064 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:58:16.598041    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f946f815244c4dc72eb0d0f39e69b5df9c66dad13246e4f72e51803a530460b1\": container with ID starting with f946f815244c4dc72eb0d0f39e69b5df9c66dad13246e4f72e51803a530460b1 not found: ID does not exist" containerID="f946f815244c4dc72eb0d0f39e69b5df9c66dad13246e4f72e51803a530460b1"
Aug 05 17:58:16.598158 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.598074    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f946f815244c4dc72eb0d0f39e69b5df9c66dad13246e4f72e51803a530460b1"} err="failed to get container status \"f946f815244c4dc72eb0d0f39e69b5df9c66dad13246e4f72e51803a530460b1\": rpc error: code = NotFound desc = could not find container \"f946f815244c4dc72eb0d0f39e69b5df9c66dad13246e4f72e51803a530460b1\": container with ID starting with f946f815244c4dc72eb0d0f39e69b5df9c66dad13246e4f72e51803a530460b1 not found: ID does not exist"
Aug 05 17:58:16.598158 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.598096    2550 scope.go:117] "RemoveContainer" containerID="a370c3947bcfbcadc96be11e5e186cbedd49dcae2dd40d6796594dd6b2ff475a"
Aug 05 17:58:16.606076 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.606061    2550 scope.go:117] "RemoveContainer" containerID="a370c3947bcfbcadc96be11e5e186cbedd49dcae2dd40d6796594dd6b2ff475a"
Aug 05 17:58:16.606276 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:58:16.606258    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a370c3947bcfbcadc96be11e5e186cbedd49dcae2dd40d6796594dd6b2ff475a\": container with ID starting with a370c3947bcfbcadc96be11e5e186cbedd49dcae2dd40d6796594dd6b2ff475a not found: ID does not exist" containerID="a370c3947bcfbcadc96be11e5e186cbedd49dcae2dd40d6796594dd6b2ff475a"
Aug 05 17:58:16.606336 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.606280    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a370c3947bcfbcadc96be11e5e186cbedd49dcae2dd40d6796594dd6b2ff475a"} err="failed to get container status \"a370c3947bcfbcadc96be11e5e186cbedd49dcae2dd40d6796594dd6b2ff475a\": rpc error: code = NotFound desc = could not find container \"a370c3947bcfbcadc96be11e5e186cbedd49dcae2dd40d6796594dd6b2ff475a\": container with ID starting with a370c3947bcfbcadc96be11e5e186cbedd49dcae2dd40d6796594dd6b2ff475a not found: ID does not exist"
Aug 05 17:58:16.606336 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.606295    2550 scope.go:117] "RemoveContainer" containerID="b550d221048a30f82a03e67d178df61454640da024fbfb7ce87b25feafedc80d"
Aug 05 17:58:16.614812 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.614788    2550 scope.go:117] "RemoveContainer" containerID="b550d221048a30f82a03e67d178df61454640da024fbfb7ce87b25feafedc80d"
Aug 05 17:58:16.615094 ip-10-0-134-46 kubenswrapper[2550]: E0805 17:58:16.615075    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b550d221048a30f82a03e67d178df61454640da024fbfb7ce87b25feafedc80d\": container with ID starting with b550d221048a30f82a03e67d178df61454640da024fbfb7ce87b25feafedc80d not found: ID does not exist" containerID="b550d221048a30f82a03e67d178df61454640da024fbfb7ce87b25feafedc80d"
Aug 05 17:58:16.615163 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.615103    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b550d221048a30f82a03e67d178df61454640da024fbfb7ce87b25feafedc80d"} err="failed to get container status \"b550d221048a30f82a03e67d178df61454640da024fbfb7ce87b25feafedc80d\": rpc error: code = NotFound desc = could not find container \"b550d221048a30f82a03e67d178df61454640da024fbfb7ce87b25feafedc80d\": container with ID starting with b550d221048a30f82a03e67d178df61454640da024fbfb7ce87b25feafedc80d not found: ID does not exist"
Aug 05 17:58:16.632850 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.632827    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv"]
Aug 05 17:58:16.642128 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.642107    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-cc16b618-759cfd857d-tfbcv"]
Aug 05 17:58:16.648434 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.648418    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8d9ee98-7202-42b7-86c8-bc8675d7e41b-plugins-config-volume\") pod \"c8d9ee98-7202-42b7-86c8-bc8675d7e41b\" (UID: \"c8d9ee98-7202-42b7-86c8-bc8675d7e41b\") "
Aug 05 17:58:16.648504 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.648489    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4jnw\" (UniqueName: \"kubernetes.io/projected/c8d9ee98-7202-42b7-86c8-bc8675d7e41b-kube-api-access-l4jnw\") pod \"c8d9ee98-7202-42b7-86c8-bc8675d7e41b\" (UID: \"c8d9ee98-7202-42b7-86c8-bc8675d7e41b\") "
Aug 05 17:58:16.648650 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.648640    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmpbp\" (UniqueName: \"kubernetes.io/projected/85545eca-7966-41aa-ae89-db4bd19feb60-kube-api-access-kmpbp\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:58:16.648715 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.648653    2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64f73236-a562-4e3d-a22d-c0d72f5182f0-openshift-service-ca\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:58:16.648715 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.648663    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85545eca-7966-41aa-ae89-db4bd19feb60-plugins-config-volume\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:58:16.648715 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.648672    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdln9\" (UniqueName: \"kubernetes.io/projected/64f73236-a562-4e3d-a22d-c0d72f5182f0-kube-api-access-cdln9\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:58:16.648715 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.648681    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64f73236-a562-4e3d-a22d-c0d72f5182f0-maas-api-tls\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:58:16.648975 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.648789    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8d9ee98-7202-42b7-86c8-bc8675d7e41b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c8d9ee98-7202-42b7-86c8-bc8675d7e41b" (UID: "c8d9ee98-7202-42b7-86c8-bc8675d7e41b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:58:16.650500 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.650481    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8d9ee98-7202-42b7-86c8-bc8675d7e41b-kube-api-access-l4jnw" (OuterVolumeSpecName: "kube-api-access-l4jnw") pod "c8d9ee98-7202-42b7-86c8-bc8675d7e41b" (UID: "c8d9ee98-7202-42b7-86c8-bc8675d7e41b"). InnerVolumeSpecName "kube-api-access-l4jnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:58:16.672177 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.672150    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj"]
Aug 05 17:58:16.676563 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.676543    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cc16b618-5bb57594c9-wz6cj"]
Aug 05 17:58:16.749470 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.749438    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8d9ee98-7202-42b7-86c8-bc8675d7e41b-plugins-config-volume\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:58:16.749470 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.749467    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4jnw\" (UniqueName: \"kubernetes.io/projected/c8d9ee98-7202-42b7-86c8-bc8675d7e41b-kube-api-access-l4jnw\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 17:58:16.910398 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.910360    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86"]
Aug 05 17:58:16.913553 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:16.913522    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-cc16b618-58b76d755b-snt86"]
Aug 05 17:58:17.700624 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:17.700590    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64f73236-a562-4e3d-a22d-c0d72f5182f0" path="/var/lib/kubelet/pods/64f73236-a562-4e3d-a22d-c0d72f5182f0/volumes"
Aug 05 17:58:17.701028 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:17.700973    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85545eca-7966-41aa-ae89-db4bd19feb60" path="/var/lib/kubelet/pods/85545eca-7966-41aa-ae89-db4bd19feb60/volumes"
Aug 05 17:58:17.701277 ip-10-0-134-46 kubenswrapper[2550]: I0805 17:58:17.701264    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8d9ee98-7202-42b7-86c8-bc8675d7e41b" path="/var/lib/kubelet/pods/c8d9ee98-7202-42b7-86c8-bc8675d7e41b/volumes"
Aug 05 18:00:43.553333 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:43.553292    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x"]
Aug 05 18:00:43.553891 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:43.553670    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64f73236-a562-4e3d-a22d-c0d72f5182f0" containerName="maas-api"
Aug 05 18:00:43.553891 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:43.553687    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="64f73236-a562-4e3d-a22d-c0d72f5182f0" containerName="maas-api"
Aug 05 18:00:43.553891 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:43.553724    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85545eca-7966-41aa-ae89-db4bd19feb60" containerName="payload-processing"
Aug 05 18:00:43.553891 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:43.553734    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="85545eca-7966-41aa-ae89-db4bd19feb60" containerName="payload-processing"
Aug 05 18:00:43.553891 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:43.553756    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8d9ee98-7202-42b7-86c8-bc8675d7e41b" containerName="payload-pre-processing"
Aug 05 18:00:43.553891 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:43.553765    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8d9ee98-7202-42b7-86c8-bc8675d7e41b" containerName="payload-pre-processing"
Aug 05 18:00:43.553891 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:43.553855    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="64f73236-a562-4e3d-a22d-c0d72f5182f0" containerName="maas-api"
Aug 05 18:00:43.553891 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:43.553865    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8d9ee98-7202-42b7-86c8-bc8675d7e41b" containerName="payload-pre-processing"
Aug 05 18:00:43.553891 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:43.553872    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="85545eca-7966-41aa-ae89-db4bd19feb60" containerName="payload-processing"
Aug 05 18:00:43.556875 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:43.556847    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x"
Aug 05 18:00:43.559191 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:43.559167    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-75d3fbe4-serving-cert\""
Aug 05 18:00:43.573472 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:43.573448    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x"]
Aug 05 18:00:43.685131 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:43.685080    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2478b3ab-57b1-4f70-ae65-ef864e317e63-maas-api-tls\") pod \"maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x\" (UID: \"2478b3ab-57b1-4f70-ae65-ef864e317e63\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x"
Aug 05 18:00:43.685335 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:43.685210    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2478b3ab-57b1-4f70-ae65-ef864e317e63-openshift-service-ca\") pod \"maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x\" (UID: \"2478b3ab-57b1-4f70-ae65-ef864e317e63\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x"
Aug 05 18:00:43.685335 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:43.685323    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22bds\" (UniqueName: \"kubernetes.io/projected/2478b3ab-57b1-4f70-ae65-ef864e317e63-kube-api-access-22bds\") pod \"maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x\" (UID: \"2478b3ab-57b1-4f70-ae65-ef864e317e63\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x"
Aug 05 18:00:43.786341 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:43.786303    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22bds\" (UniqueName: \"kubernetes.io/projected/2478b3ab-57b1-4f70-ae65-ef864e317e63-kube-api-access-22bds\") pod \"maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x\" (UID: \"2478b3ab-57b1-4f70-ae65-ef864e317e63\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x"
Aug 05 18:00:43.786529 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:43.786355    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2478b3ab-57b1-4f70-ae65-ef864e317e63-maas-api-tls\") pod \"maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x\" (UID: \"2478b3ab-57b1-4f70-ae65-ef864e317e63\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x"
Aug 05 18:00:43.786529 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:43.786389    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2478b3ab-57b1-4f70-ae65-ef864e317e63-openshift-service-ca\") pod \"maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x\" (UID: \"2478b3ab-57b1-4f70-ae65-ef864e317e63\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x"
Aug 05 18:00:43.786622 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:00:43.786520    2550 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-serving-cert: secret "maas-api-e2e-mt-75d3fbe4-serving-cert" not found
Aug 05 18:00:43.786622 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:00:43.786597    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2478b3ab-57b1-4f70-ae65-ef864e317e63-maas-api-tls podName:2478b3ab-57b1-4f70-ae65-ef864e317e63 nodeName:}" failed. No retries permitted until 2026-08-05 18:00:44.286576494 +0000 UTC m=+2351.268153914 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2478b3ab-57b1-4f70-ae65-ef864e317e63-maas-api-tls") pod "maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x" (UID: "2478b3ab-57b1-4f70-ae65-ef864e317e63") : secret "maas-api-e2e-mt-75d3fbe4-serving-cert" not found
Aug 05 18:00:43.787013 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:43.786995    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2478b3ab-57b1-4f70-ae65-ef864e317e63-openshift-service-ca\") pod \"maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x\" (UID: \"2478b3ab-57b1-4f70-ae65-ef864e317e63\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x"
Aug 05 18:00:43.797672 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:43.797650    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22bds\" (UniqueName: \"kubernetes.io/projected/2478b3ab-57b1-4f70-ae65-ef864e317e63-kube-api-access-22bds\") pod \"maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x\" (UID: \"2478b3ab-57b1-4f70-ae65-ef864e317e63\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x"
Aug 05 18:00:44.224126 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.224095    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx"]
Aug 05 18:00:44.227520 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.227506    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx"
Aug 05 18:00:44.229909 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.229894    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-75d3fbe4-dockercfg-986mv\""
Aug 05 18:00:44.229968 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.229936    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-75d3fbe4\""
Aug 05 18:00:44.238888 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.238863    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx"]
Aug 05 18:00:44.270306 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.270268    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2"]
Aug 05 18:00:44.273776 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.273757    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2"
Aug 05 18:00:44.284922 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.284893    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2"]
Aug 05 18:00:44.291468 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.291433    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2478b3ab-57b1-4f70-ae65-ef864e317e63-maas-api-tls\") pod \"maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x\" (UID: \"2478b3ab-57b1-4f70-ae65-ef864e317e63\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x"
Aug 05 18:00:44.294371 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.294339    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2478b3ab-57b1-4f70-ae65-ef864e317e63-maas-api-tls\") pod \"maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x\" (UID: \"2478b3ab-57b1-4f70-ae65-ef864e317e63\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x"
Aug 05 18:00:44.392929 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.392887    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9d9f\" (UniqueName: \"kubernetes.io/projected/b84e0fae-f53e-441a-8aa9-595b69d8a37e-kube-api-access-m9d9f\") pod \"payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx\" (UID: \"b84e0fae-f53e-441a-8aa9-595b69d8a37e\") " pod="openshift-ingress/payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx"
Aug 05 18:00:44.392929 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.392930    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8d4p\" (UniqueName: \"kubernetes.io/projected/0aa8c42e-c04c-4465-8139-f656d09cdded-kube-api-access-j8d4p\") pod \"payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2\" (UID: \"0aa8c42e-c04c-4465-8139-f656d09cdded\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2"
Aug 05 18:00:44.393145 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.393022    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0aa8c42e-c04c-4465-8139-f656d09cdded-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2\" (UID: \"0aa8c42e-c04c-4465-8139-f656d09cdded\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2"
Aug 05 18:00:44.393145 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.393069    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b84e0fae-f53e-441a-8aa9-595b69d8a37e-plugins-config-volume\") pod \"payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx\" (UID: \"b84e0fae-f53e-441a-8aa9-595b69d8a37e\") " pod="openshift-ingress/payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx"
Aug 05 18:00:44.467831 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.467794    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x"
Aug 05 18:00:44.494097 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.494067    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0aa8c42e-c04c-4465-8139-f656d09cdded-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2\" (UID: \"0aa8c42e-c04c-4465-8139-f656d09cdded\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2"
Aug 05 18:00:44.494239 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.494103    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b84e0fae-f53e-441a-8aa9-595b69d8a37e-plugins-config-volume\") pod \"payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx\" (UID: \"b84e0fae-f53e-441a-8aa9-595b69d8a37e\") " pod="openshift-ingress/payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx"
Aug 05 18:00:44.494239 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.494137    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9d9f\" (UniqueName: \"kubernetes.io/projected/b84e0fae-f53e-441a-8aa9-595b69d8a37e-kube-api-access-m9d9f\") pod \"payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx\" (UID: \"b84e0fae-f53e-441a-8aa9-595b69d8a37e\") " pod="openshift-ingress/payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx"
Aug 05 18:00:44.494239 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.494166    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8d4p\" (UniqueName: \"kubernetes.io/projected/0aa8c42e-c04c-4465-8139-f656d09cdded-kube-api-access-j8d4p\") pod \"payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2\" (UID: \"0aa8c42e-c04c-4465-8139-f656d09cdded\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2"
Aug 05 18:00:44.494863 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.494843    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0aa8c42e-c04c-4465-8139-f656d09cdded-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2\" (UID: \"0aa8c42e-c04c-4465-8139-f656d09cdded\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2"
Aug 05 18:00:44.494910 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.494873    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b84e0fae-f53e-441a-8aa9-595b69d8a37e-plugins-config-volume\") pod \"payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx\" (UID: \"b84e0fae-f53e-441a-8aa9-595b69d8a37e\") " pod="openshift-ingress/payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx"
Aug 05 18:00:44.503359 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.503335    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8d4p\" (UniqueName: \"kubernetes.io/projected/0aa8c42e-c04c-4465-8139-f656d09cdded-kube-api-access-j8d4p\") pod \"payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2\" (UID: \"0aa8c42e-c04c-4465-8139-f656d09cdded\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2"
Aug 05 18:00:44.503359 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.503346    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9d9f\" (UniqueName: \"kubernetes.io/projected/b84e0fae-f53e-441a-8aa9-595b69d8a37e-kube-api-access-m9d9f\") pod \"payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx\" (UID: \"b84e0fae-f53e-441a-8aa9-595b69d8a37e\") " pod="openshift-ingress/payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx"
Aug 05 18:00:44.538379 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.538315    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx"
Aug 05 18:00:44.586333 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.586226    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2"
Aug 05 18:00:44.613899 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.613820    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x"]
Aug 05 18:00:44.618641 ip-10-0-134-46 kubenswrapper[2550]: W0805 18:00:44.618616    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2478b3ab_57b1_4f70_ae65_ef864e317e63.slice/crio-d2596fdde32d675f435225ddd6caf0dff4cb410bcb800456dfef4abdbd0af544 WatchSource:0}: Error finding container d2596fdde32d675f435225ddd6caf0dff4cb410bcb800456dfef4abdbd0af544: Status 404 returned error can't find the container with id d2596fdde32d675f435225ddd6caf0dff4cb410bcb800456dfef4abdbd0af544
Aug 05 18:00:44.620578 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.620473    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 18:00:44.696836 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.696808    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx"]
Aug 05 18:00:44.699644 ip-10-0-134-46 kubenswrapper[2550]: W0805 18:00:44.699614    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb84e0fae_f53e_441a_8aa9_595b69d8a37e.slice/crio-7687060647a9453434d40df1c3b54bc408ec6345761b769f7ab9cce6329340b0 WatchSource:0}: Error finding container 7687060647a9453434d40df1c3b54bc408ec6345761b769f7ab9cce6329340b0: Status 404 returned error can't find the container with id 7687060647a9453434d40df1c3b54bc408ec6345761b769f7ab9cce6329340b0
Aug 05 18:00:44.742603 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:44.742576    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2"]
Aug 05 18:00:44.753772 ip-10-0-134-46 kubenswrapper[2550]: W0805 18:00:44.753749    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aa8c42e_c04c_4465_8139_f656d09cdded.slice/crio-deb4844174f1bdc21233b03673016e7f7e74947289070a5327a27494582974d1 WatchSource:0}: Error finding container deb4844174f1bdc21233b03673016e7f7e74947289070a5327a27494582974d1: Status 404 returned error can't find the container with id deb4844174f1bdc21233b03673016e7f7e74947289070a5327a27494582974d1
Aug 05 18:00:45.299618 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:45.299526    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx" event={"ID":"b84e0fae-f53e-441a-8aa9-595b69d8a37e","Type":"ContainerStarted","Data":"bf11e39cba982b371997f030c22ee69c49f23fd146f79d2af88787e8e757bc53"}
Aug 05 18:00:45.299618 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:45.299580    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx" event={"ID":"b84e0fae-f53e-441a-8aa9-595b69d8a37e","Type":"ContainerStarted","Data":"7687060647a9453434d40df1c3b54bc408ec6345761b769f7ab9cce6329340b0"}
Aug 05 18:00:45.299888 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:45.299657    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx"
Aug 05 18:00:45.301223 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:45.301200    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2" event={"ID":"0aa8c42e-c04c-4465-8139-f656d09cdded","Type":"ContainerStarted","Data":"4616e94699e2709a39089b3a48949379a187f1af9036051d1819bb8cdd468fad"}
Aug 05 18:00:45.301355 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:45.301237    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2" event={"ID":"0aa8c42e-c04c-4465-8139-f656d09cdded","Type":"ContainerStarted","Data":"deb4844174f1bdc21233b03673016e7f7e74947289070a5327a27494582974d1"}
Aug 05 18:00:45.301355 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:45.301264    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2"
Aug 05 18:00:45.302629 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:45.302603    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x" event={"ID":"2478b3ab-57b1-4f70-ae65-ef864e317e63","Type":"ContainerStarted","Data":"a12ddc0784277b173b527347e34f3200dfc3867a03bc01a5bc2d9426f5ab104f"}
Aug 05 18:00:45.302760 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:45.302650    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x" event={"ID":"2478b3ab-57b1-4f70-ae65-ef864e317e63","Type":"ContainerStarted","Data":"d2596fdde32d675f435225ddd6caf0dff4cb410bcb800456dfef4abdbd0af544"}
Aug 05 18:00:45.302760 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:45.302736    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x"
Aug 05 18:00:45.316227 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:45.316184    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx" podStartSLOduration=1.31616662 podStartE2EDuration="1.31616662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:00:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:00:45.314977531 +0000 UTC m=+2352.296554975" watchObservedRunningTime="2026-08-05 18:00:45.31616662 +0000 UTC m=+2352.297744050"
Aug 05 18:00:45.333969 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:45.333919    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2" podStartSLOduration=1.333899091 podStartE2EDuration="1.333899091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:00:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:00:45.332692819 +0000 UTC m=+2352.314270249" watchObservedRunningTime="2026-08-05 18:00:45.333899091 +0000 UTC m=+2352.315476525"
Aug 05 18:00:45.349941 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:45.349895    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x" podStartSLOduration=1.947904357 podStartE2EDuration="2.349877632s" totalImagesPullingTime="401.973275ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:00:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:00:45.348334903 +0000 UTC m=+2352.329912332" watchObservedRunningTime="2026-08-05 18:00:45.349877632 +0000 UTC m=+2352.331455061"
Aug 05 18:00:51.313244 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:51.313216    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x"
Aug 05 18:00:56.309562 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:56.309530    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2"
Aug 05 18:00:56.310004 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:00:56.309614    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx"
Aug 05 18:01:14.972147 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:14.972114    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"]
Aug 05 18:01:14.975546 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:14.975528    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"
Aug 05 18:01:14.978839 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:14.978814    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-75d3fbe4\"/\"default-dockercfg-2gkqf\""
Aug 05 18:01:14.978955 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:14.978844    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-75d3fbe4\"/\"openshift-service-ca.crt\""
Aug 05 18:01:14.978955 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:14.978945    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-75d3fbe4\"/\"kube-root-ca.crt\""
Aug 05 18:01:14.979113 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:14.979090    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-75d3fbe4\"/\"e2e-lifecycle-model-75d3fbe4-kserve-self-signed-certs\""
Aug 05 18:01:15.000310 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:15.000288    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"]
Aug 05 18:01:15.048062 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:15.048032    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6f7ef06-1583-44df-8da6-579a29e18f86-home\") pod \"e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8\" (UID: \"d6f7ef06-1583-44df-8da6-579a29e18f86\") " pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"
Aug 05 18:01:15.048062 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:15.048065    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6f7ef06-1583-44df-8da6-579a29e18f86-tls-certs\") pod \"e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8\" (UID: \"d6f7ef06-1583-44df-8da6-579a29e18f86\") " pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"
Aug 05 18:01:15.048244 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:15.048096    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6f7ef06-1583-44df-8da6-579a29e18f86-dshm\") pod \"e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8\" (UID: \"d6f7ef06-1583-44df-8da6-579a29e18f86\") " pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"
Aug 05 18:01:15.048244 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:15.048156    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6f7ef06-1583-44df-8da6-579a29e18f86-tmp-dir\") pod \"e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8\" (UID: \"d6f7ef06-1583-44df-8da6-579a29e18f86\") " pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"
Aug 05 18:01:15.048244 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:15.048185    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6f7ef06-1583-44df-8da6-579a29e18f86-model-cache\") pod \"e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8\" (UID: \"d6f7ef06-1583-44df-8da6-579a29e18f86\") " pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"
Aug 05 18:01:15.048244 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:15.048208    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw87h\" (UniqueName: \"kubernetes.io/projected/d6f7ef06-1583-44df-8da6-579a29e18f86-kube-api-access-pw87h\") pod \"e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8\" (UID: \"d6f7ef06-1583-44df-8da6-579a29e18f86\") " pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"
Aug 05 18:01:15.148787 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:15.148754    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6f7ef06-1583-44df-8da6-579a29e18f86-dshm\") pod \"e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8\" (UID: \"d6f7ef06-1583-44df-8da6-579a29e18f86\") " pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"
Aug 05 18:01:15.148977 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:15.148798    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6f7ef06-1583-44df-8da6-579a29e18f86-tmp-dir\") pod \"e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8\" (UID: \"d6f7ef06-1583-44df-8da6-579a29e18f86\") " pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"
Aug 05 18:01:15.148977 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:15.148829    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6f7ef06-1583-44df-8da6-579a29e18f86-model-cache\") pod \"e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8\" (UID: \"d6f7ef06-1583-44df-8da6-579a29e18f86\") " pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"
Aug 05 18:01:15.148977 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:15.148928    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw87h\" (UniqueName: \"kubernetes.io/projected/d6f7ef06-1583-44df-8da6-579a29e18f86-kube-api-access-pw87h\") pod \"e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8\" (UID: \"d6f7ef06-1583-44df-8da6-579a29e18f86\") " pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"
Aug 05 18:01:15.149153 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:15.149064    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6f7ef06-1583-44df-8da6-579a29e18f86-home\") pod \"e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8\" (UID: \"d6f7ef06-1583-44df-8da6-579a29e18f86\") " pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"
Aug 05 18:01:15.149153 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:15.149098    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6f7ef06-1583-44df-8da6-579a29e18f86-tls-certs\") pod \"e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8\" (UID: \"d6f7ef06-1583-44df-8da6-579a29e18f86\") " pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"
Aug 05 18:01:15.149262 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:15.149241    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6f7ef06-1583-44df-8da6-579a29e18f86-model-cache\") pod \"e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8\" (UID: \"d6f7ef06-1583-44df-8da6-579a29e18f86\") " pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"
Aug 05 18:01:15.149308 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:15.149266    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6f7ef06-1583-44df-8da6-579a29e18f86-tmp-dir\") pod \"e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8\" (UID: \"d6f7ef06-1583-44df-8da6-579a29e18f86\") " pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"
Aug 05 18:01:15.149429 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:15.149408    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6f7ef06-1583-44df-8da6-579a29e18f86-home\") pod \"e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8\" (UID: \"d6f7ef06-1583-44df-8da6-579a29e18f86\") " pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"
Aug 05 18:01:15.151052 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:15.151033    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6f7ef06-1583-44df-8da6-579a29e18f86-dshm\") pod \"e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8\" (UID: \"d6f7ef06-1583-44df-8da6-579a29e18f86\") " pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"
Aug 05 18:01:15.151449 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:15.151434    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6f7ef06-1583-44df-8da6-579a29e18f86-tls-certs\") pod \"e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8\" (UID: \"d6f7ef06-1583-44df-8da6-579a29e18f86\") " pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"
Aug 05 18:01:15.162828 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:15.162803    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw87h\" (UniqueName: \"kubernetes.io/projected/d6f7ef06-1583-44df-8da6-579a29e18f86-kube-api-access-pw87h\") pod \"e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8\" (UID: \"d6f7ef06-1583-44df-8da6-579a29e18f86\") " pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"
Aug 05 18:01:15.286747 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:15.286631    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"
Aug 05 18:01:15.417904 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:15.417880    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"]
Aug 05 18:01:15.419496 ip-10-0-134-46 kubenswrapper[2550]: W0805 18:01:15.419469    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd6f7ef06_1583_44df_8da6_579a29e18f86.slice/crio-32cbf6de9e60ee0d76615092ccc305985ce31fbc0d248500ca84e1df7fe80e54 WatchSource:0}: Error finding container 32cbf6de9e60ee0d76615092ccc305985ce31fbc0d248500ca84e1df7fe80e54: Status 404 returned error can't find the container with id 32cbf6de9e60ee0d76615092ccc305985ce31fbc0d248500ca84e1df7fe80e54
Aug 05 18:01:15.449088 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:15.449063    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8" event={"ID":"d6f7ef06-1583-44df-8da6-579a29e18f86","Type":"ContainerStarted","Data":"32cbf6de9e60ee0d76615092ccc305985ce31fbc0d248500ca84e1df7fe80e54"}
Aug 05 18:01:16.455426 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:16.455388    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8" event={"ID":"d6f7ef06-1583-44df-8da6-579a29e18f86","Type":"ContainerStarted","Data":"a367c370145d2dda2e80e279c0ca4b12a92927fa7397e0ee91d5841167233b89"}
Aug 05 18:01:16.476330 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:16.476279    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8" podStartSLOduration=2.476264218 podStartE2EDuration="2.476264218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:01:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:01:16.475297397 +0000 UTC m=+2383.456874826" watchObservedRunningTime="2026-08-05 18:01:16.476264218 +0000 UTC m=+2383.457841647"
Aug 05 18:01:25.287175 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:25.287133    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"
Aug 05 18:01:25.287566 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:25.287190    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"
Aug 05 18:01:25.300164 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:25.300133    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"
Aug 05 18:01:25.511211 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:25.511183    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"
Aug 05 18:01:33.646353 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:33.646327    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 18:01:33.783759 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:33.783733    2550 scope.go:117] "RemoveContainer" containerID="0d6e8f0ef0868dc498cfb0fdae7a2c0b268834f351e66d5b52ee22dde130928f"
Aug 05 18:01:33.787690 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:01:33.787671    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 18:02:05.961519 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:05.961436    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x"]
Aug 05 18:02:05.962069 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:05.961681    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x" podUID="2478b3ab-57b1-4f70-ae65-ef864e317e63" containerName="maas-api" containerID="cri-o://a12ddc0784277b173b527347e34f3200dfc3867a03bc01a5bc2d9426f5ab104f" gracePeriod=30
Aug 05 18:02:06.021084 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.021055    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx"]
Aug 05 18:02:06.021375 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.021325    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx" podUID="b84e0fae-f53e-441a-8aa9-595b69d8a37e" containerName="payload-processing" containerID="cri-o://bf11e39cba982b371997f030c22ee69c49f23fd146f79d2af88787e8e757bc53" gracePeriod=30
Aug 05 18:02:06.029078 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.029048    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2"]
Aug 05 18:02:06.029321 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.029291    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2" podUID="0aa8c42e-c04c-4465-8139-f656d09cdded" containerName="payload-pre-processing" containerID="cri-o://4616e94699e2709a39089b3a48949379a187f1af9036051d1819bb8cdd468fad" gracePeriod=30
Aug 05 18:02:06.292663 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.292637    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x"
Aug 05 18:02:06.305578 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.305556    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx"
Aug 05 18:02:06.316079 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.316056    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2"
Aug 05 18:02:06.387013 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.386984    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0aa8c42e-c04c-4465-8139-f656d09cdded-plugins-config-volume\") pod \"0aa8c42e-c04c-4465-8139-f656d09cdded\" (UID: \"0aa8c42e-c04c-4465-8139-f656d09cdded\") "
Aug 05 18:02:06.387147 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.387032    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9d9f\" (UniqueName: \"kubernetes.io/projected/b84e0fae-f53e-441a-8aa9-595b69d8a37e-kube-api-access-m9d9f\") pod \"b84e0fae-f53e-441a-8aa9-595b69d8a37e\" (UID: \"b84e0fae-f53e-441a-8aa9-595b69d8a37e\") "
Aug 05 18:02:06.387147 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.387065    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8d4p\" (UniqueName: \"kubernetes.io/projected/0aa8c42e-c04c-4465-8139-f656d09cdded-kube-api-access-j8d4p\") pod \"0aa8c42e-c04c-4465-8139-f656d09cdded\" (UID: \"0aa8c42e-c04c-4465-8139-f656d09cdded\") "
Aug 05 18:02:06.387147 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.387140    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b84e0fae-f53e-441a-8aa9-595b69d8a37e-plugins-config-volume\") pod \"b84e0fae-f53e-441a-8aa9-595b69d8a37e\" (UID: \"b84e0fae-f53e-441a-8aa9-595b69d8a37e\") "
Aug 05 18:02:06.387297 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.387168    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22bds\" (UniqueName: \"kubernetes.io/projected/2478b3ab-57b1-4f70-ae65-ef864e317e63-kube-api-access-22bds\") pod \"2478b3ab-57b1-4f70-ae65-ef864e317e63\" (UID: \"2478b3ab-57b1-4f70-ae65-ef864e317e63\") "
Aug 05 18:02:06.387297 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.387200    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2478b3ab-57b1-4f70-ae65-ef864e317e63-openshift-service-ca\") pod \"2478b3ab-57b1-4f70-ae65-ef864e317e63\" (UID: \"2478b3ab-57b1-4f70-ae65-ef864e317e63\") "
Aug 05 18:02:06.387679 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.387567    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2478b3ab-57b1-4f70-ae65-ef864e317e63-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2478b3ab-57b1-4f70-ae65-ef864e317e63" (UID: "2478b3ab-57b1-4f70-ae65-ef864e317e63"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:02:06.387679 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.387582    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b84e0fae-f53e-441a-8aa9-595b69d8a37e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b84e0fae-f53e-441a-8aa9-595b69d8a37e" (UID: "b84e0fae-f53e-441a-8aa9-595b69d8a37e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:02:06.387679 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.387565    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2478b3ab-57b1-4f70-ae65-ef864e317e63-maas-api-tls\") pod \"2478b3ab-57b1-4f70-ae65-ef864e317e63\" (UID: \"2478b3ab-57b1-4f70-ae65-ef864e317e63\") "
Aug 05 18:02:06.387944 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.387817    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0aa8c42e-c04c-4465-8139-f656d09cdded-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0aa8c42e-c04c-4465-8139-f656d09cdded" (UID: "0aa8c42e-c04c-4465-8139-f656d09cdded"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:02:06.388007 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.387982    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b84e0fae-f53e-441a-8aa9-595b69d8a37e-plugins-config-volume\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:02:06.388007 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.388003    2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2478b3ab-57b1-4f70-ae65-ef864e317e63-openshift-service-ca\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:02:06.388112 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.388018    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0aa8c42e-c04c-4465-8139-f656d09cdded-plugins-config-volume\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:02:06.389643 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.389613    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2478b3ab-57b1-4f70-ae65-ef864e317e63-kube-api-access-22bds" (OuterVolumeSpecName: "kube-api-access-22bds") pod "2478b3ab-57b1-4f70-ae65-ef864e317e63" (UID: "2478b3ab-57b1-4f70-ae65-ef864e317e63"). InnerVolumeSpecName "kube-api-access-22bds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:02:06.389850 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.389821    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b84e0fae-f53e-441a-8aa9-595b69d8a37e-kube-api-access-m9d9f" (OuterVolumeSpecName: "kube-api-access-m9d9f") pod "b84e0fae-f53e-441a-8aa9-595b69d8a37e" (UID: "b84e0fae-f53e-441a-8aa9-595b69d8a37e"). InnerVolumeSpecName "kube-api-access-m9d9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:02:06.389919 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.389873    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0aa8c42e-c04c-4465-8139-f656d09cdded-kube-api-access-j8d4p" (OuterVolumeSpecName: "kube-api-access-j8d4p") pod "0aa8c42e-c04c-4465-8139-f656d09cdded" (UID: "0aa8c42e-c04c-4465-8139-f656d09cdded"). InnerVolumeSpecName "kube-api-access-j8d4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:02:06.389919 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.389869    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2478b3ab-57b1-4f70-ae65-ef864e317e63-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2478b3ab-57b1-4f70-ae65-ef864e317e63" (UID: "2478b3ab-57b1-4f70-ae65-ef864e317e63"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:02:06.488547 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.488491    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22bds\" (UniqueName: \"kubernetes.io/projected/2478b3ab-57b1-4f70-ae65-ef864e317e63-kube-api-access-22bds\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:02:06.488547 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.488511    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2478b3ab-57b1-4f70-ae65-ef864e317e63-maas-api-tls\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:02:06.488547 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.488521    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9d9f\" (UniqueName: \"kubernetes.io/projected/b84e0fae-f53e-441a-8aa9-595b69d8a37e-kube-api-access-m9d9f\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:02:06.488547 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.488530    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8d4p\" (UniqueName: \"kubernetes.io/projected/0aa8c42e-c04c-4465-8139-f656d09cdded-kube-api-access-j8d4p\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:02:06.695465 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.695434    2550 generic.go:356] "Generic (PLEG): container finished" podID="b84e0fae-f53e-441a-8aa9-595b69d8a37e" containerID="bf11e39cba982b371997f030c22ee69c49f23fd146f79d2af88787e8e757bc53" exitCode=0
Aug 05 18:02:06.695632 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.695475    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx"
Aug 05 18:02:06.695632 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.695523    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx" event={"ID":"b84e0fae-f53e-441a-8aa9-595b69d8a37e","Type":"ContainerDied","Data":"bf11e39cba982b371997f030c22ee69c49f23fd146f79d2af88787e8e757bc53"}
Aug 05 18:02:06.695632 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.695578    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx" event={"ID":"b84e0fae-f53e-441a-8aa9-595b69d8a37e","Type":"ContainerDied","Data":"7687060647a9453434d40df1c3b54bc408ec6345761b769f7ab9cce6329340b0"}
Aug 05 18:02:06.695632 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.695606    2550 scope.go:117] "RemoveContainer" containerID="bf11e39cba982b371997f030c22ee69c49f23fd146f79d2af88787e8e757bc53"
Aug 05 18:02:06.697064 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.697041    2550 generic.go:356] "Generic (PLEG): container finished" podID="0aa8c42e-c04c-4465-8139-f656d09cdded" containerID="4616e94699e2709a39089b3a48949379a187f1af9036051d1819bb8cdd468fad" exitCode=0
Aug 05 18:02:06.697177 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.697072    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2" event={"ID":"0aa8c42e-c04c-4465-8139-f656d09cdded","Type":"ContainerDied","Data":"4616e94699e2709a39089b3a48949379a187f1af9036051d1819bb8cdd468fad"}
Aug 05 18:02:06.697177 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.697129    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2" event={"ID":"0aa8c42e-c04c-4465-8139-f656d09cdded","Type":"ContainerDied","Data":"deb4844174f1bdc21233b03673016e7f7e74947289070a5327a27494582974d1"}
Aug 05 18:02:06.697177 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.697086    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2"
Aug 05 18:02:06.698521 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.698502    2550 generic.go:356] "Generic (PLEG): container finished" podID="2478b3ab-57b1-4f70-ae65-ef864e317e63" containerID="a12ddc0784277b173b527347e34f3200dfc3867a03bc01a5bc2d9426f5ab104f" exitCode=0
Aug 05 18:02:06.698613 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.698531    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x"
Aug 05 18:02:06.698613 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.698531    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x" event={"ID":"2478b3ab-57b1-4f70-ae65-ef864e317e63","Type":"ContainerDied","Data":"a12ddc0784277b173b527347e34f3200dfc3867a03bc01a5bc2d9426f5ab104f"}
Aug 05 18:02:06.698761 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.698649    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x" event={"ID":"2478b3ab-57b1-4f70-ae65-ef864e317e63","Type":"ContainerDied","Data":"d2596fdde32d675f435225ddd6caf0dff4cb410bcb800456dfef4abdbd0af544"}
Aug 05 18:02:06.706025 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.706006    2550 scope.go:117] "RemoveContainer" containerID="bf11e39cba982b371997f030c22ee69c49f23fd146f79d2af88787e8e757bc53"
Aug 05 18:02:06.706314 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:02:06.706293    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf11e39cba982b371997f030c22ee69c49f23fd146f79d2af88787e8e757bc53\": container with ID starting with bf11e39cba982b371997f030c22ee69c49f23fd146f79d2af88787e8e757bc53 not found: ID does not exist" containerID="bf11e39cba982b371997f030c22ee69c49f23fd146f79d2af88787e8e757bc53"
Aug 05 18:02:06.706412 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.706324    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf11e39cba982b371997f030c22ee69c49f23fd146f79d2af88787e8e757bc53"} err="failed to get container status \"bf11e39cba982b371997f030c22ee69c49f23fd146f79d2af88787e8e757bc53\": rpc error: code = NotFound desc = could not find container \"bf11e39cba982b371997f030c22ee69c49f23fd146f79d2af88787e8e757bc53\": container with ID starting with bf11e39cba982b371997f030c22ee69c49f23fd146f79d2af88787e8e757bc53 not found: ID does not exist"
Aug 05 18:02:06.706412 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.706350    2550 scope.go:117] "RemoveContainer" containerID="4616e94699e2709a39089b3a48949379a187f1af9036051d1819bb8cdd468fad"
Aug 05 18:02:06.714640 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.714623    2550 scope.go:117] "RemoveContainer" containerID="4616e94699e2709a39089b3a48949379a187f1af9036051d1819bb8cdd468fad"
Aug 05 18:02:06.714965 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:02:06.714948    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4616e94699e2709a39089b3a48949379a187f1af9036051d1819bb8cdd468fad\": container with ID starting with 4616e94699e2709a39089b3a48949379a187f1af9036051d1819bb8cdd468fad not found: ID does not exist" containerID="4616e94699e2709a39089b3a48949379a187f1af9036051d1819bb8cdd468fad"
Aug 05 18:02:06.715043 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.714972    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4616e94699e2709a39089b3a48949379a187f1af9036051d1819bb8cdd468fad"} err="failed to get container status \"4616e94699e2709a39089b3a48949379a187f1af9036051d1819bb8cdd468fad\": rpc error: code = NotFound desc = could not find container \"4616e94699e2709a39089b3a48949379a187f1af9036051d1819bb8cdd468fad\": container with ID starting with 4616e94699e2709a39089b3a48949379a187f1af9036051d1819bb8cdd468fad not found: ID does not exist"
Aug 05 18:02:06.715043 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.714987    2550 scope.go:117] "RemoveContainer" containerID="a12ddc0784277b173b527347e34f3200dfc3867a03bc01a5bc2d9426f5ab104f"
Aug 05 18:02:06.719048 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.719030    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx"]
Aug 05 18:02:06.722316 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.722300    2550 scope.go:117] "RemoveContainer" containerID="a12ddc0784277b173b527347e34f3200dfc3867a03bc01a5bc2d9426f5ab104f"
Aug 05 18:02:06.722556 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:02:06.722539    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a12ddc0784277b173b527347e34f3200dfc3867a03bc01a5bc2d9426f5ab104f\": container with ID starting with a12ddc0784277b173b527347e34f3200dfc3867a03bc01a5bc2d9426f5ab104f not found: ID does not exist" containerID="a12ddc0784277b173b527347e34f3200dfc3867a03bc01a5bc2d9426f5ab104f"
Aug 05 18:02:06.722612 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.722563    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a12ddc0784277b173b527347e34f3200dfc3867a03bc01a5bc2d9426f5ab104f"} err="failed to get container status \"a12ddc0784277b173b527347e34f3200dfc3867a03bc01a5bc2d9426f5ab104f\": rpc error: code = NotFound desc = could not find container \"a12ddc0784277b173b527347e34f3200dfc3867a03bc01a5bc2d9426f5ab104f\": container with ID starting with a12ddc0784277b173b527347e34f3200dfc3867a03bc01a5bc2d9426f5ab104f not found: ID does not exist"
Aug 05 18:02:06.728995 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.724081    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-75d3fbe4-76db557c8-vqfvx"]
Aug 05 18:02:06.737842 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.737822    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2"]
Aug 05 18:02:06.741473 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.741421    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2"]
Aug 05 18:02:06.752846 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.752826    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x"]
Aug 05 18:02:06.758340 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:06.758322    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-75d3fbe4-77567756c5-r6j9x"]
Aug 05 18:02:07.309079 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:07.309035    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-75d3fbe4-5f5bd78656-slqn2" podUID="0aa8c42e-c04c-4465-8139-f656d09cdded" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.59:9004: i/o timeout"
Aug 05 18:02:07.700167 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:07.700133    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0aa8c42e-c04c-4465-8139-f656d09cdded" path="/var/lib/kubelet/pods/0aa8c42e-c04c-4465-8139-f656d09cdded/volumes"
Aug 05 18:02:07.700518 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:07.700504    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2478b3ab-57b1-4f70-ae65-ef864e317e63" path="/var/lib/kubelet/pods/2478b3ab-57b1-4f70-ae65-ef864e317e63/volumes"
Aug 05 18:02:07.700866 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:07.700850    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b84e0fae-f53e-441a-8aa9-595b69d8a37e" path="/var/lib/kubelet/pods/b84e0fae-f53e-441a-8aa9-595b69d8a37e/volumes"
Aug 05 18:02:14.255044 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:14.255001    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"]
Aug 05 18:02:29.274125 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.274059    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8" podUID="d6f7ef06-1583-44df-8da6-579a29e18f86" containerName="main" containerID="cri-o://a367c370145d2dda2e80e279c0ca4b12a92927fa7397e0ee91d5841167233b89" gracePeriod=45
Aug 05 18:02:29.526440 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.526384    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"
Aug 05 18:02:29.687973 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.687934    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pw87h\" (UniqueName: \"kubernetes.io/projected/d6f7ef06-1583-44df-8da6-579a29e18f86-kube-api-access-pw87h\") pod \"d6f7ef06-1583-44df-8da6-579a29e18f86\" (UID: \"d6f7ef06-1583-44df-8da6-579a29e18f86\") "
Aug 05 18:02:29.688169 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.688020    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6f7ef06-1583-44df-8da6-579a29e18f86-dshm\") pod \"d6f7ef06-1583-44df-8da6-579a29e18f86\" (UID: \"d6f7ef06-1583-44df-8da6-579a29e18f86\") "
Aug 05 18:02:29.688169 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.688065    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6f7ef06-1583-44df-8da6-579a29e18f86-home\") pod \"d6f7ef06-1583-44df-8da6-579a29e18f86\" (UID: \"d6f7ef06-1583-44df-8da6-579a29e18f86\") "
Aug 05 18:02:29.688169 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.688091    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6f7ef06-1583-44df-8da6-579a29e18f86-tls-certs\") pod \"d6f7ef06-1583-44df-8da6-579a29e18f86\" (UID: \"d6f7ef06-1583-44df-8da6-579a29e18f86\") "
Aug 05 18:02:29.688169 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.688165    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6f7ef06-1583-44df-8da6-579a29e18f86-tmp-dir\") pod \"d6f7ef06-1583-44df-8da6-579a29e18f86\" (UID: \"d6f7ef06-1583-44df-8da6-579a29e18f86\") "
Aug 05 18:02:29.688393 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.688197    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6f7ef06-1583-44df-8da6-579a29e18f86-model-cache\") pod \"d6f7ef06-1583-44df-8da6-579a29e18f86\" (UID: \"d6f7ef06-1583-44df-8da6-579a29e18f86\") "
Aug 05 18:02:29.688393 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.688318    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6f7ef06-1583-44df-8da6-579a29e18f86-home" (OuterVolumeSpecName: "home") pod "d6f7ef06-1583-44df-8da6-579a29e18f86" (UID: "d6f7ef06-1583-44df-8da6-579a29e18f86"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:02:29.688521 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.688503    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6f7ef06-1583-44df-8da6-579a29e18f86-home\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:02:29.688586 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.688473    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6f7ef06-1583-44df-8da6-579a29e18f86-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d6f7ef06-1583-44df-8da6-579a29e18f86" (UID: "d6f7ef06-1583-44df-8da6-579a29e18f86"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:02:29.688586 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.688562    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6f7ef06-1583-44df-8da6-579a29e18f86-model-cache" (OuterVolumeSpecName: "model-cache") pod "d6f7ef06-1583-44df-8da6-579a29e18f86" (UID: "d6f7ef06-1583-44df-8da6-579a29e18f86"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:02:29.690281 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.690251    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6f7ef06-1583-44df-8da6-579a29e18f86-kube-api-access-pw87h" (OuterVolumeSpecName: "kube-api-access-pw87h") pod "d6f7ef06-1583-44df-8da6-579a29e18f86" (UID: "d6f7ef06-1583-44df-8da6-579a29e18f86"). InnerVolumeSpecName "kube-api-access-pw87h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:02:29.690368 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.690305    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6f7ef06-1583-44df-8da6-579a29e18f86-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d6f7ef06-1583-44df-8da6-579a29e18f86" (UID: "d6f7ef06-1583-44df-8da6-579a29e18f86"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:02:29.690368 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.690347    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6f7ef06-1583-44df-8da6-579a29e18f86-dshm" (OuterVolumeSpecName: "dshm") pod "d6f7ef06-1583-44df-8da6-579a29e18f86" (UID: "d6f7ef06-1583-44df-8da6-579a29e18f86"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:02:29.789140 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.789050    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6f7ef06-1583-44df-8da6-579a29e18f86-dshm\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:02:29.789140 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.789080    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6f7ef06-1583-44df-8da6-579a29e18f86-tls-certs\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:02:29.789140 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.789091    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6f7ef06-1583-44df-8da6-579a29e18f86-tmp-dir\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:02:29.789140 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.789100    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6f7ef06-1583-44df-8da6-579a29e18f86-model-cache\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:02:29.789140 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.789109    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pw87h\" (UniqueName: \"kubernetes.io/projected/d6f7ef06-1583-44df-8da6-579a29e18f86-kube-api-access-pw87h\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:02:29.803327 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.803294    2550 generic.go:356] "Generic (PLEG): container finished" podID="d6f7ef06-1583-44df-8da6-579a29e18f86" containerID="a367c370145d2dda2e80e279c0ca4b12a92927fa7397e0ee91d5841167233b89" exitCode=0
Aug 05 18:02:29.803479 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.803338    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"
Aug 05 18:02:29.803479 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.803357    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8" event={"ID":"d6f7ef06-1583-44df-8da6-579a29e18f86","Type":"ContainerDied","Data":"a367c370145d2dda2e80e279c0ca4b12a92927fa7397e0ee91d5841167233b89"}
Aug 05 18:02:29.803479 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.803388    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8" event={"ID":"d6f7ef06-1583-44df-8da6-579a29e18f86","Type":"ContainerDied","Data":"32cbf6de9e60ee0d76615092ccc305985ce31fbc0d248500ca84e1df7fe80e54"}
Aug 05 18:02:29.803479 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.803405    2550 scope.go:117] "RemoveContainer" containerID="a367c370145d2dda2e80e279c0ca4b12a92927fa7397e0ee91d5841167233b89"
Aug 05 18:02:29.811558 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.811533    2550 scope.go:117] "RemoveContainer" containerID="a367c370145d2dda2e80e279c0ca4b12a92927fa7397e0ee91d5841167233b89"
Aug 05 18:02:29.811829 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:02:29.811812    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a367c370145d2dda2e80e279c0ca4b12a92927fa7397e0ee91d5841167233b89\": container with ID starting with a367c370145d2dda2e80e279c0ca4b12a92927fa7397e0ee91d5841167233b89 not found: ID does not exist" containerID="a367c370145d2dda2e80e279c0ca4b12a92927fa7397e0ee91d5841167233b89"
Aug 05 18:02:29.811883 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.811843    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a367c370145d2dda2e80e279c0ca4b12a92927fa7397e0ee91d5841167233b89"} err="failed to get container status \"a367c370145d2dda2e80e279c0ca4b12a92927fa7397e0ee91d5841167233b89\": rpc error: code = NotFound desc = could not find container \"a367c370145d2dda2e80e279c0ca4b12a92927fa7397e0ee91d5841167233b89\": container with ID starting with a367c370145d2dda2e80e279c0ca4b12a92927fa7397e0ee91d5841167233b89 not found: ID does not exist"
Aug 05 18:02:29.820810 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.820785    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"]
Aug 05 18:02:29.826324 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:29.826302    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-75d3fbe4/e2e-lifecycle-model-75d3fbe4-kserve-77485dc6cf-r6nf8"]
Aug 05 18:02:31.700594 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:02:31.700563    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6f7ef06-1583-44df-8da6-579a29e18f86" path="/var/lib/kubelet/pods/d6f7ef06-1583-44df-8da6-579a29e18f86/volumes"
Aug 05 18:04:09.701930 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:09.701900    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95"]
Aug 05 18:04:09.702508 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:09.702373    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b84e0fae-f53e-441a-8aa9-595b69d8a37e" containerName="payload-processing"
Aug 05 18:04:09.702508 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:09.702391    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="b84e0fae-f53e-441a-8aa9-595b69d8a37e" containerName="payload-processing"
Aug 05 18:04:09.702508 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:09.702406    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6f7ef06-1583-44df-8da6-579a29e18f86" containerName="main"
Aug 05 18:04:09.702508 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:09.702415    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6f7ef06-1583-44df-8da6-579a29e18f86" containerName="main"
Aug 05 18:04:09.702508 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:09.702448    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aa8c42e-c04c-4465-8139-f656d09cdded" containerName="payload-pre-processing"
Aug 05 18:04:09.702508 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:09.702456    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aa8c42e-c04c-4465-8139-f656d09cdded" containerName="payload-pre-processing"
Aug 05 18:04:09.702508 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:09.702467    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2478b3ab-57b1-4f70-ae65-ef864e317e63" containerName="maas-api"
Aug 05 18:04:09.702508 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:09.702474    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="2478b3ab-57b1-4f70-ae65-ef864e317e63" containerName="maas-api"
Aug 05 18:04:09.702946 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:09.702568    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6f7ef06-1583-44df-8da6-579a29e18f86" containerName="main"
Aug 05 18:04:09.702946 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:09.702579    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="0aa8c42e-c04c-4465-8139-f656d09cdded" containerName="payload-pre-processing"
Aug 05 18:04:09.702946 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:09.702589    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="2478b3ab-57b1-4f70-ae65-ef864e317e63" containerName="maas-api"
Aug 05 18:04:09.702946 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:09.702600    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="b84e0fae-f53e-441a-8aa9-595b69d8a37e" containerName="payload-processing"
Aug 05 18:04:09.705744 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:09.705725    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95"
Aug 05 18:04:09.708466 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:09.708444    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-83d598-serving-cert\""
Aug 05 18:04:09.709834 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:09.709811    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95"]
Aug 05 18:04:09.863575 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:09.863534    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60f75d34-1c81-4b7e-9271-3be6d78c23bf-maas-api-tls\") pod \"maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95\" (UID: \"60f75d34-1c81-4b7e-9271-3be6d78c23bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95"
Aug 05 18:04:09.863781 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:09.863597    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60f75d34-1c81-4b7e-9271-3be6d78c23bf-openshift-service-ca\") pod \"maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95\" (UID: \"60f75d34-1c81-4b7e-9271-3be6d78c23bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95"
Aug 05 18:04:09.863781 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:09.863662    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-669gc\" (UniqueName: \"kubernetes.io/projected/60f75d34-1c81-4b7e-9271-3be6d78c23bf-kube-api-access-669gc\") pod \"maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95\" (UID: \"60f75d34-1c81-4b7e-9271-3be6d78c23bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95"
Aug 05 18:04:09.964488 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:09.964384    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60f75d34-1c81-4b7e-9271-3be6d78c23bf-openshift-service-ca\") pod \"maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95\" (UID: \"60f75d34-1c81-4b7e-9271-3be6d78c23bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95"
Aug 05 18:04:09.964655 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:09.964566    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-669gc\" (UniqueName: \"kubernetes.io/projected/60f75d34-1c81-4b7e-9271-3be6d78c23bf-kube-api-access-669gc\") pod \"maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95\" (UID: \"60f75d34-1c81-4b7e-9271-3be6d78c23bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95"
Aug 05 18:04:09.964813 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:09.964768    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60f75d34-1c81-4b7e-9271-3be6d78c23bf-maas-api-tls\") pod \"maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95\" (UID: \"60f75d34-1c81-4b7e-9271-3be6d78c23bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95"
Aug 05 18:04:09.965166 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:09.965144    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60f75d34-1c81-4b7e-9271-3be6d78c23bf-openshift-service-ca\") pod \"maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95\" (UID: \"60f75d34-1c81-4b7e-9271-3be6d78c23bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95"
Aug 05 18:04:09.967129 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:09.967103    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60f75d34-1c81-4b7e-9271-3be6d78c23bf-maas-api-tls\") pod \"maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95\" (UID: \"60f75d34-1c81-4b7e-9271-3be6d78c23bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95"
Aug 05 18:04:09.977139 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:09.977113    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-669gc\" (UniqueName: \"kubernetes.io/projected/60f75d34-1c81-4b7e-9271-3be6d78c23bf-kube-api-access-669gc\") pod \"maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95\" (UID: \"60f75d34-1c81-4b7e-9271-3be6d78c23bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95"
Aug 05 18:04:10.018187 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:10.018146    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95"
Aug 05 18:04:10.153793 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:10.153764    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95"]
Aug 05 18:04:10.156266 ip-10-0-134-46 kubenswrapper[2550]: W0805 18:04:10.156238    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60f75d34_1c81_4b7e_9271_3be6d78c23bf.slice/crio-7b4e21eb5db23046cc8b20c41035a7f108673d3184e5dd2c004e42215b823c1f WatchSource:0}: Error finding container 7b4e21eb5db23046cc8b20c41035a7f108673d3184e5dd2c004e42215b823c1f: Status 404 returned error can't find the container with id 7b4e21eb5db23046cc8b20c41035a7f108673d3184e5dd2c004e42215b823c1f
Aug 05 18:04:10.268742 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:10.268645    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95" event={"ID":"60f75d34-1c81-4b7e-9271-3be6d78c23bf","Type":"ContainerStarted","Data":"7b4e21eb5db23046cc8b20c41035a7f108673d3184e5dd2c004e42215b823c1f"}
Aug 05 18:04:10.498555 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:10.498520    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg"]
Aug 05 18:04:10.503178 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:10.503162    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg"
Aug 05 18:04:10.505754 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:10.505730    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-83d598\""
Aug 05 18:04:10.505867 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:10.505783    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-83d598-dockercfg-pktlk\""
Aug 05 18:04:10.510552 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:10.510469    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg"]
Aug 05 18:04:10.551998 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:10.551920    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs"]
Aug 05 18:04:10.555305 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:10.555286    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs"
Aug 05 18:04:10.565455 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:10.565428    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs"]
Aug 05 18:04:10.671998 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:10.671961    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gj6p\" (UniqueName: \"kubernetes.io/projected/31ec43b8-aa7c-4911-9db1-de6c22ee2e86-kube-api-access-8gj6p\") pod \"payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs\" (UID: \"31ec43b8-aa7c-4911-9db1-de6c22ee2e86\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs"
Aug 05 18:04:10.672182 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:10.672019    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q558\" (UniqueName: \"kubernetes.io/projected/9d8054b1-9a7f-4d8a-b878-5d8acd566fec-kube-api-access-7q558\") pod \"payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg\" (UID: \"9d8054b1-9a7f-4d8a-b878-5d8acd566fec\") " pod="openshift-ingress/payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg"
Aug 05 18:04:10.672182 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:10.672094    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d8054b1-9a7f-4d8a-b878-5d8acd566fec-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg\" (UID: \"9d8054b1-9a7f-4d8a-b878-5d8acd566fec\") " pod="openshift-ingress/payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg"
Aug 05 18:04:10.672182 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:10.672133    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31ec43b8-aa7c-4911-9db1-de6c22ee2e86-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs\" (UID: \"31ec43b8-aa7c-4911-9db1-de6c22ee2e86\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs"
Aug 05 18:04:10.773160 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:10.773131    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gj6p\" (UniqueName: \"kubernetes.io/projected/31ec43b8-aa7c-4911-9db1-de6c22ee2e86-kube-api-access-8gj6p\") pod \"payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs\" (UID: \"31ec43b8-aa7c-4911-9db1-de6c22ee2e86\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs"
Aug 05 18:04:10.773538 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:10.773198    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q558\" (UniqueName: \"kubernetes.io/projected/9d8054b1-9a7f-4d8a-b878-5d8acd566fec-kube-api-access-7q558\") pod \"payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg\" (UID: \"9d8054b1-9a7f-4d8a-b878-5d8acd566fec\") " pod="openshift-ingress/payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg"
Aug 05 18:04:10.773538 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:10.773240    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d8054b1-9a7f-4d8a-b878-5d8acd566fec-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg\" (UID: \"9d8054b1-9a7f-4d8a-b878-5d8acd566fec\") " pod="openshift-ingress/payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg"
Aug 05 18:04:10.773538 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:10.773270    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31ec43b8-aa7c-4911-9db1-de6c22ee2e86-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs\" (UID: \"31ec43b8-aa7c-4911-9db1-de6c22ee2e86\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs"
Aug 05 18:04:10.774035 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:10.774013    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d8054b1-9a7f-4d8a-b878-5d8acd566fec-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg\" (UID: \"9d8054b1-9a7f-4d8a-b878-5d8acd566fec\") " pod="openshift-ingress/payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg"
Aug 05 18:04:10.774217 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:10.774195    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31ec43b8-aa7c-4911-9db1-de6c22ee2e86-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs\" (UID: \"31ec43b8-aa7c-4911-9db1-de6c22ee2e86\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs"
Aug 05 18:04:10.781518 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:10.781486    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q558\" (UniqueName: \"kubernetes.io/projected/9d8054b1-9a7f-4d8a-b878-5d8acd566fec-kube-api-access-7q558\") pod \"payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg\" (UID: \"9d8054b1-9a7f-4d8a-b878-5d8acd566fec\") " pod="openshift-ingress/payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg"
Aug 05 18:04:10.781810 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:10.781790    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gj6p\" (UniqueName: \"kubernetes.io/projected/31ec43b8-aa7c-4911-9db1-de6c22ee2e86-kube-api-access-8gj6p\") pod \"payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs\" (UID: \"31ec43b8-aa7c-4911-9db1-de6c22ee2e86\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs"
Aug 05 18:04:10.815682 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:10.815657    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg"
Aug 05 18:04:10.866664 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:10.866533    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs"
Aug 05 18:04:10.965119 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:10.965076    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg"]
Aug 05 18:04:10.968362 ip-10-0-134-46 kubenswrapper[2550]: W0805 18:04:10.968315    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d8054b1_9a7f_4d8a_b878_5d8acd566fec.slice/crio-204a4ccd8c0d5c75de1cb9e31ec128b660c910a260b3f7550271c41521f6ece3 WatchSource:0}: Error finding container 204a4ccd8c0d5c75de1cb9e31ec128b660c910a260b3f7550271c41521f6ece3: Status 404 returned error can't find the container with id 204a4ccd8c0d5c75de1cb9e31ec128b660c910a260b3f7550271c41521f6ece3
Aug 05 18:04:11.027583 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:11.027555    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs"]
Aug 05 18:04:11.032904 ip-10-0-134-46 kubenswrapper[2550]: W0805 18:04:11.032864    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31ec43b8_aa7c_4911_9db1_de6c22ee2e86.slice/crio-6b1321aeca51147a2a63d0f9c04ff083d85f9f7d9d3b3e52044d3780c26dd2e0 WatchSource:0}: Error finding container 6b1321aeca51147a2a63d0f9c04ff083d85f9f7d9d3b3e52044d3780c26dd2e0: Status 404 returned error can't find the container with id 6b1321aeca51147a2a63d0f9c04ff083d85f9f7d9d3b3e52044d3780c26dd2e0
Aug 05 18:04:11.277081 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:11.277043    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg" event={"ID":"9d8054b1-9a7f-4d8a-b878-5d8acd566fec","Type":"ContainerStarted","Data":"f291247e98224283ea7c46bbb6963aaef8db6209efbeae04fdc37416406e1904"}
Aug 05 18:04:11.277296 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:11.277114    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg" event={"ID":"9d8054b1-9a7f-4d8a-b878-5d8acd566fec","Type":"ContainerStarted","Data":"204a4ccd8c0d5c75de1cb9e31ec128b660c910a260b3f7550271c41521f6ece3"}
Aug 05 18:04:11.277296 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:11.277155    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg"
Aug 05 18:04:11.278756 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:11.278729    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95" event={"ID":"60f75d34-1c81-4b7e-9271-3be6d78c23bf","Type":"ContainerStarted","Data":"3ef17c4411199c34bc6f39514986aac3efc5ef5e0068520758d6d82743fc29c6"}
Aug 05 18:04:11.278891 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:11.278807    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95"
Aug 05 18:04:11.280337 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:11.280315    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs" event={"ID":"31ec43b8-aa7c-4911-9db1-de6c22ee2e86","Type":"ContainerStarted","Data":"b4227991835df636990c0f78b3c3d704187c2cb5e9245c15edc1185841e6a949"}
Aug 05 18:04:11.280423 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:11.280359    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs" event={"ID":"31ec43b8-aa7c-4911-9db1-de6c22ee2e86","Type":"ContainerStarted","Data":"6b1321aeca51147a2a63d0f9c04ff083d85f9f7d9d3b3e52044d3780c26dd2e0"}
Aug 05 18:04:11.280423 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:11.280387    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs"
Aug 05 18:04:11.296608 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:11.296562    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg" podStartSLOduration=1.296547662 podStartE2EDuration="1.296547662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:04:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:04:11.293917 +0000 UTC m=+2558.275494430" watchObservedRunningTime="2026-08-05 18:04:11.296547662 +0000 UTC m=+2558.278125091"
Aug 05 18:04:11.311252 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:11.311202    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs" podStartSLOduration=1.311186405 podStartE2EDuration="1.311186405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:04:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:04:11.309216943 +0000 UTC m=+2558.290794372" watchObservedRunningTime="2026-08-05 18:04:11.311186405 +0000 UTC m=+2558.292763834"
Aug 05 18:04:11.330350 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:11.330245    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95" podStartSLOduration=1.722945187 podStartE2EDuration="2.330221566s" totalImagesPullingTime="607.276379ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:04:11.327680487 +0000 UTC m=+2558.309257915" watchObservedRunningTime="2026-08-05 18:04:11.330221566 +0000 UTC m=+2558.311798996"
Aug 05 18:04:17.290771 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:17.290741    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95"
Aug 05 18:04:22.287269 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:22.287230    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg"
Aug 05 18:04:22.287736 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:22.287447    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs"
Aug 05 18:04:37.920115 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:37.920080    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb"]
Aug 05 18:04:37.924548 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:37.924530    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb"
Aug 05 18:04:37.927850 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:37.927828    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-ed4837-serving-cert\""
Aug 05 18:04:37.936477 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:37.936455    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb"]
Aug 05 18:04:37.986498 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:37.986473    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5db07b76-3930-4d67-b118-c8306eba26f7-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb\" (UID: \"5db07b76-3930-4d67-b118-c8306eba26f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb"
Aug 05 18:04:37.986610 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:37.986507    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5db07b76-3930-4d67-b118-c8306eba26f7-maas-api-tls\") pod \"maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb\" (UID: \"5db07b76-3930-4d67-b118-c8306eba26f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb"
Aug 05 18:04:37.986610 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:37.986542    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgkkf\" (UniqueName: \"kubernetes.io/projected/5db07b76-3930-4d67-b118-c8306eba26f7-kube-api-access-kgkkf\") pod \"maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb\" (UID: \"5db07b76-3930-4d67-b118-c8306eba26f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb"
Aug 05 18:04:38.087603 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:38.087567    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5db07b76-3930-4d67-b118-c8306eba26f7-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb\" (UID: \"5db07b76-3930-4d67-b118-c8306eba26f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb"
Aug 05 18:04:38.087603 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:38.087610    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5db07b76-3930-4d67-b118-c8306eba26f7-maas-api-tls\") pod \"maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb\" (UID: \"5db07b76-3930-4d67-b118-c8306eba26f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb"
Aug 05 18:04:38.087857 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:04:38.087740    2550 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-serving-cert: secret "maas-api-e2e-inf-b-ed4837-serving-cert" not found
Aug 05 18:04:38.087857 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:38.087741    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgkkf\" (UniqueName: \"kubernetes.io/projected/5db07b76-3930-4d67-b118-c8306eba26f7-kube-api-access-kgkkf\") pod \"maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb\" (UID: \"5db07b76-3930-4d67-b118-c8306eba26f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb"
Aug 05 18:04:38.087857 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:04:38.087797    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5db07b76-3930-4d67-b118-c8306eba26f7-maas-api-tls podName:5db07b76-3930-4d67-b118-c8306eba26f7 nodeName:}" failed. No retries permitted until 2026-08-05 18:04:38.587776232 +0000 UTC m=+2585.569353638 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5db07b76-3930-4d67-b118-c8306eba26f7-maas-api-tls") pod "maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb" (UID: "5db07b76-3930-4d67-b118-c8306eba26f7") : secret "maas-api-e2e-inf-b-ed4837-serving-cert" not found
Aug 05 18:04:38.088339 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:38.088319    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5db07b76-3930-4d67-b118-c8306eba26f7-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb\" (UID: \"5db07b76-3930-4d67-b118-c8306eba26f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb"
Aug 05 18:04:38.099137 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:38.099108    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgkkf\" (UniqueName: \"kubernetes.io/projected/5db07b76-3930-4d67-b118-c8306eba26f7-kube-api-access-kgkkf\") pod \"maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb\" (UID: \"5db07b76-3930-4d67-b118-c8306eba26f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb"
Aug 05 18:04:38.593198 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:38.593166    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5db07b76-3930-4d67-b118-c8306eba26f7-maas-api-tls\") pod \"maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb\" (UID: \"5db07b76-3930-4d67-b118-c8306eba26f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb"
Aug 05 18:04:38.593367 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:04:38.593312    2550 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-serving-cert: secret "maas-api-e2e-inf-b-ed4837-serving-cert" not found
Aug 05 18:04:38.593415 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:04:38.593372    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5db07b76-3930-4d67-b118-c8306eba26f7-maas-api-tls podName:5db07b76-3930-4d67-b118-c8306eba26f7 nodeName:}" failed. No retries permitted until 2026-08-05 18:04:39.593357285 +0000 UTC m=+2586.574934692 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5db07b76-3930-4d67-b118-c8306eba26f7-maas-api-tls") pod "maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb" (UID: "5db07b76-3930-4d67-b118-c8306eba26f7") : secret "maas-api-e2e-inf-b-ed4837-serving-cert" not found
Aug 05 18:04:38.658538 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:38.658502    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb"]
Aug 05 18:04:38.662807 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:38.662785    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb"
Aug 05 18:04:38.665416 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:38.665390    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-ed4837\""
Aug 05 18:04:38.666211 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:38.666191    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb"]
Aug 05 18:04:38.694745 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:38.694716    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tb6v6\" (UniqueName: \"kubernetes.io/projected/81dc4899-8951-4289-b661-9da6c7db7f90-kube-api-access-tb6v6\") pod \"payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb\" (UID: \"81dc4899-8951-4289-b661-9da6c7db7f90\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb"
Aug 05 18:04:38.694885 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:38.694797    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81dc4899-8951-4289-b661-9da6c7db7f90-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb\" (UID: \"81dc4899-8951-4289-b661-9da6c7db7f90\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb"
Aug 05 18:04:38.734387 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:38.734357    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv"]
Aug 05 18:04:38.737732 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:38.737692    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv"
Aug 05 18:04:38.740261 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:38.740240    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-ed4837-dockercfg-hcqrt\""
Aug 05 18:04:38.748186 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:38.748167    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv"]
Aug 05 18:04:38.796116 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:38.796086    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tb6v6\" (UniqueName: \"kubernetes.io/projected/81dc4899-8951-4289-b661-9da6c7db7f90-kube-api-access-tb6v6\") pod \"payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb\" (UID: \"81dc4899-8951-4289-b661-9da6c7db7f90\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb"
Aug 05 18:04:38.796398 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:38.796128    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tffhd\" (UniqueName: \"kubernetes.io/projected/9673bf92-fcd6-4901-9d11-971f8d05258b-kube-api-access-tffhd\") pod \"payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv\" (UID: \"9673bf92-fcd6-4901-9d11-971f8d05258b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv"
Aug 05 18:04:38.796398 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:38.796170    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81dc4899-8951-4289-b661-9da6c7db7f90-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb\" (UID: \"81dc4899-8951-4289-b661-9da6c7db7f90\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb"
Aug 05 18:04:38.796398 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:38.796204    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9673bf92-fcd6-4901-9d11-971f8d05258b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv\" (UID: \"9673bf92-fcd6-4901-9d11-971f8d05258b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv"
Aug 05 18:04:38.797172 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:38.797148    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81dc4899-8951-4289-b661-9da6c7db7f90-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb\" (UID: \"81dc4899-8951-4289-b661-9da6c7db7f90\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb"
Aug 05 18:04:38.806509 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:38.806485    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tb6v6\" (UniqueName: \"kubernetes.io/projected/81dc4899-8951-4289-b661-9da6c7db7f90-kube-api-access-tb6v6\") pod \"payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb\" (UID: \"81dc4899-8951-4289-b661-9da6c7db7f90\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb"
Aug 05 18:04:38.898013 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:38.897925    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tffhd\" (UniqueName: \"kubernetes.io/projected/9673bf92-fcd6-4901-9d11-971f8d05258b-kube-api-access-tffhd\") pod \"payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv\" (UID: \"9673bf92-fcd6-4901-9d11-971f8d05258b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv"
Aug 05 18:04:38.898013 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:38.897973    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9673bf92-fcd6-4901-9d11-971f8d05258b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv\" (UID: \"9673bf92-fcd6-4901-9d11-971f8d05258b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv"
Aug 05 18:04:38.898598 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:38.898578    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9673bf92-fcd6-4901-9d11-971f8d05258b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv\" (UID: \"9673bf92-fcd6-4901-9d11-971f8d05258b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv"
Aug 05 18:04:38.906754 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:38.906730    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tffhd\" (UniqueName: \"kubernetes.io/projected/9673bf92-fcd6-4901-9d11-971f8d05258b-kube-api-access-tffhd\") pod \"payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv\" (UID: \"9673bf92-fcd6-4901-9d11-971f8d05258b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv"
Aug 05 18:04:38.975442 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:38.975410    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb"
Aug 05 18:04:39.047918 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:39.047891    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv"
Aug 05 18:04:39.184978 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:39.184949    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv"]
Aug 05 18:04:39.187220 ip-10-0-134-46 kubenswrapper[2550]: W0805 18:04:39.187187    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9673bf92_fcd6_4901_9d11_971f8d05258b.slice/crio-b2f20aca888c1dc7818cb9e3542ed47340cd8e7f480a2aece448fb96e1be1c47 WatchSource:0}: Error finding container b2f20aca888c1dc7818cb9e3542ed47340cd8e7f480a2aece448fb96e1be1c47: Status 404 returned error can't find the container with id b2f20aca888c1dc7818cb9e3542ed47340cd8e7f480a2aece448fb96e1be1c47
Aug 05 18:04:39.323442 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:39.323391    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb"]
Aug 05 18:04:39.326500 ip-10-0-134-46 kubenswrapper[2550]: W0805 18:04:39.326460    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81dc4899_8951_4289_b661_9da6c7db7f90.slice/crio-129edf93962f119f5a8d4f93df9463b2c2f5f48cb12cd5d19d060d0668e6ebae WatchSource:0}: Error finding container 129edf93962f119f5a8d4f93df9463b2c2f5f48cb12cd5d19d060d0668e6ebae: Status 404 returned error can't find the container with id 129edf93962f119f5a8d4f93df9463b2c2f5f48cb12cd5d19d060d0668e6ebae
Aug 05 18:04:39.413720 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:39.410732    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv" event={"ID":"9673bf92-fcd6-4901-9d11-971f8d05258b","Type":"ContainerStarted","Data":"2c700e870b38b1d5057cf4a96db502ea93cdcb7f9b6f4c93b3c27672d1c05920"}
Aug 05 18:04:39.413720 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:39.410796    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv" event={"ID":"9673bf92-fcd6-4901-9d11-971f8d05258b","Type":"ContainerStarted","Data":"b2f20aca888c1dc7818cb9e3542ed47340cd8e7f480a2aece448fb96e1be1c47"}
Aug 05 18:04:39.413720 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:39.410833    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv"
Aug 05 18:04:39.413720 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:39.412797    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb" event={"ID":"81dc4899-8951-4289-b661-9da6c7db7f90","Type":"ContainerStarted","Data":"c2a3be94dc84ee76c8320910ddfab53323d07a4c58fa67221f0a04e026789050"}
Aug 05 18:04:39.413720 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:39.412847    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb" event={"ID":"81dc4899-8951-4289-b661-9da6c7db7f90","Type":"ContainerStarted","Data":"129edf93962f119f5a8d4f93df9463b2c2f5f48cb12cd5d19d060d0668e6ebae"}
Aug 05 18:04:39.413720 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:39.413297    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb"
Aug 05 18:04:39.433872 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:39.433813    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv" podStartSLOduration=1.433793649 podStartE2EDuration="1.433793649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:04:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:04:39.431476126 +0000 UTC m=+2586.413053559" watchObservedRunningTime="2026-08-05 18:04:39.433793649 +0000 UTC m=+2586.415371079"
Aug 05 18:04:39.456647 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:39.456562    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb" podStartSLOduration=1.456543966 podStartE2EDuration="1.456543966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:04:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:04:39.456111372 +0000 UTC m=+2586.437688813" watchObservedRunningTime="2026-08-05 18:04:39.456543966 +0000 UTC m=+2586.438121397"
Aug 05 18:04:39.605303 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:39.605269    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5db07b76-3930-4d67-b118-c8306eba26f7-maas-api-tls\") pod \"maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb\" (UID: \"5db07b76-3930-4d67-b118-c8306eba26f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb"
Aug 05 18:04:39.607712 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:39.607680    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5db07b76-3930-4d67-b118-c8306eba26f7-maas-api-tls\") pod \"maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb\" (UID: \"5db07b76-3930-4d67-b118-c8306eba26f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb"
Aug 05 18:04:39.735454 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:39.735376    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb"
Aug 05 18:04:39.878291 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:39.878267    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb"]
Aug 05 18:04:39.879765 ip-10-0-134-46 kubenswrapper[2550]: W0805 18:04:39.879743    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5db07b76_3930_4d67_b118_c8306eba26f7.slice/crio-61d5c381a3d2cd0bc111c0f2eb17df7aa3d6f73e34bdde2412896c8e31ac218e WatchSource:0}: Error finding container 61d5c381a3d2cd0bc111c0f2eb17df7aa3d6f73e34bdde2412896c8e31ac218e: Status 404 returned error can't find the container with id 61d5c381a3d2cd0bc111c0f2eb17df7aa3d6f73e34bdde2412896c8e31ac218e
Aug 05 18:04:40.419495 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:40.419455    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb" event={"ID":"5db07b76-3930-4d67-b118-c8306eba26f7","Type":"ContainerStarted","Data":"61d5c381a3d2cd0bc111c0f2eb17df7aa3d6f73e34bdde2412896c8e31ac218e"}
Aug 05 18:04:41.425884 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:41.425848    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb" event={"ID":"5db07b76-3930-4d67-b118-c8306eba26f7","Type":"ContainerStarted","Data":"b2f8a432137a34c5f4681c77212a1f927cab5d2e74db449d035a0b7a0e7bcaef"}
Aug 05 18:04:41.426335 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:41.426102    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb"
Aug 05 18:04:41.449845 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:41.449805    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb" podStartSLOduration=3.843190264 podStartE2EDuration="4.449792124s" totalImagesPullingTime="606.60186ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:04:41.449568694 +0000 UTC m=+2588.431146123" watchObservedRunningTime="2026-08-05 18:04:41.449792124 +0000 UTC m=+2588.431369552"
Aug 05 18:04:47.436718 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:47.436668    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb"
Aug 05 18:04:50.421564 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:50.421530    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv"
Aug 05 18:04:51.428366 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:04:51.428338    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb"
Aug 05 18:05:53.158521 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.158492    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"]
Aug 05 18:05:53.162145 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.162126    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"
Aug 05 18:05:53.165643 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.165621    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-ed4837\"/\"default-dockercfg-22mct\""
Aug 05 18:05:53.165643 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.165634    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-ed4837\"/\"test-model-ed4837-kserve-self-signed-certs\""
Aug 05 18:05:53.165827 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.165621    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-ed4837\"/\"openshift-service-ca.crt\""
Aug 05 18:05:53.165827 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.165661    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-ed4837\"/\"kube-root-ca.crt\""
Aug 05 18:05:53.173190 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.173167    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"]
Aug 05 18:05:53.252306 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.252272    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15d500f6-368d-4910-a932-96920bcc2e04-model-cache\") pod \"test-model-ed4837-kserve-5d84f85887-7gc2j\" (UID: \"15d500f6-368d-4910-a932-96920bcc2e04\") " pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"
Aug 05 18:05:53.252306 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.252309    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15d500f6-368d-4910-a932-96920bcc2e04-home\") pod \"test-model-ed4837-kserve-5d84f85887-7gc2j\" (UID: \"15d500f6-368d-4910-a932-96920bcc2e04\") " pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"
Aug 05 18:05:53.252489 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.252328    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15d500f6-368d-4910-a932-96920bcc2e04-dshm\") pod \"test-model-ed4837-kserve-5d84f85887-7gc2j\" (UID: \"15d500f6-368d-4910-a932-96920bcc2e04\") " pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"
Aug 05 18:05:53.252489 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.252382    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-542hp\" (UniqueName: \"kubernetes.io/projected/15d500f6-368d-4910-a932-96920bcc2e04-kube-api-access-542hp\") pod \"test-model-ed4837-kserve-5d84f85887-7gc2j\" (UID: \"15d500f6-368d-4910-a932-96920bcc2e04\") " pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"
Aug 05 18:05:53.252489 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.252441    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15d500f6-368d-4910-a932-96920bcc2e04-tmp-dir\") pod \"test-model-ed4837-kserve-5d84f85887-7gc2j\" (UID: \"15d500f6-368d-4910-a932-96920bcc2e04\") " pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"
Aug 05 18:05:53.252594 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.252525    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15d500f6-368d-4910-a932-96920bcc2e04-tls-certs\") pod \"test-model-ed4837-kserve-5d84f85887-7gc2j\" (UID: \"15d500f6-368d-4910-a932-96920bcc2e04\") " pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"
Aug 05 18:05:53.353893 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.353861    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15d500f6-368d-4910-a932-96920bcc2e04-tls-certs\") pod \"test-model-ed4837-kserve-5d84f85887-7gc2j\" (UID: \"15d500f6-368d-4910-a932-96920bcc2e04\") " pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"
Aug 05 18:05:53.354069 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.353916    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15d500f6-368d-4910-a932-96920bcc2e04-model-cache\") pod \"test-model-ed4837-kserve-5d84f85887-7gc2j\" (UID: \"15d500f6-368d-4910-a932-96920bcc2e04\") " pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"
Aug 05 18:05:53.354069 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.354026    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15d500f6-368d-4910-a932-96920bcc2e04-home\") pod \"test-model-ed4837-kserve-5d84f85887-7gc2j\" (UID: \"15d500f6-368d-4910-a932-96920bcc2e04\") " pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"
Aug 05 18:05:53.354168 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.354078    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15d500f6-368d-4910-a932-96920bcc2e04-dshm\") pod \"test-model-ed4837-kserve-5d84f85887-7gc2j\" (UID: \"15d500f6-368d-4910-a932-96920bcc2e04\") " pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"
Aug 05 18:05:53.354168 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.354104    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-542hp\" (UniqueName: \"kubernetes.io/projected/15d500f6-368d-4910-a932-96920bcc2e04-kube-api-access-542hp\") pod \"test-model-ed4837-kserve-5d84f85887-7gc2j\" (UID: \"15d500f6-368d-4910-a932-96920bcc2e04\") " pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"
Aug 05 18:05:53.354168 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.354160    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15d500f6-368d-4910-a932-96920bcc2e04-tmp-dir\") pod \"test-model-ed4837-kserve-5d84f85887-7gc2j\" (UID: \"15d500f6-368d-4910-a932-96920bcc2e04\") " pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"
Aug 05 18:05:53.354313 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.354299    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15d500f6-368d-4910-a932-96920bcc2e04-model-cache\") pod \"test-model-ed4837-kserve-5d84f85887-7gc2j\" (UID: \"15d500f6-368d-4910-a932-96920bcc2e04\") " pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"
Aug 05 18:05:53.354377 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.354320    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15d500f6-368d-4910-a932-96920bcc2e04-home\") pod \"test-model-ed4837-kserve-5d84f85887-7gc2j\" (UID: \"15d500f6-368d-4910-a932-96920bcc2e04\") " pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"
Aug 05 18:05:53.354480 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.354463    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15d500f6-368d-4910-a932-96920bcc2e04-tmp-dir\") pod \"test-model-ed4837-kserve-5d84f85887-7gc2j\" (UID: \"15d500f6-368d-4910-a932-96920bcc2e04\") " pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"
Aug 05 18:05:53.356277 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.356249    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15d500f6-368d-4910-a932-96920bcc2e04-dshm\") pod \"test-model-ed4837-kserve-5d84f85887-7gc2j\" (UID: \"15d500f6-368d-4910-a932-96920bcc2e04\") " pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"
Aug 05 18:05:53.356524 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.356505    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15d500f6-368d-4910-a932-96920bcc2e04-tls-certs\") pod \"test-model-ed4837-kserve-5d84f85887-7gc2j\" (UID: \"15d500f6-368d-4910-a932-96920bcc2e04\") " pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"
Aug 05 18:05:53.362942 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.362922    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-542hp\" (UniqueName: \"kubernetes.io/projected/15d500f6-368d-4910-a932-96920bcc2e04-kube-api-access-542hp\") pod \"test-model-ed4837-kserve-5d84f85887-7gc2j\" (UID: \"15d500f6-368d-4910-a932-96920bcc2e04\") " pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"
Aug 05 18:05:53.471152 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.471073    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"
Aug 05 18:05:53.604388 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.604359    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"]
Aug 05 18:05:53.606319 ip-10-0-134-46 kubenswrapper[2550]: W0805 18:05:53.606295    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod15d500f6_368d_4910_a932_96920bcc2e04.slice/crio-30864a6c96d09ef999b4aa39c7411394631a7c34a4cffa9212f9f2474df439f6 WatchSource:0}: Error finding container 30864a6c96d09ef999b4aa39c7411394631a7c34a4cffa9212f9f2474df439f6: Status 404 returned error can't find the container with id 30864a6c96d09ef999b4aa39c7411394631a7c34a4cffa9212f9f2474df439f6
Aug 05 18:05:53.791276 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.791190    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j" event={"ID":"15d500f6-368d-4910-a932-96920bcc2e04","Type":"ContainerStarted","Data":"a994ca8d6c145f2d280ee232a7dd0588b246f4bf31d307cef055a5c0b3e35890"}
Aug 05 18:05:53.791276 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.791229    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j" event={"ID":"15d500f6-368d-4910-a932-96920bcc2e04","Type":"ContainerStarted","Data":"30864a6c96d09ef999b4aa39c7411394631a7c34a4cffa9212f9f2474df439f6"}
Aug 05 18:05:53.810815 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:05:53.810768    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j" podStartSLOduration=0.810754552 podStartE2EDuration="810.754552ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:05:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:05:53.809582829 +0000 UTC m=+2660.791160259" watchObservedRunningTime="2026-08-05 18:05:53.810754552 +0000 UTC m=+2660.792331980"
Aug 05 18:06:03.471177 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:03.471144    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"
Aug 05 18:06:03.471610 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:03.471194    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"
Aug 05 18:06:03.483597 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:03.483573    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"
Aug 05 18:06:03.850283 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:03.850203    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"
Aug 05 18:06:33.673973 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:33.673946    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 18:06:33.815308 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:33.815281    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 18:06:37.674185 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:37.674148    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"]
Aug 05 18:06:48.456656 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:48.456621    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95"]
Aug 05 18:06:48.457389 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:48.456902    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95" podUID="60f75d34-1c81-4b7e-9271-3be6d78c23bf" containerName="maas-api" containerID="cri-o://3ef17c4411199c34bc6f39514986aac3efc5ef5e0068520758d6d82743fc29c6" gracePeriod=30
Aug 05 18:06:48.516192 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:48.516156    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg"]
Aug 05 18:06:48.516472 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:48.516443    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg" podUID="9d8054b1-9a7f-4d8a-b878-5d8acd566fec" containerName="payload-processing" containerID="cri-o://f291247e98224283ea7c46bbb6963aaef8db6209efbeae04fdc37416406e1904" gracePeriod=30
Aug 05 18:06:48.527766 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:48.527163    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs"]
Aug 05 18:06:48.527766 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:48.527376    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs" podUID="31ec43b8-aa7c-4911-9db1-de6c22ee2e86" containerName="payload-pre-processing" containerID="cri-o://b4227991835df636990c0f78b3c3d704187c2cb5e9245c15edc1185841e6a949" gracePeriod=30
Aug 05 18:06:48.804585 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:48.804556    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg"
Aug 05 18:06:48.820504 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:48.820475    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs"
Aug 05 18:06:48.921137 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:48.921110    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95"
Aug 05 18:06:48.940247 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:48.940218    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8gj6p\" (UniqueName: \"kubernetes.io/projected/31ec43b8-aa7c-4911-9db1-de6c22ee2e86-kube-api-access-8gj6p\") pod \"31ec43b8-aa7c-4911-9db1-de6c22ee2e86\" (UID: \"31ec43b8-aa7c-4911-9db1-de6c22ee2e86\") "
Aug 05 18:06:48.940426 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:48.940257    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d8054b1-9a7f-4d8a-b878-5d8acd566fec-plugins-config-volume\") pod \"9d8054b1-9a7f-4d8a-b878-5d8acd566fec\" (UID: \"9d8054b1-9a7f-4d8a-b878-5d8acd566fec\") "
Aug 05 18:06:48.940426 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:48.940336    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31ec43b8-aa7c-4911-9db1-de6c22ee2e86-plugins-config-volume\") pod \"31ec43b8-aa7c-4911-9db1-de6c22ee2e86\" (UID: \"31ec43b8-aa7c-4911-9db1-de6c22ee2e86\") "
Aug 05 18:06:48.940426 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:48.940378    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7q558\" (UniqueName: \"kubernetes.io/projected/9d8054b1-9a7f-4d8a-b878-5d8acd566fec-kube-api-access-7q558\") pod \"9d8054b1-9a7f-4d8a-b878-5d8acd566fec\" (UID: \"9d8054b1-9a7f-4d8a-b878-5d8acd566fec\") "
Aug 05 18:06:48.941166 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:48.941136    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31ec43b8-aa7c-4911-9db1-de6c22ee2e86-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "31ec43b8-aa7c-4911-9db1-de6c22ee2e86" (UID: "31ec43b8-aa7c-4911-9db1-de6c22ee2e86"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:06:48.941844 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:48.941660    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d8054b1-9a7f-4d8a-b878-5d8acd566fec-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9d8054b1-9a7f-4d8a-b878-5d8acd566fec" (UID: "9d8054b1-9a7f-4d8a-b878-5d8acd566fec"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:06:48.942869 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:48.942848    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31ec43b8-aa7c-4911-9db1-de6c22ee2e86-kube-api-access-8gj6p" (OuterVolumeSpecName: "kube-api-access-8gj6p") pod "31ec43b8-aa7c-4911-9db1-de6c22ee2e86" (UID: "31ec43b8-aa7c-4911-9db1-de6c22ee2e86"). InnerVolumeSpecName "kube-api-access-8gj6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:06:48.942989 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:48.942963    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d8054b1-9a7f-4d8a-b878-5d8acd566fec-kube-api-access-7q558" (OuterVolumeSpecName: "kube-api-access-7q558") pod "9d8054b1-9a7f-4d8a-b878-5d8acd566fec" (UID: "9d8054b1-9a7f-4d8a-b878-5d8acd566fec"). InnerVolumeSpecName "kube-api-access-7q558". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:06:49.041758 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.041648    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60f75d34-1c81-4b7e-9271-3be6d78c23bf-maas-api-tls\") pod \"60f75d34-1c81-4b7e-9271-3be6d78c23bf\" (UID: \"60f75d34-1c81-4b7e-9271-3be6d78c23bf\") "
Aug 05 18:06:49.041758 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.041709    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60f75d34-1c81-4b7e-9271-3be6d78c23bf-openshift-service-ca\") pod \"60f75d34-1c81-4b7e-9271-3be6d78c23bf\" (UID: \"60f75d34-1c81-4b7e-9271-3be6d78c23bf\") "
Aug 05 18:06:49.041935 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.041774    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-669gc\" (UniqueName: \"kubernetes.io/projected/60f75d34-1c81-4b7e-9271-3be6d78c23bf-kube-api-access-669gc\") pod \"60f75d34-1c81-4b7e-9271-3be6d78c23bf\" (UID: \"60f75d34-1c81-4b7e-9271-3be6d78c23bf\") "
Aug 05 18:06:49.041976 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.041932    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7q558\" (UniqueName: \"kubernetes.io/projected/9d8054b1-9a7f-4d8a-b878-5d8acd566fec-kube-api-access-7q558\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:06:49.041976 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.041944    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gj6p\" (UniqueName: \"kubernetes.io/projected/31ec43b8-aa7c-4911-9db1-de6c22ee2e86-kube-api-access-8gj6p\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:06:49.041976 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.041956    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d8054b1-9a7f-4d8a-b878-5d8acd566fec-plugins-config-volume\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:06:49.041976 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.041965    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31ec43b8-aa7c-4911-9db1-de6c22ee2e86-plugins-config-volume\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:06:49.042129 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.042094    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60f75d34-1c81-4b7e-9271-3be6d78c23bf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "60f75d34-1c81-4b7e-9271-3be6d78c23bf" (UID: "60f75d34-1c81-4b7e-9271-3be6d78c23bf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:06:49.043849 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.043830    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60f75d34-1c81-4b7e-9271-3be6d78c23bf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "60f75d34-1c81-4b7e-9271-3be6d78c23bf" (UID: "60f75d34-1c81-4b7e-9271-3be6d78c23bf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:06:49.043849 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.043839    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60f75d34-1c81-4b7e-9271-3be6d78c23bf-kube-api-access-669gc" (OuterVolumeSpecName: "kube-api-access-669gc") pod "60f75d34-1c81-4b7e-9271-3be6d78c23bf" (UID: "60f75d34-1c81-4b7e-9271-3be6d78c23bf"). InnerVolumeSpecName "kube-api-access-669gc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:06:49.059953 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.059918    2550 generic.go:356] "Generic (PLEG): container finished" podID="31ec43b8-aa7c-4911-9db1-de6c22ee2e86" containerID="b4227991835df636990c0f78b3c3d704187c2cb5e9245c15edc1185841e6a949" exitCode=0
Aug 05 18:06:49.060132 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.059968    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs"
Aug 05 18:06:49.060132 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.060002    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs" event={"ID":"31ec43b8-aa7c-4911-9db1-de6c22ee2e86","Type":"ContainerDied","Data":"b4227991835df636990c0f78b3c3d704187c2cb5e9245c15edc1185841e6a949"}
Aug 05 18:06:49.060132 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.060076    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs" event={"ID":"31ec43b8-aa7c-4911-9db1-de6c22ee2e86","Type":"ContainerDied","Data":"6b1321aeca51147a2a63d0f9c04ff083d85f9f7d9d3b3e52044d3780c26dd2e0"}
Aug 05 18:06:49.060132 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.060126    2550 scope.go:117] "RemoveContainer" containerID="b4227991835df636990c0f78b3c3d704187c2cb5e9245c15edc1185841e6a949"
Aug 05 18:06:49.061363 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.061344    2550 generic.go:356] "Generic (PLEG): container finished" podID="9d8054b1-9a7f-4d8a-b878-5d8acd566fec" containerID="f291247e98224283ea7c46bbb6963aaef8db6209efbeae04fdc37416406e1904" exitCode=0
Aug 05 18:06:49.061461 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.061393    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg"
Aug 05 18:06:49.061461 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.061427    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg" event={"ID":"9d8054b1-9a7f-4d8a-b878-5d8acd566fec","Type":"ContainerDied","Data":"f291247e98224283ea7c46bbb6963aaef8db6209efbeae04fdc37416406e1904"}
Aug 05 18:06:49.061580 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.061475    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg" event={"ID":"9d8054b1-9a7f-4d8a-b878-5d8acd566fec","Type":"ContainerDied","Data":"204a4ccd8c0d5c75de1cb9e31ec128b660c910a260b3f7550271c41521f6ece3"}
Aug 05 18:06:49.063037 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.063008    2550 generic.go:356] "Generic (PLEG): container finished" podID="60f75d34-1c81-4b7e-9271-3be6d78c23bf" containerID="3ef17c4411199c34bc6f39514986aac3efc5ef5e0068520758d6d82743fc29c6" exitCode=0
Aug 05 18:06:49.063151 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.063050    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95"
Aug 05 18:06:49.063151 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.063077    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95" event={"ID":"60f75d34-1c81-4b7e-9271-3be6d78c23bf","Type":"ContainerDied","Data":"3ef17c4411199c34bc6f39514986aac3efc5ef5e0068520758d6d82743fc29c6"}
Aug 05 18:06:49.063151 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.063119    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95" event={"ID":"60f75d34-1c81-4b7e-9271-3be6d78c23bf","Type":"ContainerDied","Data":"7b4e21eb5db23046cc8b20c41035a7f108673d3184e5dd2c004e42215b823c1f"}
Aug 05 18:06:49.069876 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.069851    2550 scope.go:117] "RemoveContainer" containerID="b4227991835df636990c0f78b3c3d704187c2cb5e9245c15edc1185841e6a949"
Aug 05 18:06:49.070406 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:06:49.070161    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4227991835df636990c0f78b3c3d704187c2cb5e9245c15edc1185841e6a949\": container with ID starting with b4227991835df636990c0f78b3c3d704187c2cb5e9245c15edc1185841e6a949 not found: ID does not exist" containerID="b4227991835df636990c0f78b3c3d704187c2cb5e9245c15edc1185841e6a949"
Aug 05 18:06:49.070406 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.070190    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4227991835df636990c0f78b3c3d704187c2cb5e9245c15edc1185841e6a949"} err="failed to get container status \"b4227991835df636990c0f78b3c3d704187c2cb5e9245c15edc1185841e6a949\": rpc error: code = NotFound desc = could not find container \"b4227991835df636990c0f78b3c3d704187c2cb5e9245c15edc1185841e6a949\": container with ID starting with b4227991835df636990c0f78b3c3d704187c2cb5e9245c15edc1185841e6a949 not found: ID does not exist"
Aug 05 18:06:49.070406 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.070213    2550 scope.go:117] "RemoveContainer" containerID="f291247e98224283ea7c46bbb6963aaef8db6209efbeae04fdc37416406e1904"
Aug 05 18:06:49.078734 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.078715    2550 scope.go:117] "RemoveContainer" containerID="f291247e98224283ea7c46bbb6963aaef8db6209efbeae04fdc37416406e1904"
Aug 05 18:06:49.078986 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:06:49.078967    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f291247e98224283ea7c46bbb6963aaef8db6209efbeae04fdc37416406e1904\": container with ID starting with f291247e98224283ea7c46bbb6963aaef8db6209efbeae04fdc37416406e1904 not found: ID does not exist" containerID="f291247e98224283ea7c46bbb6963aaef8db6209efbeae04fdc37416406e1904"
Aug 05 18:06:49.079038 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.078992    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f291247e98224283ea7c46bbb6963aaef8db6209efbeae04fdc37416406e1904"} err="failed to get container status \"f291247e98224283ea7c46bbb6963aaef8db6209efbeae04fdc37416406e1904\": rpc error: code = NotFound desc = could not find container \"f291247e98224283ea7c46bbb6963aaef8db6209efbeae04fdc37416406e1904\": container with ID starting with f291247e98224283ea7c46bbb6963aaef8db6209efbeae04fdc37416406e1904 not found: ID does not exist"
Aug 05 18:06:49.079038 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.079010    2550 scope.go:117] "RemoveContainer" containerID="3ef17c4411199c34bc6f39514986aac3efc5ef5e0068520758d6d82743fc29c6"
Aug 05 18:06:49.085451 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.085427    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs"]
Aug 05 18:06:49.087343 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.087327    2550 scope.go:117] "RemoveContainer" containerID="3ef17c4411199c34bc6f39514986aac3efc5ef5e0068520758d6d82743fc29c6"
Aug 05 18:06:49.087626 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:06:49.087603    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ef17c4411199c34bc6f39514986aac3efc5ef5e0068520758d6d82743fc29c6\": container with ID starting with 3ef17c4411199c34bc6f39514986aac3efc5ef5e0068520758d6d82743fc29c6 not found: ID does not exist" containerID="3ef17c4411199c34bc6f39514986aac3efc5ef5e0068520758d6d82743fc29c6"
Aug 05 18:06:49.087721 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.087635    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ef17c4411199c34bc6f39514986aac3efc5ef5e0068520758d6d82743fc29c6"} err="failed to get container status \"3ef17c4411199c34bc6f39514986aac3efc5ef5e0068520758d6d82743fc29c6\": rpc error: code = NotFound desc = could not find container \"3ef17c4411199c34bc6f39514986aac3efc5ef5e0068520758d6d82743fc29c6\": container with ID starting with 3ef17c4411199c34bc6f39514986aac3efc5ef5e0068520758d6d82743fc29c6 not found: ID does not exist"
Aug 05 18:06:49.089181 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.089163    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-83d598-9c4f8b88d-r27zs"]
Aug 05 18:06:49.101562 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.101539    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg"]
Aug 05 18:06:49.106275 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.106255    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-83d598-bdbb878fb-24qtg"]
Aug 05 18:06:49.119107 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.119083    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95"]
Aug 05 18:06:49.124057 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.124036    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-83d598-6b998f45f8-m5w95"]
Aug 05 18:06:49.142711 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.142675    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60f75d34-1c81-4b7e-9271-3be6d78c23bf-maas-api-tls\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:06:49.142837 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.142728    2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60f75d34-1c81-4b7e-9271-3be6d78c23bf-openshift-service-ca\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:06:49.142837 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.142740    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-669gc\" (UniqueName: \"kubernetes.io/projected/60f75d34-1c81-4b7e-9271-3be6d78c23bf-kube-api-access-669gc\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:06:49.701175 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.701141    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31ec43b8-aa7c-4911-9db1-de6c22ee2e86" path="/var/lib/kubelet/pods/31ec43b8-aa7c-4911-9db1-de6c22ee2e86/volumes"
Aug 05 18:06:49.701543 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.701511    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60f75d34-1c81-4b7e-9271-3be6d78c23bf" path="/var/lib/kubelet/pods/60f75d34-1c81-4b7e-9271-3be6d78c23bf/volumes"
Aug 05 18:06:49.701835 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:49.701822    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d8054b1-9a7f-4d8a-b878-5d8acd566fec" path="/var/lib/kubelet/pods/9d8054b1-9a7f-4d8a-b878-5d8acd566fec/volumes"
Aug 05 18:06:52.693120 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:52.693074    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j" podUID="15d500f6-368d-4910-a932-96920bcc2e04" containerName="main" containerID="cri-o://a994ca8d6c145f2d280ee232a7dd0588b246f4bf31d307cef055a5c0b3e35890" gracePeriod=45
Aug 05 18:06:52.937507 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:52.937483    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"
Aug 05 18:06:53.079271 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:53.079173    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15d500f6-368d-4910-a932-96920bcc2e04-tmp-dir\") pod \"15d500f6-368d-4910-a932-96920bcc2e04\" (UID: \"15d500f6-368d-4910-a932-96920bcc2e04\") "
Aug 05 18:06:53.079271 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:53.079252    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15d500f6-368d-4910-a932-96920bcc2e04-dshm\") pod \"15d500f6-368d-4910-a932-96920bcc2e04\" (UID: \"15d500f6-368d-4910-a932-96920bcc2e04\") "
Aug 05 18:06:53.079490 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:53.079286    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15d500f6-368d-4910-a932-96920bcc2e04-tls-certs\") pod \"15d500f6-368d-4910-a932-96920bcc2e04\" (UID: \"15d500f6-368d-4910-a932-96920bcc2e04\") "
Aug 05 18:06:53.079490 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:53.079321    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15d500f6-368d-4910-a932-96920bcc2e04-model-cache\") pod \"15d500f6-368d-4910-a932-96920bcc2e04\" (UID: \"15d500f6-368d-4910-a932-96920bcc2e04\") "
Aug 05 18:06:53.079490 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:53.079384    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15d500f6-368d-4910-a932-96920bcc2e04-home\") pod \"15d500f6-368d-4910-a932-96920bcc2e04\" (UID: \"15d500f6-368d-4910-a932-96920bcc2e04\") "
Aug 05 18:06:53.079490 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:53.079410    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-542hp\" (UniqueName: \"kubernetes.io/projected/15d500f6-368d-4910-a932-96920bcc2e04-kube-api-access-542hp\") pod \"15d500f6-368d-4910-a932-96920bcc2e04\" (UID: \"15d500f6-368d-4910-a932-96920bcc2e04\") "
Aug 05 18:06:53.079742 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:53.079527    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15d500f6-368d-4910-a932-96920bcc2e04-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "15d500f6-368d-4910-a932-96920bcc2e04" (UID: "15d500f6-368d-4910-a932-96920bcc2e04"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:06:53.079742 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:53.079607    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15d500f6-368d-4910-a932-96920bcc2e04-model-cache" (OuterVolumeSpecName: "model-cache") pod "15d500f6-368d-4910-a932-96920bcc2e04" (UID: "15d500f6-368d-4910-a932-96920bcc2e04"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:06:53.079742 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:53.079686    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15d500f6-368d-4910-a932-96920bcc2e04-home" (OuterVolumeSpecName: "home") pod "15d500f6-368d-4910-a932-96920bcc2e04" (UID: "15d500f6-368d-4910-a932-96920bcc2e04"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:06:53.079906 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:53.079785    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15d500f6-368d-4910-a932-96920bcc2e04-tmp-dir\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:06:53.079906 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:53.079808    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15d500f6-368d-4910-a932-96920bcc2e04-model-cache\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:06:53.082684 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:53.082018    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15d500f6-368d-4910-a932-96920bcc2e04-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "15d500f6-368d-4910-a932-96920bcc2e04" (UID: "15d500f6-368d-4910-a932-96920bcc2e04"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:06:53.082684 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:53.082430    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15d500f6-368d-4910-a932-96920bcc2e04-dshm" (OuterVolumeSpecName: "dshm") pod "15d500f6-368d-4910-a932-96920bcc2e04" (UID: "15d500f6-368d-4910-a932-96920bcc2e04"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:06:53.082684 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:53.082443    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15d500f6-368d-4910-a932-96920bcc2e04-kube-api-access-542hp" (OuterVolumeSpecName: "kube-api-access-542hp") pod "15d500f6-368d-4910-a932-96920bcc2e04" (UID: "15d500f6-368d-4910-a932-96920bcc2e04"). InnerVolumeSpecName "kube-api-access-542hp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:06:53.088152 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:53.088124    2550 generic.go:356] "Generic (PLEG): container finished" podID="15d500f6-368d-4910-a932-96920bcc2e04" containerID="a994ca8d6c145f2d280ee232a7dd0588b246f4bf31d307cef055a5c0b3e35890" exitCode=0
Aug 05 18:06:53.088274 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:53.088170    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"
Aug 05 18:06:53.088274 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:53.088214    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j" event={"ID":"15d500f6-368d-4910-a932-96920bcc2e04","Type":"ContainerDied","Data":"a994ca8d6c145f2d280ee232a7dd0588b246f4bf31d307cef055a5c0b3e35890"}
Aug 05 18:06:53.088274 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:53.088256    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j" event={"ID":"15d500f6-368d-4910-a932-96920bcc2e04","Type":"ContainerDied","Data":"30864a6c96d09ef999b4aa39c7411394631a7c34a4cffa9212f9f2474df439f6"}
Aug 05 18:06:53.088274 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:53.088273    2550 scope.go:117] "RemoveContainer" containerID="a994ca8d6c145f2d280ee232a7dd0588b246f4bf31d307cef055a5c0b3e35890"
Aug 05 18:06:53.100593 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:53.100574    2550 scope.go:117] "RemoveContainer" containerID="a994ca8d6c145f2d280ee232a7dd0588b246f4bf31d307cef055a5c0b3e35890"
Aug 05 18:06:53.100870 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:06:53.100849    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a994ca8d6c145f2d280ee232a7dd0588b246f4bf31d307cef055a5c0b3e35890\": container with ID starting with a994ca8d6c145f2d280ee232a7dd0588b246f4bf31d307cef055a5c0b3e35890 not found: ID does not exist" containerID="a994ca8d6c145f2d280ee232a7dd0588b246f4bf31d307cef055a5c0b3e35890"
Aug 05 18:06:53.100938 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:53.100878    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a994ca8d6c145f2d280ee232a7dd0588b246f4bf31d307cef055a5c0b3e35890"} err="failed to get container status \"a994ca8d6c145f2d280ee232a7dd0588b246f4bf31d307cef055a5c0b3e35890\": rpc error: code = NotFound desc = could not find container \"a994ca8d6c145f2d280ee232a7dd0588b246f4bf31d307cef055a5c0b3e35890\": container with ID starting with a994ca8d6c145f2d280ee232a7dd0588b246f4bf31d307cef055a5c0b3e35890 not found: ID does not exist"
Aug 05 18:06:53.112246 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:53.112223    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"]
Aug 05 18:06:53.118100 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:53.118078    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-ed4837/test-model-ed4837-kserve-5d84f85887-7gc2j"]
Aug 05 18:06:53.180324 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:53.180303    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15d500f6-368d-4910-a932-96920bcc2e04-home\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:06:53.180324 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:53.180321    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-542hp\" (UniqueName: \"kubernetes.io/projected/15d500f6-368d-4910-a932-96920bcc2e04-kube-api-access-542hp\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:06:53.180461 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:53.180332    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15d500f6-368d-4910-a932-96920bcc2e04-dshm\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:06:53.180461 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:53.180342    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15d500f6-368d-4910-a932-96920bcc2e04-tls-certs\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:06:53.700755 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:06:53.700722    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15d500f6-368d-4910-a932-96920bcc2e04" path="/var/lib/kubelet/pods/15d500f6-368d-4910-a932-96920bcc2e04/volumes"
Aug 05 18:07:06.358980 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:06.358950    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb"]
Aug 05 18:07:06.359461 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:06.359209    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb" podUID="5db07b76-3930-4d67-b118-c8306eba26f7" containerName="maas-api" containerID="cri-o://b2f8a432137a34c5f4681c77212a1f927cab5d2e74db449d035a0b7a0e7bcaef" gracePeriod=30
Aug 05 18:07:06.407980 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:06.407951    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb"]
Aug 05 18:07:06.408212 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:06.408174    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb" podUID="81dc4899-8951-4289-b661-9da6c7db7f90" containerName="payload-processing" containerID="cri-o://c2a3be94dc84ee76c8320910ddfab53323d07a4c58fa67221f0a04e026789050" gracePeriod=30
Aug 05 18:07:06.412285 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:06.412249    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv"]
Aug 05 18:07:06.412795 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:06.412689    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv" podUID="9673bf92-fcd6-4901-9d11-971f8d05258b" containerName="payload-pre-processing" containerID="cri-o://2c700e870b38b1d5057cf4a96db502ea93cdcb7f9b6f4c93b3c27672d1c05920" gracePeriod=30
Aug 05 18:07:06.682103 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:06.682073    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb"
Aug 05 18:07:06.728168 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:06.728144    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv"
Aug 05 18:07:06.741637 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:06.741615    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb"
Aug 05 18:07:06.804395 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:06.804364    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5db07b76-3930-4d67-b118-c8306eba26f7-openshift-service-ca\") pod \"5db07b76-3930-4d67-b118-c8306eba26f7\" (UID: \"5db07b76-3930-4d67-b118-c8306eba26f7\") "
Aug 05 18:07:06.804552 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:06.804455    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5db07b76-3930-4d67-b118-c8306eba26f7-maas-api-tls\") pod \"5db07b76-3930-4d67-b118-c8306eba26f7\" (UID: \"5db07b76-3930-4d67-b118-c8306eba26f7\") "
Aug 05 18:07:06.804552 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:06.804481    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgkkf\" (UniqueName: \"kubernetes.io/projected/5db07b76-3930-4d67-b118-c8306eba26f7-kube-api-access-kgkkf\") pod \"5db07b76-3930-4d67-b118-c8306eba26f7\" (UID: \"5db07b76-3930-4d67-b118-c8306eba26f7\") "
Aug 05 18:07:06.804779 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:06.804756    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5db07b76-3930-4d67-b118-c8306eba26f7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5db07b76-3930-4d67-b118-c8306eba26f7" (UID: "5db07b76-3930-4d67-b118-c8306eba26f7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:07:06.806462 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:06.806433    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5db07b76-3930-4d67-b118-c8306eba26f7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5db07b76-3930-4d67-b118-c8306eba26f7" (UID: "5db07b76-3930-4d67-b118-c8306eba26f7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:07:06.806462 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:06.806446    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5db07b76-3930-4d67-b118-c8306eba26f7-kube-api-access-kgkkf" (OuterVolumeSpecName: "kube-api-access-kgkkf") pod "5db07b76-3930-4d67-b118-c8306eba26f7" (UID: "5db07b76-3930-4d67-b118-c8306eba26f7"). InnerVolumeSpecName "kube-api-access-kgkkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:07:06.905228 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:06.905200    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tffhd\" (UniqueName: \"kubernetes.io/projected/9673bf92-fcd6-4901-9d11-971f8d05258b-kube-api-access-tffhd\") pod \"9673bf92-fcd6-4901-9d11-971f8d05258b\" (UID: \"9673bf92-fcd6-4901-9d11-971f8d05258b\") "
Aug 05 18:07:06.905405 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:06.905274    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81dc4899-8951-4289-b661-9da6c7db7f90-plugins-config-volume\") pod \"81dc4899-8951-4289-b661-9da6c7db7f90\" (UID: \"81dc4899-8951-4289-b661-9da6c7db7f90\") "
Aug 05 18:07:06.905405 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:06.905308    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9673bf92-fcd6-4901-9d11-971f8d05258b-plugins-config-volume\") pod \"9673bf92-fcd6-4901-9d11-971f8d05258b\" (UID: \"9673bf92-fcd6-4901-9d11-971f8d05258b\") "
Aug 05 18:07:06.905405 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:06.905343    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tb6v6\" (UniqueName: \"kubernetes.io/projected/81dc4899-8951-4289-b661-9da6c7db7f90-kube-api-access-tb6v6\") pod \"81dc4899-8951-4289-b661-9da6c7db7f90\" (UID: \"81dc4899-8951-4289-b661-9da6c7db7f90\") "
Aug 05 18:07:06.905601 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:06.905586    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5db07b76-3930-4d67-b118-c8306eba26f7-maas-api-tls\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:07:06.905657 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:06.905607    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgkkf\" (UniqueName: \"kubernetes.io/projected/5db07b76-3930-4d67-b118-c8306eba26f7-kube-api-access-kgkkf\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:07:06.905657 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:06.905622    2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5db07b76-3930-4d67-b118-c8306eba26f7-openshift-service-ca\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:07:06.905744 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:06.905723    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9673bf92-fcd6-4901-9d11-971f8d05258b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9673bf92-fcd6-4901-9d11-971f8d05258b" (UID: "9673bf92-fcd6-4901-9d11-971f8d05258b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:07:06.905781 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:06.905721    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81dc4899-8951-4289-b661-9da6c7db7f90-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "81dc4899-8951-4289-b661-9da6c7db7f90" (UID: "81dc4899-8951-4289-b661-9da6c7db7f90"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:07:06.907328 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:06.907306    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9673bf92-fcd6-4901-9d11-971f8d05258b-kube-api-access-tffhd" (OuterVolumeSpecName: "kube-api-access-tffhd") pod "9673bf92-fcd6-4901-9d11-971f8d05258b" (UID: "9673bf92-fcd6-4901-9d11-971f8d05258b"). InnerVolumeSpecName "kube-api-access-tffhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:07:06.907492 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:06.907471    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81dc4899-8951-4289-b661-9da6c7db7f90-kube-api-access-tb6v6" (OuterVolumeSpecName: "kube-api-access-tb6v6") pod "81dc4899-8951-4289-b661-9da6c7db7f90" (UID: "81dc4899-8951-4289-b661-9da6c7db7f90"). InnerVolumeSpecName "kube-api-access-tb6v6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:07:07.006168 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.006138    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tffhd\" (UniqueName: \"kubernetes.io/projected/9673bf92-fcd6-4901-9d11-971f8d05258b-kube-api-access-tffhd\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:07:07.006168 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.006162    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81dc4899-8951-4289-b661-9da6c7db7f90-plugins-config-volume\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:07:07.006168 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.006172    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9673bf92-fcd6-4901-9d11-971f8d05258b-plugins-config-volume\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:07:07.006390 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.006180    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tb6v6\" (UniqueName: \"kubernetes.io/projected/81dc4899-8951-4289-b661-9da6c7db7f90-kube-api-access-tb6v6\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:07:07.160637 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.160560    2550 generic.go:356] "Generic (PLEG): container finished" podID="81dc4899-8951-4289-b661-9da6c7db7f90" containerID="c2a3be94dc84ee76c8320910ddfab53323d07a4c58fa67221f0a04e026789050" exitCode=0
Aug 05 18:07:07.160637 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.160604    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb"
Aug 05 18:07:07.160892 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.160642    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb" event={"ID":"81dc4899-8951-4289-b661-9da6c7db7f90","Type":"ContainerDied","Data":"c2a3be94dc84ee76c8320910ddfab53323d07a4c58fa67221f0a04e026789050"}
Aug 05 18:07:07.160892 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.160717    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb" event={"ID":"81dc4899-8951-4289-b661-9da6c7db7f90","Type":"ContainerDied","Data":"129edf93962f119f5a8d4f93df9463b2c2f5f48cb12cd5d19d060d0668e6ebae"}
Aug 05 18:07:07.160892 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.160741    2550 scope.go:117] "RemoveContainer" containerID="c2a3be94dc84ee76c8320910ddfab53323d07a4c58fa67221f0a04e026789050"
Aug 05 18:07:07.161839 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.161820    2550 generic.go:356] "Generic (PLEG): container finished" podID="5db07b76-3930-4d67-b118-c8306eba26f7" containerID="b2f8a432137a34c5f4681c77212a1f927cab5d2e74db449d035a0b7a0e7bcaef" exitCode=0
Aug 05 18:07:07.161890 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.161866    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb"
Aug 05 18:07:07.161936 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.161912    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb" event={"ID":"5db07b76-3930-4d67-b118-c8306eba26f7","Type":"ContainerDied","Data":"b2f8a432137a34c5f4681c77212a1f927cab5d2e74db449d035a0b7a0e7bcaef"}
Aug 05 18:07:07.161991 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.161956    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb" event={"ID":"5db07b76-3930-4d67-b118-c8306eba26f7","Type":"ContainerDied","Data":"61d5c381a3d2cd0bc111c0f2eb17df7aa3d6f73e34bdde2412896c8e31ac218e"}
Aug 05 18:07:07.163195 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.163172    2550 generic.go:356] "Generic (PLEG): container finished" podID="9673bf92-fcd6-4901-9d11-971f8d05258b" containerID="2c700e870b38b1d5057cf4a96db502ea93cdcb7f9b6f4c93b3c27672d1c05920" exitCode=0
Aug 05 18:07:07.163317 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.163216    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv"
Aug 05 18:07:07.163317 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.163218    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv" event={"ID":"9673bf92-fcd6-4901-9d11-971f8d05258b","Type":"ContainerDied","Data":"2c700e870b38b1d5057cf4a96db502ea93cdcb7f9b6f4c93b3c27672d1c05920"}
Aug 05 18:07:07.163417 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.163320    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv" event={"ID":"9673bf92-fcd6-4901-9d11-971f8d05258b","Type":"ContainerDied","Data":"b2f20aca888c1dc7818cb9e3542ed47340cd8e7f480a2aece448fb96e1be1c47"}
Aug 05 18:07:07.170218 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.170196    2550 scope.go:117] "RemoveContainer" containerID="c2a3be94dc84ee76c8320910ddfab53323d07a4c58fa67221f0a04e026789050"
Aug 05 18:07:07.170465 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:07:07.170448    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2a3be94dc84ee76c8320910ddfab53323d07a4c58fa67221f0a04e026789050\": container with ID starting with c2a3be94dc84ee76c8320910ddfab53323d07a4c58fa67221f0a04e026789050 not found: ID does not exist" containerID="c2a3be94dc84ee76c8320910ddfab53323d07a4c58fa67221f0a04e026789050"
Aug 05 18:07:07.170537 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.170472    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2a3be94dc84ee76c8320910ddfab53323d07a4c58fa67221f0a04e026789050"} err="failed to get container status \"c2a3be94dc84ee76c8320910ddfab53323d07a4c58fa67221f0a04e026789050\": rpc error: code = NotFound desc = could not find container \"c2a3be94dc84ee76c8320910ddfab53323d07a4c58fa67221f0a04e026789050\": container with ID starting with c2a3be94dc84ee76c8320910ddfab53323d07a4c58fa67221f0a04e026789050 not found: ID does not exist"
Aug 05 18:07:07.170537 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.170487    2550 scope.go:117] "RemoveContainer" containerID="b2f8a432137a34c5f4681c77212a1f927cab5d2e74db449d035a0b7a0e7bcaef"
Aug 05 18:07:07.179095 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.179082    2550 scope.go:117] "RemoveContainer" containerID="b2f8a432137a34c5f4681c77212a1f927cab5d2e74db449d035a0b7a0e7bcaef"
Aug 05 18:07:07.179294 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:07:07.179273    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2f8a432137a34c5f4681c77212a1f927cab5d2e74db449d035a0b7a0e7bcaef\": container with ID starting with b2f8a432137a34c5f4681c77212a1f927cab5d2e74db449d035a0b7a0e7bcaef not found: ID does not exist" containerID="b2f8a432137a34c5f4681c77212a1f927cab5d2e74db449d035a0b7a0e7bcaef"
Aug 05 18:07:07.179354 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.179300    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2f8a432137a34c5f4681c77212a1f927cab5d2e74db449d035a0b7a0e7bcaef"} err="failed to get container status \"b2f8a432137a34c5f4681c77212a1f927cab5d2e74db449d035a0b7a0e7bcaef\": rpc error: code = NotFound desc = could not find container \"b2f8a432137a34c5f4681c77212a1f927cab5d2e74db449d035a0b7a0e7bcaef\": container with ID starting with b2f8a432137a34c5f4681c77212a1f927cab5d2e74db449d035a0b7a0e7bcaef not found: ID does not exist"
Aug 05 18:07:07.179354 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.179315    2550 scope.go:117] "RemoveContainer" containerID="2c700e870b38b1d5057cf4a96db502ea93cdcb7f9b6f4c93b3c27672d1c05920"
Aug 05 18:07:07.186547 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.186482    2550 scope.go:117] "RemoveContainer" containerID="2c700e870b38b1d5057cf4a96db502ea93cdcb7f9b6f4c93b3c27672d1c05920"
Aug 05 18:07:07.186857 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:07:07.186832    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c700e870b38b1d5057cf4a96db502ea93cdcb7f9b6f4c93b3c27672d1c05920\": container with ID starting with 2c700e870b38b1d5057cf4a96db502ea93cdcb7f9b6f4c93b3c27672d1c05920 not found: ID does not exist" containerID="2c700e870b38b1d5057cf4a96db502ea93cdcb7f9b6f4c93b3c27672d1c05920"
Aug 05 18:07:07.186965 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.186863    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c700e870b38b1d5057cf4a96db502ea93cdcb7f9b6f4c93b3c27672d1c05920"} err="failed to get container status \"2c700e870b38b1d5057cf4a96db502ea93cdcb7f9b6f4c93b3c27672d1c05920\": rpc error: code = NotFound desc = could not find container \"2c700e870b38b1d5057cf4a96db502ea93cdcb7f9b6f4c93b3c27672d1c05920\": container with ID starting with 2c700e870b38b1d5057cf4a96db502ea93cdcb7f9b6f4c93b3c27672d1c05920 not found: ID does not exist"
Aug 05 18:07:07.188823 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.188805    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv"]
Aug 05 18:07:07.192168 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.192143    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ed4837-5c4c96b7fb-82lbv"]
Aug 05 18:07:07.201771 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.201753    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb"]
Aug 05 18:07:07.205849 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.205829    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ed4837-cc989956b-x9whb"]
Aug 05 18:07:07.217888 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.217870    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb"]
Aug 05 18:07:07.221501 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.221484    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ed4837-ddc86f7fc-9fcgb"]
Aug 05 18:07:07.699777 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.699744    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5db07b76-3930-4d67-b118-c8306eba26f7" path="/var/lib/kubelet/pods/5db07b76-3930-4d67-b118-c8306eba26f7/volumes"
Aug 05 18:07:07.700155 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.700095    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81dc4899-8951-4289-b661-9da6c7db7f90" path="/var/lib/kubelet/pods/81dc4899-8951-4289-b661-9da6c7db7f90/volumes"
Aug 05 18:07:07.700397 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:07.700383    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9673bf92-fcd6-4901-9d11-971f8d05258b" path="/var/lib/kubelet/pods/9673bf92-fcd6-4901-9d11-971f8d05258b/volumes"
Aug 05 18:07:14.464279 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.464248    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"]
Aug 05 18:07:14.464804 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.464575    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31ec43b8-aa7c-4911-9db1-de6c22ee2e86" containerName="payload-pre-processing"
Aug 05 18:07:14.464804 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.464586    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="31ec43b8-aa7c-4911-9db1-de6c22ee2e86" containerName="payload-pre-processing"
Aug 05 18:07:14.464804 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.464602    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15d500f6-368d-4910-a932-96920bcc2e04" containerName="main"
Aug 05 18:07:14.464804 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.464607    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="15d500f6-368d-4910-a932-96920bcc2e04" containerName="main"
Aug 05 18:07:14.464804 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.464623    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9673bf92-fcd6-4901-9d11-971f8d05258b" containerName="payload-pre-processing"
Aug 05 18:07:14.464804 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.464628    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="9673bf92-fcd6-4901-9d11-971f8d05258b" containerName="payload-pre-processing"
Aug 05 18:07:14.464804 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.464633    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5db07b76-3930-4d67-b118-c8306eba26f7" containerName="maas-api"
Aug 05 18:07:14.464804 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.464639    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="5db07b76-3930-4d67-b118-c8306eba26f7" containerName="maas-api"
Aug 05 18:07:14.464804 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.464644    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d8054b1-9a7f-4d8a-b878-5d8acd566fec" containerName="payload-processing"
Aug 05 18:07:14.464804 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.464649    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d8054b1-9a7f-4d8a-b878-5d8acd566fec" containerName="payload-processing"
Aug 05 18:07:14.464804 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.464657    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60f75d34-1c81-4b7e-9271-3be6d78c23bf" containerName="maas-api"
Aug 05 18:07:14.464804 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.464662    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="60f75d34-1c81-4b7e-9271-3be6d78c23bf" containerName="maas-api"
Aug 05 18:07:14.464804 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.464668    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81dc4899-8951-4289-b661-9da6c7db7f90" containerName="payload-processing"
Aug 05 18:07:14.464804 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.464673    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="81dc4899-8951-4289-b661-9da6c7db7f90" containerName="payload-processing"
Aug 05 18:07:14.464804 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.464766    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d8054b1-9a7f-4d8a-b878-5d8acd566fec" containerName="payload-processing"
Aug 05 18:07:14.464804 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.464780    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="31ec43b8-aa7c-4911-9db1-de6c22ee2e86" containerName="payload-pre-processing"
Aug 05 18:07:14.464804 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.464791    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="81dc4899-8951-4289-b661-9da6c7db7f90" containerName="payload-processing"
Aug 05 18:07:14.464804 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.464801    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="9673bf92-fcd6-4901-9d11-971f8d05258b" containerName="payload-pre-processing"
Aug 05 18:07:14.464804 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.464817    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="60f75d34-1c81-4b7e-9271-3be6d78c23bf" containerName="maas-api"
Aug 05 18:07:14.465823 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.464829    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="5db07b76-3930-4d67-b118-c8306eba26f7" containerName="maas-api"
Aug 05 18:07:14.465823 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.464838    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="15d500f6-368d-4910-a932-96920bcc2e04" containerName="main"
Aug 05 18:07:14.467807 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.467788    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.470216 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.470193    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-b0e02a-openshift-default-dockercfg-whmn7\""
Aug 05 18:07:14.480376 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.480350    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"]
Aug 05 18:07:14.568545 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.568512    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-workload-certs\") pod \"e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") " pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.568795 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.568574    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-istio-envoy\") pod \"e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") " pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.568795 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.568646    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2td47\" (UniqueName: \"kubernetes.io/projected/9cebbe6e-049e-4027-9449-bbf2ca345297-kube-api-access-2td47\") pod \"e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") " pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.568795 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.568730    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cebbe6e-049e-4027-9449-bbf2ca345297-istio-token\") pod \"e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") " pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.569000 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.568815    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-credential-socket\") pod \"e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") " pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.569000 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.568860    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-workload-socket\") pod \"e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") " pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.569000 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.568891    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cebbe6e-049e-4027-9449-bbf2ca345297-istiod-ca-cert\") pod \"e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") " pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.569000 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.568934    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-istio-data\") pod \"e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") " pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.569000 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.568967    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cebbe6e-049e-4027-9449-bbf2ca345297-istio-podinfo\") pod \"e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") " pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.670232 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.670202    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-workload-certs\") pod \"e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") " pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.670407 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.670248    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-istio-envoy\") pod \"e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") " pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.670407 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.670274    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2td47\" (UniqueName: \"kubernetes.io/projected/9cebbe6e-049e-4027-9449-bbf2ca345297-kube-api-access-2td47\") pod \"e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") " pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.670407 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.670315    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cebbe6e-049e-4027-9449-bbf2ca345297-istio-token\") pod \"e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") " pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.670407 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.670336    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-credential-socket\") pod \"e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") " pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.670407 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.670360    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-workload-socket\") pod \"e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") " pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.670407 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.670375    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cebbe6e-049e-4027-9449-bbf2ca345297-istiod-ca-cert\") pod \"e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") " pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.670771 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.670408    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-istio-data\") pod \"e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") " pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.670771 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.670444    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cebbe6e-049e-4027-9449-bbf2ca345297-istio-podinfo\") pod \"e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") " pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.670771 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.670610    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-workload-certs\") pod \"e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") " pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.670934 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.670800    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-workload-socket\") pod \"e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") " pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.670934 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.670817    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-istio-data\") pod \"e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") " pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.671044 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.670975    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-credential-socket\") pod \"e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") " pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.671169 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.671148    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cebbe6e-049e-4027-9449-bbf2ca345297-istiod-ca-cert\") pod \"e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") " pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.672507 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.672487    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-istio-envoy\") pod \"e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") " pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.672850 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.672832    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cebbe6e-049e-4027-9449-bbf2ca345297-istio-podinfo\") pod \"e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") " pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.678077 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.678048    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cebbe6e-049e-4027-9449-bbf2ca345297-istio-token\") pod \"e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") " pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.678171 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.678122    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2td47\" (UniqueName: \"kubernetes.io/projected/9cebbe6e-049e-4027-9449-bbf2ca345297-kube-api-access-2td47\") pod \"e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") " pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.778362 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.778273    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:14.915973 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.915940    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"]
Aug 05 18:07:14.918577 ip-10-0-134-46 kubenswrapper[2550]: W0805 18:07:14.918547    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cebbe6e_049e_4027_9449_bbf2ca345297.slice/crio-e6eafe9644ffcdfbd30d2db05b67a142f37ca59345778b67c1e4954a95ae8d6a WatchSource:0}: Error finding container e6eafe9644ffcdfbd30d2db05b67a142f37ca59345778b67c1e4954a95ae8d6a: Status 404 returned error can't find the container with id e6eafe9644ffcdfbd30d2db05b67a142f37ca59345778b67c1e4954a95ae8d6a
Aug 05 18:07:14.920971 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.920931    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 18:07:14.921051 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.921016    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 18:07:14.921090 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:14.921057    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 18:07:15.205960 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:15.205927    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb" event={"ID":"9cebbe6e-049e-4027-9449-bbf2ca345297","Type":"ContainerStarted","Data":"839d1fd9024bc5e05f1e1a7aa273a9a978746186e2fc6f8110f29756e6d113ee"}
Aug 05 18:07:15.206155 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:15.205985    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb" event={"ID":"9cebbe6e-049e-4027-9449-bbf2ca345297","Type":"ContainerStarted","Data":"e6eafe9644ffcdfbd30d2db05b67a142f37ca59345778b67c1e4954a95ae8d6a"}
Aug 05 18:07:15.226726 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:15.226667    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb" podStartSLOduration=1.226650857 podStartE2EDuration="1.226650857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:07:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:07:15.225737909 +0000 UTC m=+2742.207315340" watchObservedRunningTime="2026-08-05 18:07:15.226650857 +0000 UTC m=+2742.208228286"
Aug 05 18:07:15.778723 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:15.778673    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:15.783486 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:15.783459    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:16.210878 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:16.210845    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:16.211884 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:16.211853    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:18.383717 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:18.383668    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd"]
Aug 05 18:07:18.387278 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:18.387256    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd"
Aug 05 18:07:18.389427 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:18.389408    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-b0e02a-serving-cert\""
Aug 05 18:07:18.397461 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:18.397440    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd"]
Aug 05 18:07:18.503332 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:18.503301    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a54206a-c639-435a-92ec-ca9dcd879f32-openshift-service-ca\") pod \"maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd\" (UID: \"5a54206a-c639-435a-92ec-ca9dcd879f32\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd"
Aug 05 18:07:18.503499 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:18.503347    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a54206a-c639-435a-92ec-ca9dcd879f32-maas-api-tls\") pod \"maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd\" (UID: \"5a54206a-c639-435a-92ec-ca9dcd879f32\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd"
Aug 05 18:07:18.503499 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:18.503429    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d794\" (UniqueName: \"kubernetes.io/projected/5a54206a-c639-435a-92ec-ca9dcd879f32-kube-api-access-4d794\") pod \"maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd\" (UID: \"5a54206a-c639-435a-92ec-ca9dcd879f32\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd"
Aug 05 18:07:18.604667 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:18.604642    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d794\" (UniqueName: \"kubernetes.io/projected/5a54206a-c639-435a-92ec-ca9dcd879f32-kube-api-access-4d794\") pod \"maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd\" (UID: \"5a54206a-c639-435a-92ec-ca9dcd879f32\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd"
Aug 05 18:07:18.604837 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:18.604693    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a54206a-c639-435a-92ec-ca9dcd879f32-openshift-service-ca\") pod \"maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd\" (UID: \"5a54206a-c639-435a-92ec-ca9dcd879f32\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd"
Aug 05 18:07:18.604837 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:18.604760    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a54206a-c639-435a-92ec-ca9dcd879f32-maas-api-tls\") pod \"maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd\" (UID: \"5a54206a-c639-435a-92ec-ca9dcd879f32\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd"
Aug 05 18:07:18.604916 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:07:18.604858    2550 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-serving-cert: secret "maas-api-e2e-api-a-b0e02a-serving-cert" not found
Aug 05 18:07:18.604953 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:07:18.604916    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a54206a-c639-435a-92ec-ca9dcd879f32-maas-api-tls podName:5a54206a-c639-435a-92ec-ca9dcd879f32 nodeName:}" failed. No retries permitted until 2026-08-05 18:07:19.104899287 +0000 UTC m=+2746.086476699 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5a54206a-c639-435a-92ec-ca9dcd879f32-maas-api-tls") pod "maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd" (UID: "5a54206a-c639-435a-92ec-ca9dcd879f32") : secret "maas-api-e2e-api-a-b0e02a-serving-cert" not found
Aug 05 18:07:18.605255 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:18.605237    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a54206a-c639-435a-92ec-ca9dcd879f32-openshift-service-ca\") pod \"maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd\" (UID: \"5a54206a-c639-435a-92ec-ca9dcd879f32\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd"
Aug 05 18:07:18.618884 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:18.618858    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d794\" (UniqueName: \"kubernetes.io/projected/5a54206a-c639-435a-92ec-ca9dcd879f32-kube-api-access-4d794\") pod \"maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd\" (UID: \"5a54206a-c639-435a-92ec-ca9dcd879f32\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd"
Aug 05 18:07:19.026289 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:19.026250    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9"]
Aug 05 18:07:19.029848 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:19.029827    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9"
Aug 05 18:07:19.032573 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:19.032554    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-b0e02a-dockercfg-gqqr4\""
Aug 05 18:07:19.032684 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:19.032596    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-b0e02a\""
Aug 05 18:07:19.038161 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:19.038136    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9"]
Aug 05 18:07:19.109371 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:19.109337    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tn9nj\" (UniqueName: \"kubernetes.io/projected/854423ae-2cd8-43a5-8c2c-e3dc1d257c7d-kube-api-access-tn9nj\") pod \"payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9\" (UID: \"854423ae-2cd8-43a5-8c2c-e3dc1d257c7d\") " pod="openshift-ingress/payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9"
Aug 05 18:07:19.109525 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:19.109446    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/854423ae-2cd8-43a5-8c2c-e3dc1d257c7d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9\" (UID: \"854423ae-2cd8-43a5-8c2c-e3dc1d257c7d\") " pod="openshift-ingress/payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9"
Aug 05 18:07:19.109525 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:19.109491    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a54206a-c639-435a-92ec-ca9dcd879f32-maas-api-tls\") pod \"maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd\" (UID: \"5a54206a-c639-435a-92ec-ca9dcd879f32\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd"
Aug 05 18:07:19.112155 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:19.112129    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a54206a-c639-435a-92ec-ca9dcd879f32-maas-api-tls\") pod \"maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd\" (UID: \"5a54206a-c639-435a-92ec-ca9dcd879f32\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd"
Aug 05 18:07:19.210079 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:19.210048    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/854423ae-2cd8-43a5-8c2c-e3dc1d257c7d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9\" (UID: \"854423ae-2cd8-43a5-8c2c-e3dc1d257c7d\") " pod="openshift-ingress/payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9"
Aug 05 18:07:19.210225 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:19.210143    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn9nj\" (UniqueName: \"kubernetes.io/projected/854423ae-2cd8-43a5-8c2c-e3dc1d257c7d-kube-api-access-tn9nj\") pod \"payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9\" (UID: \"854423ae-2cd8-43a5-8c2c-e3dc1d257c7d\") " pod="openshift-ingress/payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9"
Aug 05 18:07:19.210863 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:19.210842    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/854423ae-2cd8-43a5-8c2c-e3dc1d257c7d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9\" (UID: \"854423ae-2cd8-43a5-8c2c-e3dc1d257c7d\") " pod="openshift-ingress/payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9"
Aug 05 18:07:19.218310 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:19.218287    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tn9nj\" (UniqueName: \"kubernetes.io/projected/854423ae-2cd8-43a5-8c2c-e3dc1d257c7d-kube-api-access-tn9nj\") pod \"payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9\" (UID: \"854423ae-2cd8-43a5-8c2c-e3dc1d257c7d\") " pod="openshift-ingress/payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9"
Aug 05 18:07:19.298269 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:19.298209    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd"
Aug 05 18:07:19.342512 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:19.342487    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9"
Aug 05 18:07:19.440878 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:19.440852    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd"]
Aug 05 18:07:19.443584 ip-10-0-134-46 kubenswrapper[2550]: W0805 18:07:19.443526    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a54206a_c639_435a_92ec_ca9dcd879f32.slice/crio-20a410367b6cc1bc78396895b8bee4612ee4de3d64c3b83f28fc9d95c6b93724 WatchSource:0}: Error finding container 20a410367b6cc1bc78396895b8bee4612ee4de3d64c3b83f28fc9d95c6b93724: Status 404 returned error can't find the container with id 20a410367b6cc1bc78396895b8bee4612ee4de3d64c3b83f28fc9d95c6b93724
Aug 05 18:07:19.445453 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:19.445430    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 18:07:19.502623 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:19.501433    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9"]
Aug 05 18:07:20.235109 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:20.235067    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd" event={"ID":"5a54206a-c639-435a-92ec-ca9dcd879f32","Type":"ContainerStarted","Data":"882f94619524db5d1c7f3eca175cf8083f34135083aa21ef1f2f5c3e228fc81a"}
Aug 05 18:07:20.235294 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:20.235137    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd"
Aug 05 18:07:20.235294 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:20.235159    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd" event={"ID":"5a54206a-c639-435a-92ec-ca9dcd879f32","Type":"ContainerStarted","Data":"20a410367b6cc1bc78396895b8bee4612ee4de3d64c3b83f28fc9d95c6b93724"}
Aug 05 18:07:20.236766 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:20.236736    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9" event={"ID":"854423ae-2cd8-43a5-8c2c-e3dc1d257c7d","Type":"ContainerStarted","Data":"0c877de1c2036000b0879eb91e92964b2095ee538d39906a358efeff49a95183"}
Aug 05 18:07:20.236887 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:20.236781    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9" event={"ID":"854423ae-2cd8-43a5-8c2c-e3dc1d257c7d","Type":"ContainerStarted","Data":"932b362f04d49a72a4411b5fec78d936009b738257c18c49c0a0bc79740afc5f"}
Aug 05 18:07:20.236887 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:20.236801    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9"
Aug 05 18:07:20.255872 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:20.255814    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd" podStartSLOduration=1.8334057339999998 podStartE2EDuration="2.255799037s" totalImagesPullingTime="422.393303ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:07:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:07:20.255043299 +0000 UTC m=+2747.236620729" watchObservedRunningTime="2026-08-05 18:07:20.255799037 +0000 UTC m=+2747.237376466"
Aug 05 18:07:20.276268 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:20.276226    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9" podStartSLOduration=1.276212071 podStartE2EDuration="1.276212071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:07:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:07:20.274747397 +0000 UTC m=+2747.256324828" watchObservedRunningTime="2026-08-05 18:07:20.276212071 +0000 UTC m=+2747.257789499"
Aug 05 18:07:26.247148 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:26.247113    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd"
Aug 05 18:07:31.213106 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:31.213071    2550 patch_prober.go:28] interesting pod/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Aug 05 18:07:31.213532 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:31.213125    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb" podUID="9cebbe6e-049e-4027-9449-bbf2ca345297" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Aug 05 18:07:31.243420 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:31.243393    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9"
Aug 05 18:07:31.297980 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:31.297952    2550 generic.go:356] "Generic (PLEG): container finished" podID="9cebbe6e-049e-4027-9449-bbf2ca345297" containerID="839d1fd9024bc5e05f1e1a7aa273a9a978746186e2fc6f8110f29756e6d113ee" exitCode=0
Aug 05 18:07:31.298166 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:31.298028    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb" event={"ID":"9cebbe6e-049e-4027-9449-bbf2ca345297","Type":"ContainerDied","Data":"839d1fd9024bc5e05f1e1a7aa273a9a978746186e2fc6f8110f29756e6d113ee"}
Aug 05 18:07:31.298484 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:31.298465    2550 scope.go:117] "RemoveContainer" containerID="839d1fd9024bc5e05f1e1a7aa273a9a978746186e2fc6f8110f29756e6d113ee"
Aug 05 18:07:31.299580 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:31.299556    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 18:07:31.299647 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:31.299620    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 18:07:31.299686 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:31.299666    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 18:07:32.304736 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:32.304685    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb" event={"ID":"9cebbe6e-049e-4027-9449-bbf2ca345297","Type":"ContainerStarted","Data":"9d4ce2a1431418300222903b2ff32cd3827e7f82132cf071afe834536466f9f0"}
Aug 05 18:07:32.778487 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:32.778442    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:32.778745 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:32.778690    2550 patch_prober.go:28] interesting pod/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Aug 05 18:07:32.778806 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:32.778769    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb" podUID="9cebbe6e-049e-4027-9449-bbf2ca345297" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Aug 05 18:07:33.779035 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:33.778995    2550 patch_prober.go:28] interesting pod/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Aug 05 18:07:33.779525 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:33.779072    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb" podUID="9cebbe6e-049e-4027-9449-bbf2ca345297" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Aug 05 18:07:34.779287 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:34.779245    2550 patch_prober.go:28] interesting pod/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Aug 05 18:07:34.779721 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:34.779304    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb" podUID="9cebbe6e-049e-4027-9449-bbf2ca345297" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Aug 05 18:07:35.779315 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:35.779280    2550 patch_prober.go:28] interesting pod/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Aug 05 18:07:35.779746 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:35.779344    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb" podUID="9cebbe6e-049e-4027-9449-bbf2ca345297" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Aug 05 18:07:36.778819 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:36.778783    2550 patch_prober.go:28] interesting pod/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Aug 05 18:07:36.779037 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:36.778848    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb" podUID="9cebbe6e-049e-4027-9449-bbf2ca345297" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Aug 05 18:07:37.783111 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:37.783080    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:37.783618 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:37.783339    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:37.784137 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:37.784119    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:07:47.009901 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:47.009864    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-625864-758d5c8d9b-m87g4"]
Aug 05 18:07:47.017785 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:47.016813    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-625864-758d5c8d9b-m87g4"
Aug 05 18:07:47.020495 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:47.020467    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-625864-serving-cert\""
Aug 05 18:07:47.028979 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:47.028740    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-625864-758d5c8d9b-m87g4"]
Aug 05 18:07:47.170880 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:47.170841    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msjsj\" (UniqueName: \"kubernetes.io/projected/944bb3c9-92ae-4556-beaf-5e7578ca9e6f-kube-api-access-msjsj\") pod \"maas-api-e2e-api-b-625864-758d5c8d9b-m87g4\" (UID: \"944bb3c9-92ae-4556-beaf-5e7578ca9e6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-625864-758d5c8d9b-m87g4"
Aug 05 18:07:47.171082 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:47.170945    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/944bb3c9-92ae-4556-beaf-5e7578ca9e6f-maas-api-tls\") pod \"maas-api-e2e-api-b-625864-758d5c8d9b-m87g4\" (UID: \"944bb3c9-92ae-4556-beaf-5e7578ca9e6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-625864-758d5c8d9b-m87g4"
Aug 05 18:07:47.171082 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:47.170972    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/944bb3c9-92ae-4556-beaf-5e7578ca9e6f-openshift-service-ca\") pod \"maas-api-e2e-api-b-625864-758d5c8d9b-m87g4\" (UID: \"944bb3c9-92ae-4556-beaf-5e7578ca9e6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-625864-758d5c8d9b-m87g4"
Aug 05 18:07:47.272465 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:47.272366    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msjsj\" (UniqueName: \"kubernetes.io/projected/944bb3c9-92ae-4556-beaf-5e7578ca9e6f-kube-api-access-msjsj\") pod \"maas-api-e2e-api-b-625864-758d5c8d9b-m87g4\" (UID: \"944bb3c9-92ae-4556-beaf-5e7578ca9e6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-625864-758d5c8d9b-m87g4"
Aug 05 18:07:47.272465 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:47.272453    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/944bb3c9-92ae-4556-beaf-5e7578ca9e6f-maas-api-tls\") pod \"maas-api-e2e-api-b-625864-758d5c8d9b-m87g4\" (UID: \"944bb3c9-92ae-4556-beaf-5e7578ca9e6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-625864-758d5c8d9b-m87g4"
Aug 05 18:07:47.272747 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:47.272485    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/944bb3c9-92ae-4556-beaf-5e7578ca9e6f-openshift-service-ca\") pod \"maas-api-e2e-api-b-625864-758d5c8d9b-m87g4\" (UID: \"944bb3c9-92ae-4556-beaf-5e7578ca9e6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-625864-758d5c8d9b-m87g4"
Aug 05 18:07:47.272747 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:07:47.272583    2550 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-625864-serving-cert: secret "maas-api-e2e-api-b-625864-serving-cert" not found
Aug 05 18:07:47.272747 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:07:47.272669    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/944bb3c9-92ae-4556-beaf-5e7578ca9e6f-maas-api-tls podName:944bb3c9-92ae-4556-beaf-5e7578ca9e6f nodeName:}" failed. No retries permitted until 2026-08-05 18:07:47.772647335 +0000 UTC m=+2774.754224760 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/944bb3c9-92ae-4556-beaf-5e7578ca9e6f-maas-api-tls") pod "maas-api-e2e-api-b-625864-758d5c8d9b-m87g4" (UID: "944bb3c9-92ae-4556-beaf-5e7578ca9e6f") : secret "maas-api-e2e-api-b-625864-serving-cert" not found
Aug 05 18:07:47.273213 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:47.273190    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/944bb3c9-92ae-4556-beaf-5e7578ca9e6f-openshift-service-ca\") pod \"maas-api-e2e-api-b-625864-758d5c8d9b-m87g4\" (UID: \"944bb3c9-92ae-4556-beaf-5e7578ca9e6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-625864-758d5c8d9b-m87g4"
Aug 05 18:07:47.281817 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:47.281796    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msjsj\" (UniqueName: \"kubernetes.io/projected/944bb3c9-92ae-4556-beaf-5e7578ca9e6f-kube-api-access-msjsj\") pod \"maas-api-e2e-api-b-625864-758d5c8d9b-m87g4\" (UID: \"944bb3c9-92ae-4556-beaf-5e7578ca9e6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-625864-758d5c8d9b-m87g4"
Aug 05 18:07:47.715780 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:47.715745    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-625864-bcd6fbccb-lm272"]
Aug 05 18:07:47.719558 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:47.719527    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-625864-bcd6fbccb-lm272"
Aug 05 18:07:47.721912 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:47.721893    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-625864\""
Aug 05 18:07:47.722015 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:47.721941    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-625864-dockercfg-nlbqq\""
Aug 05 18:07:47.726977 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:47.726951    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-625864-bcd6fbccb-lm272"]
Aug 05 18:07:47.777459 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:47.777424    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/944bb3c9-92ae-4556-beaf-5e7578ca9e6f-maas-api-tls\") pod \"maas-api-e2e-api-b-625864-758d5c8d9b-m87g4\" (UID: \"944bb3c9-92ae-4556-beaf-5e7578ca9e6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-625864-758d5c8d9b-m87g4"
Aug 05 18:07:47.779951 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:47.779928    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/944bb3c9-92ae-4556-beaf-5e7578ca9e6f-maas-api-tls\") pod \"maas-api-e2e-api-b-625864-758d5c8d9b-m87g4\" (UID: \"944bb3c9-92ae-4556-beaf-5e7578ca9e6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-625864-758d5c8d9b-m87g4"
Aug 05 18:07:47.878678 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:47.878632    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f87e4883-c130-446d-a0ac-316f75d07012-plugins-config-volume\") pod \"payload-processing-e2e-api-b-625864-bcd6fbccb-lm272\" (UID: \"f87e4883-c130-446d-a0ac-316f75d07012\") " pod="openshift-ingress/payload-processing-e2e-api-b-625864-bcd6fbccb-lm272"
Aug 05 18:07:47.878678 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:47.878680    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slcwr\" (UniqueName: \"kubernetes.io/projected/f87e4883-c130-446d-a0ac-316f75d07012-kube-api-access-slcwr\") pod \"payload-processing-e2e-api-b-625864-bcd6fbccb-lm272\" (UID: \"f87e4883-c130-446d-a0ac-316f75d07012\") " pod="openshift-ingress/payload-processing-e2e-api-b-625864-bcd6fbccb-lm272"
Aug 05 18:07:47.932966 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:47.932927    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-625864-758d5c8d9b-m87g4"
Aug 05 18:07:47.980252 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:47.980150    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f87e4883-c130-446d-a0ac-316f75d07012-plugins-config-volume\") pod \"payload-processing-e2e-api-b-625864-bcd6fbccb-lm272\" (UID: \"f87e4883-c130-446d-a0ac-316f75d07012\") " pod="openshift-ingress/payload-processing-e2e-api-b-625864-bcd6fbccb-lm272"
Aug 05 18:07:47.980252 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:47.980195    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slcwr\" (UniqueName: \"kubernetes.io/projected/f87e4883-c130-446d-a0ac-316f75d07012-kube-api-access-slcwr\") pod \"payload-processing-e2e-api-b-625864-bcd6fbccb-lm272\" (UID: \"f87e4883-c130-446d-a0ac-316f75d07012\") " pod="openshift-ingress/payload-processing-e2e-api-b-625864-bcd6fbccb-lm272"
Aug 05 18:07:47.980896 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:47.980877    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f87e4883-c130-446d-a0ac-316f75d07012-plugins-config-volume\") pod \"payload-processing-e2e-api-b-625864-bcd6fbccb-lm272\" (UID: \"f87e4883-c130-446d-a0ac-316f75d07012\") " pod="openshift-ingress/payload-processing-e2e-api-b-625864-bcd6fbccb-lm272"
Aug 05 18:07:47.988345 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:47.988319    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slcwr\" (UniqueName: \"kubernetes.io/projected/f87e4883-c130-446d-a0ac-316f75d07012-kube-api-access-slcwr\") pod \"payload-processing-e2e-api-b-625864-bcd6fbccb-lm272\" (UID: \"f87e4883-c130-446d-a0ac-316f75d07012\") " pod="openshift-ingress/payload-processing-e2e-api-b-625864-bcd6fbccb-lm272"
Aug 05 18:07:48.033053 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:48.033021    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-625864-bcd6fbccb-lm272"
Aug 05 18:07:48.074935 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:48.074909    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-625864-758d5c8d9b-m87g4"]
Aug 05 18:07:48.076502 ip-10-0-134-46 kubenswrapper[2550]: W0805 18:07:48.076476    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod944bb3c9_92ae_4556_beaf_5e7578ca9e6f.slice/crio-e0c97bbbac2adbdd173f9c1c9a236d7a46a064d044a371d5660fe84e4b5154b4 WatchSource:0}: Error finding container e0c97bbbac2adbdd173f9c1c9a236d7a46a064d044a371d5660fe84e4b5154b4: Status 404 returned error can't find the container with id e0c97bbbac2adbdd173f9c1c9a236d7a46a064d044a371d5660fe84e4b5154b4
Aug 05 18:07:48.177215 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:48.177191    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-625864-bcd6fbccb-lm272"]
Aug 05 18:07:48.179376 ip-10-0-134-46 kubenswrapper[2550]: W0805 18:07:48.179350    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf87e4883_c130_446d_a0ac_316f75d07012.slice/crio-bb68d88a7b382df1e5cdb854286f7a00118b55d6cf5a2303db088c62908291a8 WatchSource:0}: Error finding container bb68d88a7b382df1e5cdb854286f7a00118b55d6cf5a2303db088c62908291a8: Status 404 returned error can't find the container with id bb68d88a7b382df1e5cdb854286f7a00118b55d6cf5a2303db088c62908291a8
Aug 05 18:07:48.393260 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:48.393222    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-625864-bcd6fbccb-lm272" event={"ID":"f87e4883-c130-446d-a0ac-316f75d07012","Type":"ContainerStarted","Data":"c3ab90dacf1963f2c3dd8150a4b2a8a601f33dde6bc8adff5aec6f2249e995ab"}
Aug 05 18:07:48.393260 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:48.393282    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-625864-bcd6fbccb-lm272"
Aug 05 18:07:48.393541 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:48.393299    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-625864-bcd6fbccb-lm272" event={"ID":"f87e4883-c130-446d-a0ac-316f75d07012","Type":"ContainerStarted","Data":"bb68d88a7b382df1e5cdb854286f7a00118b55d6cf5a2303db088c62908291a8"}
Aug 05 18:07:48.394329 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:48.394305    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-625864-758d5c8d9b-m87g4" event={"ID":"944bb3c9-92ae-4556-beaf-5e7578ca9e6f","Type":"ContainerStarted","Data":"e0c97bbbac2adbdd173f9c1c9a236d7a46a064d044a371d5660fe84e4b5154b4"}
Aug 05 18:07:48.414890 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:48.414729    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-625864-bcd6fbccb-lm272" podStartSLOduration=1.414694028 podStartE2EDuration="1.414694028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:07:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:07:48.411636256 +0000 UTC m=+2775.393213697" watchObservedRunningTime="2026-08-05 18:07:48.414694028 +0000 UTC m=+2775.396271458"
Aug 05 18:07:49.401276 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:49.401242    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-625864-758d5c8d9b-m87g4" event={"ID":"944bb3c9-92ae-4556-beaf-5e7578ca9e6f","Type":"ContainerStarted","Data":"f18f071d4662cd8c2c3a8f6c3b5fcbcf6b37b17b5f50984f59d36b2ba02735b1"}
Aug 05 18:07:49.401809 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:49.401343    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-625864-758d5c8d9b-m87g4"
Aug 05 18:07:49.420921 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:49.420871    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-625864-758d5c8d9b-m87g4" podStartSLOduration=3.096441029 podStartE2EDuration="3.420854645s" totalImagesPullingTime="324.413616ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:07:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:07:49.418823 +0000 UTC m=+2776.400400431" watchObservedRunningTime="2026-08-05 18:07:49.420854645 +0000 UTC m=+2776.402432074"
Aug 05 18:07:55.413565 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:55.413533    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-625864-758d5c8d9b-m87g4"
Aug 05 18:07:59.403421 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:07:59.403386    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-625864-bcd6fbccb-lm272"
Aug 05 18:08:29.257167 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.257136    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd"]
Aug 05 18:08:29.257851 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.257395    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd" podUID="5a54206a-c639-435a-92ec-ca9dcd879f32" containerName="maas-api" containerID="cri-o://882f94619524db5d1c7f3eca175cf8083f34135083aa21ef1f2f5c3e228fc81a" gracePeriod=30
Aug 05 18:08:29.317491 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.317459    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9"]
Aug 05 18:08:29.317690 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.317666    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9" podUID="854423ae-2cd8-43a5-8c2c-e3dc1d257c7d" containerName="payload-processing" containerID="cri-o://0c877de1c2036000b0879eb91e92964b2095ee538d39906a358efeff49a95183" gracePeriod=30
Aug 05 18:08:29.575151 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.575120    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd"
Aug 05 18:08:29.600439 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.600406    2550 generic.go:356] "Generic (PLEG): container finished" podID="5a54206a-c639-435a-92ec-ca9dcd879f32" containerID="882f94619524db5d1c7f3eca175cf8083f34135083aa21ef1f2f5c3e228fc81a" exitCode=0
Aug 05 18:08:29.600619 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.600597    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd"
Aug 05 18:08:29.600784 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.600599    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd" event={"ID":"5a54206a-c639-435a-92ec-ca9dcd879f32","Type":"ContainerDied","Data":"882f94619524db5d1c7f3eca175cf8083f34135083aa21ef1f2f5c3e228fc81a"}
Aug 05 18:08:29.600871 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.600836    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd" event={"ID":"5a54206a-c639-435a-92ec-ca9dcd879f32","Type":"ContainerDied","Data":"20a410367b6cc1bc78396895b8bee4612ee4de3d64c3b83f28fc9d95c6b93724"}
Aug 05 18:08:29.600927 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.600873    2550 scope.go:117] "RemoveContainer" containerID="882f94619524db5d1c7f3eca175cf8083f34135083aa21ef1f2f5c3e228fc81a"
Aug 05 18:08:29.602524 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.602237    2550 generic.go:356] "Generic (PLEG): container finished" podID="854423ae-2cd8-43a5-8c2c-e3dc1d257c7d" containerID="0c877de1c2036000b0879eb91e92964b2095ee538d39906a358efeff49a95183" exitCode=0
Aug 05 18:08:29.602524 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.602299    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9" event={"ID":"854423ae-2cd8-43a5-8c2c-e3dc1d257c7d","Type":"ContainerDied","Data":"0c877de1c2036000b0879eb91e92964b2095ee538d39906a358efeff49a95183"}
Aug 05 18:08:29.612208 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.612190    2550 scope.go:117] "RemoveContainer" containerID="882f94619524db5d1c7f3eca175cf8083f34135083aa21ef1f2f5c3e228fc81a"
Aug 05 18:08:29.612548 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:08:29.612522    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"882f94619524db5d1c7f3eca175cf8083f34135083aa21ef1f2f5c3e228fc81a\": container with ID starting with 882f94619524db5d1c7f3eca175cf8083f34135083aa21ef1f2f5c3e228fc81a not found: ID does not exist" containerID="882f94619524db5d1c7f3eca175cf8083f34135083aa21ef1f2f5c3e228fc81a"
Aug 05 18:08:29.612617 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.612563    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"882f94619524db5d1c7f3eca175cf8083f34135083aa21ef1f2f5c3e228fc81a"} err="failed to get container status \"882f94619524db5d1c7f3eca175cf8083f34135083aa21ef1f2f5c3e228fc81a\": rpc error: code = NotFound desc = could not find container \"882f94619524db5d1c7f3eca175cf8083f34135083aa21ef1f2f5c3e228fc81a\": container with ID starting with 882f94619524db5d1c7f3eca175cf8083f34135083aa21ef1f2f5c3e228fc81a not found: ID does not exist"
Aug 05 18:08:29.615220 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.615203    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9"
Aug 05 18:08:29.757557 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.757532    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/854423ae-2cd8-43a5-8c2c-e3dc1d257c7d-plugins-config-volume\") pod \"854423ae-2cd8-43a5-8c2c-e3dc1d257c7d\" (UID: \"854423ae-2cd8-43a5-8c2c-e3dc1d257c7d\") "
Aug 05 18:08:29.757739 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.757623    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a54206a-c639-435a-92ec-ca9dcd879f32-maas-api-tls\") pod \"5a54206a-c639-435a-92ec-ca9dcd879f32\" (UID: \"5a54206a-c639-435a-92ec-ca9dcd879f32\") "
Aug 05 18:08:29.757739 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.757652    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a54206a-c639-435a-92ec-ca9dcd879f32-openshift-service-ca\") pod \"5a54206a-c639-435a-92ec-ca9dcd879f32\" (UID: \"5a54206a-c639-435a-92ec-ca9dcd879f32\") "
Aug 05 18:08:29.757739 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.757678    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tn9nj\" (UniqueName: \"kubernetes.io/projected/854423ae-2cd8-43a5-8c2c-e3dc1d257c7d-kube-api-access-tn9nj\") pod \"854423ae-2cd8-43a5-8c2c-e3dc1d257c7d\" (UID: \"854423ae-2cd8-43a5-8c2c-e3dc1d257c7d\") "
Aug 05 18:08:29.757906 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.757737    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4d794\" (UniqueName: \"kubernetes.io/projected/5a54206a-c639-435a-92ec-ca9dcd879f32-kube-api-access-4d794\") pod \"5a54206a-c639-435a-92ec-ca9dcd879f32\" (UID: \"5a54206a-c639-435a-92ec-ca9dcd879f32\") "
Aug 05 18:08:29.758010 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.757980    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/854423ae-2cd8-43a5-8c2c-e3dc1d257c7d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "854423ae-2cd8-43a5-8c2c-e3dc1d257c7d" (UID: "854423ae-2cd8-43a5-8c2c-e3dc1d257c7d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:08:29.758074 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.758027    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a54206a-c639-435a-92ec-ca9dcd879f32-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5a54206a-c639-435a-92ec-ca9dcd879f32" (UID: "5a54206a-c639-435a-92ec-ca9dcd879f32"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:08:29.759883 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.759858    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a54206a-c639-435a-92ec-ca9dcd879f32-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5a54206a-c639-435a-92ec-ca9dcd879f32" (UID: "5a54206a-c639-435a-92ec-ca9dcd879f32"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:08:29.760000 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.759879    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/854423ae-2cd8-43a5-8c2c-e3dc1d257c7d-kube-api-access-tn9nj" (OuterVolumeSpecName: "kube-api-access-tn9nj") pod "854423ae-2cd8-43a5-8c2c-e3dc1d257c7d" (UID: "854423ae-2cd8-43a5-8c2c-e3dc1d257c7d"). InnerVolumeSpecName "kube-api-access-tn9nj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:08:29.760000 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.759885    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a54206a-c639-435a-92ec-ca9dcd879f32-kube-api-access-4d794" (OuterVolumeSpecName: "kube-api-access-4d794") pod "5a54206a-c639-435a-92ec-ca9dcd879f32" (UID: "5a54206a-c639-435a-92ec-ca9dcd879f32"). InnerVolumeSpecName "kube-api-access-4d794". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:08:29.858589 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.858513    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/854423ae-2cd8-43a5-8c2c-e3dc1d257c7d-plugins-config-volume\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:08:29.858589 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.858545    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a54206a-c639-435a-92ec-ca9dcd879f32-maas-api-tls\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:08:29.858589 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.858559    2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a54206a-c639-435a-92ec-ca9dcd879f32-openshift-service-ca\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:08:29.858589 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.858571    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tn9nj\" (UniqueName: \"kubernetes.io/projected/854423ae-2cd8-43a5-8c2c-e3dc1d257c7d-kube-api-access-tn9nj\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:08:29.858589 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.858583    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4d794\" (UniqueName: \"kubernetes.io/projected/5a54206a-c639-435a-92ec-ca9dcd879f32-kube-api-access-4d794\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:08:29.922432 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.922402    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd"]
Aug 05 18:08:29.928192 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:29.928169    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b0e02a-5bdc8bf9b6-wl4xd"]
Aug 05 18:08:30.608597 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:30.608517    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9"
Aug 05 18:08:30.608597 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:30.608529    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9" event={"ID":"854423ae-2cd8-43a5-8c2c-e3dc1d257c7d","Type":"ContainerDied","Data":"932b362f04d49a72a4411b5fec78d936009b738257c18c49c0a0bc79740afc5f"}
Aug 05 18:08:30.608597 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:30.608589    2550 scope.go:117] "RemoveContainer" containerID="0c877de1c2036000b0879eb91e92964b2095ee538d39906a358efeff49a95183"
Aug 05 18:08:30.631179 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:30.631153    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9"]
Aug 05 18:08:30.635711 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:30.635675    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b0e02a-6df478f744-p9lj9"]
Aug 05 18:08:31.703046 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:31.703001    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a54206a-c639-435a-92ec-ca9dcd879f32" path="/var/lib/kubelet/pods/5a54206a-c639-435a-92ec-ca9dcd879f32/volumes"
Aug 05 18:08:31.703583 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:31.703561    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="854423ae-2cd8-43a5-8c2c-e3dc1d257c7d" path="/var/lib/kubelet/pods/854423ae-2cd8-43a5-8c2c-e3dc1d257c7d/volumes"
Aug 05 18:08:36.027334 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:36.027302    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"]
Aug 05 18:08:36.027854 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:36.027500    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb" podUID="9cebbe6e-049e-4027-9449-bbf2ca345297" containerName="istio-proxy" containerID="cri-o://9d4ce2a1431418300222903b2ff32cd3827e7f82132cf071afe834536466f9f0" gracePeriod=30
Aug 05 18:08:37.785794 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:37.785758    2550 patch_prober.go:28] interesting pod/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Aug 05 18:08:37.786167 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:37.785818    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb" podUID="9cebbe6e-049e-4027-9449-bbf2ca345297" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 05 18:08:41.070274 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:08:41.070246    2550 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cebbe6e_049e_4027_9449_bbf2ca345297.slice/crio-9d4ce2a1431418300222903b2ff32cd3827e7f82132cf071afe834536466f9f0.scope\": RecentStats: unable to find data in memory cache]"
Aug 05 18:08:41.270571 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.270548    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:08:41.358509 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.358432    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-workload-certs\") pod \"9cebbe6e-049e-4027-9449-bbf2ca345297\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") "
Aug 05 18:08:41.358509 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.358490    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-workload-socket\") pod \"9cebbe6e-049e-4027-9449-bbf2ca345297\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") "
Aug 05 18:08:41.358753 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.358517    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2td47\" (UniqueName: \"kubernetes.io/projected/9cebbe6e-049e-4027-9449-bbf2ca345297-kube-api-access-2td47\") pod \"9cebbe6e-049e-4027-9449-bbf2ca345297\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") "
Aug 05 18:08:41.358753 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.358545    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-istio-data\") pod \"9cebbe6e-049e-4027-9449-bbf2ca345297\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") "
Aug 05 18:08:41.358753 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.358578    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cebbe6e-049e-4027-9449-bbf2ca345297-istio-token\") pod \"9cebbe6e-049e-4027-9449-bbf2ca345297\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") "
Aug 05 18:08:41.358753 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.358624    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-istio-envoy\") pod \"9cebbe6e-049e-4027-9449-bbf2ca345297\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") "
Aug 05 18:08:41.358753 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.358648    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-credential-socket\") pod \"9cebbe6e-049e-4027-9449-bbf2ca345297\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") "
Aug 05 18:08:41.358753 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.358680    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cebbe6e-049e-4027-9449-bbf2ca345297-istio-podinfo\") pod \"9cebbe6e-049e-4027-9449-bbf2ca345297\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") "
Aug 05 18:08:41.358753 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.358716    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9cebbe6e-049e-4027-9449-bbf2ca345297" (UID: "9cebbe6e-049e-4027-9449-bbf2ca345297"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:08:41.358753 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.358739    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cebbe6e-049e-4027-9449-bbf2ca345297-istiod-ca-cert\") pod \"9cebbe6e-049e-4027-9449-bbf2ca345297\" (UID: \"9cebbe6e-049e-4027-9449-bbf2ca345297\") "
Aug 05 18:08:41.359176 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.358844    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-istio-data" (OuterVolumeSpecName: "istio-data") pod "9cebbe6e-049e-4027-9449-bbf2ca345297" (UID: "9cebbe6e-049e-4027-9449-bbf2ca345297"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:08:41.359176 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.358862    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9cebbe6e-049e-4027-9449-bbf2ca345297" (UID: "9cebbe6e-049e-4027-9449-bbf2ca345297"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:08:41.359176 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.359031    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9cebbe6e-049e-4027-9449-bbf2ca345297" (UID: "9cebbe6e-049e-4027-9449-bbf2ca345297"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:08:41.359176 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.359046    2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-workload-certs\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:08:41.359176 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.359093    2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-workload-socket\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:08:41.359176 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.359111    2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-istio-data\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:08:41.359176 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.359162    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cebbe6e-049e-4027-9449-bbf2ca345297-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9cebbe6e-049e-4027-9449-bbf2ca345297" (UID: "9cebbe6e-049e-4027-9449-bbf2ca345297"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:08:41.360863 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.360808    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9cebbe6e-049e-4027-9449-bbf2ca345297" (UID: "9cebbe6e-049e-4027-9449-bbf2ca345297"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:08:41.360863 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.360828    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cebbe6e-049e-4027-9449-bbf2ca345297-kube-api-access-2td47" (OuterVolumeSpecName: "kube-api-access-2td47") pod "9cebbe6e-049e-4027-9449-bbf2ca345297" (UID: "9cebbe6e-049e-4027-9449-bbf2ca345297"). InnerVolumeSpecName "kube-api-access-2td47". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:08:41.361007 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.360905    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9cebbe6e-049e-4027-9449-bbf2ca345297-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9cebbe6e-049e-4027-9449-bbf2ca345297" (UID: "9cebbe6e-049e-4027-9449-bbf2ca345297"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 05 18:08:41.361007 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.360923    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cebbe6e-049e-4027-9449-bbf2ca345297-istio-token" (OuterVolumeSpecName: "istio-token") pod "9cebbe6e-049e-4027-9449-bbf2ca345297" (UID: "9cebbe6e-049e-4027-9449-bbf2ca345297"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:08:41.460355 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.460322    2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-istio-envoy\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:08:41.460355 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.460351    2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cebbe6e-049e-4027-9449-bbf2ca345297-credential-socket\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:08:41.460355 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.460361    2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cebbe6e-049e-4027-9449-bbf2ca345297-istio-podinfo\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:08:41.460577 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.460370    2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cebbe6e-049e-4027-9449-bbf2ca345297-istiod-ca-cert\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:08:41.460577 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.460378    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2td47\" (UniqueName: \"kubernetes.io/projected/9cebbe6e-049e-4027-9449-bbf2ca345297-kube-api-access-2td47\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:08:41.460577 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.460387    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cebbe6e-049e-4027-9449-bbf2ca345297-istio-token\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:08:41.663952 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.663923    2550 generic.go:356] "Generic (PLEG): container finished" podID="9cebbe6e-049e-4027-9449-bbf2ca345297" containerID="9d4ce2a1431418300222903b2ff32cd3827e7f82132cf071afe834536466f9f0" exitCode=0
Aug 05 18:08:41.664115 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.663970    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"
Aug 05 18:08:41.664115 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.664014    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb" event={"ID":"9cebbe6e-049e-4027-9449-bbf2ca345297","Type":"ContainerDied","Data":"9d4ce2a1431418300222903b2ff32cd3827e7f82132cf071afe834536466f9f0"}
Aug 05 18:08:41.664115 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.664072    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb" event={"ID":"9cebbe6e-049e-4027-9449-bbf2ca345297","Type":"ContainerDied","Data":"e6eafe9644ffcdfbd30d2db05b67a142f37ca59345778b67c1e4954a95ae8d6a"}
Aug 05 18:08:41.664115 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.664100    2550 scope.go:117] "RemoveContainer" containerID="9d4ce2a1431418300222903b2ff32cd3827e7f82132cf071afe834536466f9f0"
Aug 05 18:08:41.672730 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.672692    2550 scope.go:117] "RemoveContainer" containerID="839d1fd9024bc5e05f1e1a7aa273a9a978746186e2fc6f8110f29756e6d113ee"
Aug 05 18:08:41.680122 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.680106    2550 scope.go:117] "RemoveContainer" containerID="9d4ce2a1431418300222903b2ff32cd3827e7f82132cf071afe834536466f9f0"
Aug 05 18:08:41.680358 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:08:41.680341    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d4ce2a1431418300222903b2ff32cd3827e7f82132cf071afe834536466f9f0\": container with ID starting with 9d4ce2a1431418300222903b2ff32cd3827e7f82132cf071afe834536466f9f0 not found: ID does not exist" containerID="9d4ce2a1431418300222903b2ff32cd3827e7f82132cf071afe834536466f9f0"
Aug 05 18:08:41.680405 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.680366    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d4ce2a1431418300222903b2ff32cd3827e7f82132cf071afe834536466f9f0"} err="failed to get container status \"9d4ce2a1431418300222903b2ff32cd3827e7f82132cf071afe834536466f9f0\": rpc error: code = NotFound desc = could not find container \"9d4ce2a1431418300222903b2ff32cd3827e7f82132cf071afe834536466f9f0\": container with ID starting with 9d4ce2a1431418300222903b2ff32cd3827e7f82132cf071afe834536466f9f0 not found: ID does not exist"
Aug 05 18:08:41.680405 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.680382    2550 scope.go:117] "RemoveContainer" containerID="839d1fd9024bc5e05f1e1a7aa273a9a978746186e2fc6f8110f29756e6d113ee"
Aug 05 18:08:41.680571 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:08:41.680554    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"839d1fd9024bc5e05f1e1a7aa273a9a978746186e2fc6f8110f29756e6d113ee\": container with ID starting with 839d1fd9024bc5e05f1e1a7aa273a9a978746186e2fc6f8110f29756e6d113ee not found: ID does not exist" containerID="839d1fd9024bc5e05f1e1a7aa273a9a978746186e2fc6f8110f29756e6d113ee"
Aug 05 18:08:41.680613 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.680573    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"839d1fd9024bc5e05f1e1a7aa273a9a978746186e2fc6f8110f29756e6d113ee"} err="failed to get container status \"839d1fd9024bc5e05f1e1a7aa273a9a978746186e2fc6f8110f29756e6d113ee\": rpc error: code = NotFound desc = could not find container \"839d1fd9024bc5e05f1e1a7aa273a9a978746186e2fc6f8110f29756e6d113ee\": container with ID starting with 839d1fd9024bc5e05f1e1a7aa273a9a978746186e2fc6f8110f29756e6d113ee not found: ID does not exist"
Aug 05 18:08:41.686345 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.686326    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"]
Aug 05 18:08:41.690233 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.690210    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-b0e02a-openshift-default-9945cdcd4-plnjb"]
Aug 05 18:08:41.700062 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:41.700043    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cebbe6e-049e-4027-9449-bbf2ca345297" path="/var/lib/kubelet/pods/9cebbe6e-049e-4027-9449-bbf2ca345297/volumes"
Aug 05 18:08:51.886007 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:51.885966    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-625864-758d5c8d9b-m87g4"]
Aug 05 18:08:51.886424 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:51.886219    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-625864-758d5c8d9b-m87g4" podUID="944bb3c9-92ae-4556-beaf-5e7578ca9e6f" containerName="maas-api" containerID="cri-o://f18f071d4662cd8c2c3a8f6c3b5fcbcf6b37b17b5f50984f59d36b2ba02735b1" gracePeriod=30
Aug 05 18:08:51.939463 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:51.939419    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-625864-bcd6fbccb-lm272"]
Aug 05 18:08:51.939686 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:51.939661    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-625864-bcd6fbccb-lm272" podUID="f87e4883-c130-446d-a0ac-316f75d07012" containerName="payload-processing" containerID="cri-o://c3ab90dacf1963f2c3dd8150a4b2a8a601f33dde6bc8adff5aec6f2249e995ab" gracePeriod=30
Aug 05 18:08:52.199426 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.199402    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-625864-758d5c8d9b-m87g4"
Aug 05 18:08:52.201997 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.201980    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-625864-bcd6fbccb-lm272"
Aug 05 18:08:52.257513 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.257480    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f87e4883-c130-446d-a0ac-316f75d07012-plugins-config-volume\") pod \"f87e4883-c130-446d-a0ac-316f75d07012\" (UID: \"f87e4883-c130-446d-a0ac-316f75d07012\") "
Aug 05 18:08:52.257688 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.257549    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msjsj\" (UniqueName: \"kubernetes.io/projected/944bb3c9-92ae-4556-beaf-5e7578ca9e6f-kube-api-access-msjsj\") pod \"944bb3c9-92ae-4556-beaf-5e7578ca9e6f\" (UID: \"944bb3c9-92ae-4556-beaf-5e7578ca9e6f\") "
Aug 05 18:08:52.257688 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.257589    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/944bb3c9-92ae-4556-beaf-5e7578ca9e6f-maas-api-tls\") pod \"944bb3c9-92ae-4556-beaf-5e7578ca9e6f\" (UID: \"944bb3c9-92ae-4556-beaf-5e7578ca9e6f\") "
Aug 05 18:08:52.257688 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.257607    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/944bb3c9-92ae-4556-beaf-5e7578ca9e6f-openshift-service-ca\") pod \"944bb3c9-92ae-4556-beaf-5e7578ca9e6f\" (UID: \"944bb3c9-92ae-4556-beaf-5e7578ca9e6f\") "
Aug 05 18:08:52.257688 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.257630    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-slcwr\" (UniqueName: \"kubernetes.io/projected/f87e4883-c130-446d-a0ac-316f75d07012-kube-api-access-slcwr\") pod \"f87e4883-c130-446d-a0ac-316f75d07012\" (UID: \"f87e4883-c130-446d-a0ac-316f75d07012\") "
Aug 05 18:08:52.257987 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.257955    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f87e4883-c130-446d-a0ac-316f75d07012-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f87e4883-c130-446d-a0ac-316f75d07012" (UID: "f87e4883-c130-446d-a0ac-316f75d07012"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:08:52.258109 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.258053    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/944bb3c9-92ae-4556-beaf-5e7578ca9e6f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "944bb3c9-92ae-4556-beaf-5e7578ca9e6f" (UID: "944bb3c9-92ae-4556-beaf-5e7578ca9e6f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:08:52.259789 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.259766    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/944bb3c9-92ae-4556-beaf-5e7578ca9e6f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "944bb3c9-92ae-4556-beaf-5e7578ca9e6f" (UID: "944bb3c9-92ae-4556-beaf-5e7578ca9e6f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:08:52.260193 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.260169    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/944bb3c9-92ae-4556-beaf-5e7578ca9e6f-kube-api-access-msjsj" (OuterVolumeSpecName: "kube-api-access-msjsj") pod "944bb3c9-92ae-4556-beaf-5e7578ca9e6f" (UID: "944bb3c9-92ae-4556-beaf-5e7578ca9e6f"). InnerVolumeSpecName "kube-api-access-msjsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:08:52.260193 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.260178    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f87e4883-c130-446d-a0ac-316f75d07012-kube-api-access-slcwr" (OuterVolumeSpecName: "kube-api-access-slcwr") pod "f87e4883-c130-446d-a0ac-316f75d07012" (UID: "f87e4883-c130-446d-a0ac-316f75d07012"). InnerVolumeSpecName "kube-api-access-slcwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:08:52.358653 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.358613    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f87e4883-c130-446d-a0ac-316f75d07012-plugins-config-volume\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:08:52.358653 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.358646    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msjsj\" (UniqueName: \"kubernetes.io/projected/944bb3c9-92ae-4556-beaf-5e7578ca9e6f-kube-api-access-msjsj\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:08:52.358653 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.358662    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/944bb3c9-92ae-4556-beaf-5e7578ca9e6f-maas-api-tls\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:08:52.358941 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.358674    2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/944bb3c9-92ae-4556-beaf-5e7578ca9e6f-openshift-service-ca\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:08:52.358941 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.358686    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slcwr\" (UniqueName: \"kubernetes.io/projected/f87e4883-c130-446d-a0ac-316f75d07012-kube-api-access-slcwr\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:08:52.716626 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.716592    2550 generic.go:356] "Generic (PLEG): container finished" podID="944bb3c9-92ae-4556-beaf-5e7578ca9e6f" containerID="f18f071d4662cd8c2c3a8f6c3b5fcbcf6b37b17b5f50984f59d36b2ba02735b1" exitCode=0
Aug 05 18:08:52.716854 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.716636    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-625864-758d5c8d9b-m87g4"
Aug 05 18:08:52.716854 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.716694    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-625864-758d5c8d9b-m87g4" event={"ID":"944bb3c9-92ae-4556-beaf-5e7578ca9e6f","Type":"ContainerDied","Data":"f18f071d4662cd8c2c3a8f6c3b5fcbcf6b37b17b5f50984f59d36b2ba02735b1"}
Aug 05 18:08:52.716854 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.716754    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-625864-758d5c8d9b-m87g4" event={"ID":"944bb3c9-92ae-4556-beaf-5e7578ca9e6f","Type":"ContainerDied","Data":"e0c97bbbac2adbdd173f9c1c9a236d7a46a064d044a371d5660fe84e4b5154b4"}
Aug 05 18:08:52.716854 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.716782    2550 scope.go:117] "RemoveContainer" containerID="f18f071d4662cd8c2c3a8f6c3b5fcbcf6b37b17b5f50984f59d36b2ba02735b1"
Aug 05 18:08:52.718136 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.718117    2550 generic.go:356] "Generic (PLEG): container finished" podID="f87e4883-c130-446d-a0ac-316f75d07012" containerID="c3ab90dacf1963f2c3dd8150a4b2a8a601f33dde6bc8adff5aec6f2249e995ab" exitCode=0
Aug 05 18:08:52.718229 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.718160    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-625864-bcd6fbccb-lm272"
Aug 05 18:08:52.718229 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.718172    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-625864-bcd6fbccb-lm272" event={"ID":"f87e4883-c130-446d-a0ac-316f75d07012","Type":"ContainerDied","Data":"c3ab90dacf1963f2c3dd8150a4b2a8a601f33dde6bc8adff5aec6f2249e995ab"}
Aug 05 18:08:52.718229 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.718207    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-625864-bcd6fbccb-lm272" event={"ID":"f87e4883-c130-446d-a0ac-316f75d07012","Type":"ContainerDied","Data":"bb68d88a7b382df1e5cdb854286f7a00118b55d6cf5a2303db088c62908291a8"}
Aug 05 18:08:52.726669 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.726651    2550 scope.go:117] "RemoveContainer" containerID="f18f071d4662cd8c2c3a8f6c3b5fcbcf6b37b17b5f50984f59d36b2ba02735b1"
Aug 05 18:08:52.726994 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:08:52.726977    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f18f071d4662cd8c2c3a8f6c3b5fcbcf6b37b17b5f50984f59d36b2ba02735b1\": container with ID starting with f18f071d4662cd8c2c3a8f6c3b5fcbcf6b37b17b5f50984f59d36b2ba02735b1 not found: ID does not exist" containerID="f18f071d4662cd8c2c3a8f6c3b5fcbcf6b37b17b5f50984f59d36b2ba02735b1"
Aug 05 18:08:52.727053 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.727001    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f18f071d4662cd8c2c3a8f6c3b5fcbcf6b37b17b5f50984f59d36b2ba02735b1"} err="failed to get container status \"f18f071d4662cd8c2c3a8f6c3b5fcbcf6b37b17b5f50984f59d36b2ba02735b1\": rpc error: code = NotFound desc = could not find container \"f18f071d4662cd8c2c3a8f6c3b5fcbcf6b37b17b5f50984f59d36b2ba02735b1\": container with ID starting with f18f071d4662cd8c2c3a8f6c3b5fcbcf6b37b17b5f50984f59d36b2ba02735b1 not found: ID does not exist"
Aug 05 18:08:52.727053 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.727019    2550 scope.go:117] "RemoveContainer" containerID="c3ab90dacf1963f2c3dd8150a4b2a8a601f33dde6bc8adff5aec6f2249e995ab"
Aug 05 18:08:52.734674 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.734657    2550 scope.go:117] "RemoveContainer" containerID="c3ab90dacf1963f2c3dd8150a4b2a8a601f33dde6bc8adff5aec6f2249e995ab"
Aug 05 18:08:52.734954 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:08:52.734935    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3ab90dacf1963f2c3dd8150a4b2a8a601f33dde6bc8adff5aec6f2249e995ab\": container with ID starting with c3ab90dacf1963f2c3dd8150a4b2a8a601f33dde6bc8adff5aec6f2249e995ab not found: ID does not exist" containerID="c3ab90dacf1963f2c3dd8150a4b2a8a601f33dde6bc8adff5aec6f2249e995ab"
Aug 05 18:08:52.735001 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.734962    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3ab90dacf1963f2c3dd8150a4b2a8a601f33dde6bc8adff5aec6f2249e995ab"} err="failed to get container status \"c3ab90dacf1963f2c3dd8150a4b2a8a601f33dde6bc8adff5aec6f2249e995ab\": rpc error: code = NotFound desc = could not find container \"c3ab90dacf1963f2c3dd8150a4b2a8a601f33dde6bc8adff5aec6f2249e995ab\": container with ID starting with c3ab90dacf1963f2c3dd8150a4b2a8a601f33dde6bc8adff5aec6f2249e995ab not found: ID does not exist"
Aug 05 18:08:52.739429 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.739406    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-625864-758d5c8d9b-m87g4"]
Aug 05 18:08:52.744766 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.744746    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-625864-758d5c8d9b-m87g4"]
Aug 05 18:08:52.755717 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.755672    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-625864-bcd6fbccb-lm272"]
Aug 05 18:08:52.760220 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:52.760199    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-625864-bcd6fbccb-lm272"]
Aug 05 18:08:53.709685 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:53.709646    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="944bb3c9-92ae-4556-beaf-5e7578ca9e6f" path="/var/lib/kubelet/pods/944bb3c9-92ae-4556-beaf-5e7578ca9e6f/volumes"
Aug 05 18:08:53.710073 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:53.710021    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f87e4883-c130-446d-a0ac-316f75d07012" path="/var/lib/kubelet/pods/f87e4883-c130-446d-a0ac-316f75d07012/volumes"
Aug 05 18:08:59.610610 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.610544    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"]
Aug 05 18:08:59.611161 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.611076    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="944bb3c9-92ae-4556-beaf-5e7578ca9e6f" containerName="maas-api"
Aug 05 18:08:59.611161 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.611094    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="944bb3c9-92ae-4556-beaf-5e7578ca9e6f" containerName="maas-api"
Aug 05 18:08:59.611161 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.611110    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="854423ae-2cd8-43a5-8c2c-e3dc1d257c7d" containerName="payload-processing"
Aug 05 18:08:59.611161 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.611118    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="854423ae-2cd8-43a5-8c2c-e3dc1d257c7d" containerName="payload-processing"
Aug 05 18:08:59.611161 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.611135    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f87e4883-c130-446d-a0ac-316f75d07012" containerName="payload-processing"
Aug 05 18:08:59.611161 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.611143    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="f87e4883-c130-446d-a0ac-316f75d07012" containerName="payload-processing"
Aug 05 18:08:59.611161 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.611159    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cebbe6e-049e-4027-9449-bbf2ca345297" containerName="istio-proxy"
Aug 05 18:08:59.611513 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.611168    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cebbe6e-049e-4027-9449-bbf2ca345297" containerName="istio-proxy"
Aug 05 18:08:59.611513 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.611178    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a54206a-c639-435a-92ec-ca9dcd879f32" containerName="maas-api"
Aug 05 18:08:59.611513 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.611188    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a54206a-c639-435a-92ec-ca9dcd879f32" containerName="maas-api"
Aug 05 18:08:59.611513 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.611202    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cebbe6e-049e-4027-9449-bbf2ca345297" containerName="istio-proxy"
Aug 05 18:08:59.611513 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.611210    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cebbe6e-049e-4027-9449-bbf2ca345297" containerName="istio-proxy"
Aug 05 18:08:59.611513 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.611292    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="854423ae-2cd8-43a5-8c2c-e3dc1d257c7d" containerName="payload-processing"
Aug 05 18:08:59.611513 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.611305    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cebbe6e-049e-4027-9449-bbf2ca345297" containerName="istio-proxy"
Aug 05 18:08:59.611513 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.611312    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cebbe6e-049e-4027-9449-bbf2ca345297" containerName="istio-proxy"
Aug 05 18:08:59.611513 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.611323    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="944bb3c9-92ae-4556-beaf-5e7578ca9e6f" containerName="maas-api"
Aug 05 18:08:59.611513 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.611334    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a54206a-c639-435a-92ec-ca9dcd879f32" containerName="maas-api"
Aug 05 18:08:59.611513 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.611342    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="f87e4883-c130-446d-a0ac-316f75d07012" containerName="payload-processing"
Aug 05 18:08:59.615933 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.615912    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:08:59.618472 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.618448    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-ba69fc-openshift-default-dockercfg-6wcxw\""
Aug 05 18:08:59.625279 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.625256    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"]
Aug 05 18:08:59.723480 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.723444    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4de64bda-e7bf-422f-991f-238208f1cab8-istio-token\") pod \"e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") " pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:08:59.723654 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.723486    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4de64bda-e7bf-422f-991f-238208f1cab8-istio-podinfo\") pod \"e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") " pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:08:59.723654 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.723541    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-credential-socket\") pod \"e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") " pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:08:59.723654 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.723559    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-istio-envoy\") pod \"e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") " pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:08:59.723654 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.723597    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-workload-socket\") pod \"e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") " pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:08:59.723654 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.723619    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-istio-data\") pod \"e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") " pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:08:59.723654 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.723635    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4de64bda-e7bf-422f-991f-238208f1cab8-istiod-ca-cert\") pod \"e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") " pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:08:59.723886 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.723761    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-workload-certs\") pod \"e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") " pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:08:59.723886 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.723803    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dct7\" (UniqueName: \"kubernetes.io/projected/4de64bda-e7bf-422f-991f-238208f1cab8-kube-api-access-9dct7\") pod \"e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") " pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:08:59.824468 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.824430    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-istio-data\") pod \"e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") " pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:08:59.824468 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.824473    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4de64bda-e7bf-422f-991f-238208f1cab8-istiod-ca-cert\") pod \"e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") " pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:08:59.824762 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.824509    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-workload-certs\") pod \"e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") " pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:08:59.824762 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.824547    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dct7\" (UniqueName: \"kubernetes.io/projected/4de64bda-e7bf-422f-991f-238208f1cab8-kube-api-access-9dct7\") pod \"e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") " pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:08:59.824762 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.824576    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4de64bda-e7bf-422f-991f-238208f1cab8-istio-token\") pod \"e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") " pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:08:59.824762 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.824595    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4de64bda-e7bf-422f-991f-238208f1cab8-istio-podinfo\") pod \"e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") " pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:08:59.824974 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.824760    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-credential-socket\") pod \"e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") " pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:08:59.824974 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.824803    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-istio-envoy\") pod \"e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") " pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:08:59.824974 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.824852    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-workload-socket\") pod \"e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") " pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:08:59.824974 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.824944    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-istio-data\") pod \"e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") " pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:08:59.825187 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.824977    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-workload-certs\") pod \"e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") " pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:08:59.825187 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.825095    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-credential-socket\") pod \"e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") " pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:08:59.825300 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.825196    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-workload-socket\") pod \"e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") " pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:08:59.825384 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.825361    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4de64bda-e7bf-422f-991f-238208f1cab8-istiod-ca-cert\") pod \"e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") " pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:08:59.827049 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.827026    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-istio-envoy\") pod \"e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") " pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:08:59.827268 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.827252    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4de64bda-e7bf-422f-991f-238208f1cab8-istio-podinfo\") pod \"e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") " pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:08:59.833012 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.832987    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dct7\" (UniqueName: \"kubernetes.io/projected/4de64bda-e7bf-422f-991f-238208f1cab8-kube-api-access-9dct7\") pod \"e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") " pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:08:59.833289 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.833275    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4de64bda-e7bf-422f-991f-238208f1cab8-istio-token\") pod \"e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") " pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:08:59.928565 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:08:59.928529    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:09:00.070011 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:00.069983    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"]
Aug 05 18:09:00.071729 ip-10-0-134-46 kubenswrapper[2550]: W0805 18:09:00.071684    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4de64bda_e7bf_422f_991f_238208f1cab8.slice/crio-cb356721c8532c1d4a161eb9487ff94be7cbbf704cec74683310976e363b3cb3 WatchSource:0}: Error finding container cb356721c8532c1d4a161eb9487ff94be7cbbf704cec74683310976e363b3cb3: Status 404 returned error can't find the container with id cb356721c8532c1d4a161eb9487ff94be7cbbf704cec74683310976e363b3cb3
Aug 05 18:09:00.073672 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:00.073637    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 18:09:00.073809 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:00.073724    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 18:09:00.073809 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:00.073782    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 18:09:00.761343 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:00.761307    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp" event={"ID":"4de64bda-e7bf-422f-991f-238208f1cab8","Type":"ContainerStarted","Data":"31935cdd534616fa26b3a9e158fbe7f43e18c3b429ae2a6a80bc750ca95b7bfb"}
Aug 05 18:09:00.761757 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:00.761361    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp" event={"ID":"4de64bda-e7bf-422f-991f-238208f1cab8","Type":"ContainerStarted","Data":"cb356721c8532c1d4a161eb9487ff94be7cbbf704cec74683310976e363b3cb3"}
Aug 05 18:09:00.782350 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:00.782304    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp" podStartSLOduration=1.7822868779999999 podStartE2EDuration="1.782286878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:08:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:09:00.780747977 +0000 UTC m=+2847.762325407" watchObservedRunningTime="2026-08-05 18:09:00.782286878 +0000 UTC m=+2847.763864306"
Aug 05 18:09:00.929318 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:00.929283    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:09:01.934558 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:01.934530    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:09:02.771297 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:02.771262    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:09:02.772279 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:02.772262    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:09:03.784180 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:03.784144    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t"]
Aug 05 18:09:03.787641 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:03.787619    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t"
Aug 05 18:09:03.789983 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:03.789964    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-ba69fc-serving-cert\""
Aug 05 18:09:03.798804 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:03.798770    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t"]
Aug 05 18:09:03.863403 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:03.863367    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49rph\" (UniqueName: \"kubernetes.io/projected/75b1ba90-be69-4811-9ee5-9b686b19ad2d-kube-api-access-49rph\") pod \"maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t\" (UID: \"75b1ba90-be69-4811-9ee5-9b686b19ad2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t"
Aug 05 18:09:03.863586 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:03.863474    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75b1ba90-be69-4811-9ee5-9b686b19ad2d-maas-api-tls\") pod \"maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t\" (UID: \"75b1ba90-be69-4811-9ee5-9b686b19ad2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t"
Aug 05 18:09:03.863586 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:03.863495    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75b1ba90-be69-4811-9ee5-9b686b19ad2d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t\" (UID: \"75b1ba90-be69-4811-9ee5-9b686b19ad2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t"
Aug 05 18:09:03.964280 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:03.964245    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75b1ba90-be69-4811-9ee5-9b686b19ad2d-maas-api-tls\") pod \"maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t\" (UID: \"75b1ba90-be69-4811-9ee5-9b686b19ad2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t"
Aug 05 18:09:03.964489 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:03.964282    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75b1ba90-be69-4811-9ee5-9b686b19ad2d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t\" (UID: \"75b1ba90-be69-4811-9ee5-9b686b19ad2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t"
Aug 05 18:09:03.964489 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:03.964356    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49rph\" (UniqueName: \"kubernetes.io/projected/75b1ba90-be69-4811-9ee5-9b686b19ad2d-kube-api-access-49rph\") pod \"maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t\" (UID: \"75b1ba90-be69-4811-9ee5-9b686b19ad2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t"
Aug 05 18:09:03.964489 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:09:03.964417    2550 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-serving-cert: secret "maas-api-e2e-shared-a-ba69fc-serving-cert" not found
Aug 05 18:09:03.964662 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:09:03.964500    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75b1ba90-be69-4811-9ee5-9b686b19ad2d-maas-api-tls podName:75b1ba90-be69-4811-9ee5-9b686b19ad2d nodeName:}" failed. No retries permitted until 2026-08-05 18:09:04.46447665 +0000 UTC m=+2851.446054073 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/75b1ba90-be69-4811-9ee5-9b686b19ad2d-maas-api-tls") pod "maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t" (UID: "75b1ba90-be69-4811-9ee5-9b686b19ad2d") : secret "maas-api-e2e-shared-a-ba69fc-serving-cert" not found
Aug 05 18:09:03.965044 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:03.965025    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75b1ba90-be69-4811-9ee5-9b686b19ad2d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t\" (UID: \"75b1ba90-be69-4811-9ee5-9b686b19ad2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t"
Aug 05 18:09:03.973097 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:03.973076    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49rph\" (UniqueName: \"kubernetes.io/projected/75b1ba90-be69-4811-9ee5-9b686b19ad2d-kube-api-access-49rph\") pod \"maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t\" (UID: \"75b1ba90-be69-4811-9ee5-9b686b19ad2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t"
Aug 05 18:09:04.469039 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:04.468998    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75b1ba90-be69-4811-9ee5-9b686b19ad2d-maas-api-tls\") pod \"maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t\" (UID: \"75b1ba90-be69-4811-9ee5-9b686b19ad2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t"
Aug 05 18:09:04.471413 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:04.471384    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75b1ba90-be69-4811-9ee5-9b686b19ad2d-maas-api-tls\") pod \"maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t\" (UID: \"75b1ba90-be69-4811-9ee5-9b686b19ad2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t"
Aug 05 18:09:04.492687 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:04.492656    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz"]
Aug 05 18:09:04.496328 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:04.496307    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz"
Aug 05 18:09:04.499064 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:04.498968    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-ba69fc\""
Aug 05 18:09:04.499195 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:04.499071    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-ba69fc-dockercfg-n9dbx\""
Aug 05 18:09:04.508690 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:04.508666    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz"]
Aug 05 18:09:04.569921 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:04.569885    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k42b7\" (UniqueName: \"kubernetes.io/projected/f5e241b7-3f96-47e4-af63-e81a1c9e47d3-kube-api-access-k42b7\") pod \"payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz\" (UID: \"f5e241b7-3f96-47e4-af63-e81a1c9e47d3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz"
Aug 05 18:09:04.570092 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:04.569940    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5e241b7-3f96-47e4-af63-e81a1c9e47d3-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz\" (UID: \"f5e241b7-3f96-47e4-af63-e81a1c9e47d3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz"
Aug 05 18:09:04.670758 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:04.670692    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k42b7\" (UniqueName: \"kubernetes.io/projected/f5e241b7-3f96-47e4-af63-e81a1c9e47d3-kube-api-access-k42b7\") pod \"payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz\" (UID: \"f5e241b7-3f96-47e4-af63-e81a1c9e47d3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz"
Aug 05 18:09:04.670957 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:04.670777    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5e241b7-3f96-47e4-af63-e81a1c9e47d3-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz\" (UID: \"f5e241b7-3f96-47e4-af63-e81a1c9e47d3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz"
Aug 05 18:09:04.671467 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:04.671450    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5e241b7-3f96-47e4-af63-e81a1c9e47d3-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz\" (UID: \"f5e241b7-3f96-47e4-af63-e81a1c9e47d3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz"
Aug 05 18:09:04.679322 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:04.679294    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k42b7\" (UniqueName: \"kubernetes.io/projected/f5e241b7-3f96-47e4-af63-e81a1c9e47d3-kube-api-access-k42b7\") pod \"payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz\" (UID: \"f5e241b7-3f96-47e4-af63-e81a1c9e47d3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz"
Aug 05 18:09:04.699171 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:04.699138    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t"
Aug 05 18:09:04.808086 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:04.808047    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz"
Aug 05 18:09:04.842339 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:04.842309    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t"]
Aug 05 18:09:04.844410 ip-10-0-134-46 kubenswrapper[2550]: W0805 18:09:04.844377    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75b1ba90_be69_4811_9ee5_9b686b19ad2d.slice/crio-ac19adf5d9b71b45c7a78dd5f0bf75ddd16f5aa01fd99a9d7f157885b2845374 WatchSource:0}: Error finding container ac19adf5d9b71b45c7a78dd5f0bf75ddd16f5aa01fd99a9d7f157885b2845374: Status 404 returned error can't find the container with id ac19adf5d9b71b45c7a78dd5f0bf75ddd16f5aa01fd99a9d7f157885b2845374
Aug 05 18:09:04.957933 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:04.957904    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz"]
Aug 05 18:09:04.960287 ip-10-0-134-46 kubenswrapper[2550]: W0805 18:09:04.960250    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5e241b7_3f96_47e4_af63_e81a1c9e47d3.slice/crio-6ba8bbc9f3c6a17655b52255b01349314174d9fbfa4300e893ebec90e1bdd7b1 WatchSource:0}: Error finding container 6ba8bbc9f3c6a17655b52255b01349314174d9fbfa4300e893ebec90e1bdd7b1: Status 404 returned error can't find the container with id 6ba8bbc9f3c6a17655b52255b01349314174d9fbfa4300e893ebec90e1bdd7b1
Aug 05 18:09:05.786968 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:05.786931    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t" event={"ID":"75b1ba90-be69-4811-9ee5-9b686b19ad2d","Type":"ContainerStarted","Data":"a0f884edc389bdd02f60b8267982fcfde1c1af56d7384476c00b6a78fb1de7a5"}
Aug 05 18:09:05.787166 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:05.786993    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t" event={"ID":"75b1ba90-be69-4811-9ee5-9b686b19ad2d","Type":"ContainerStarted","Data":"ac19adf5d9b71b45c7a78dd5f0bf75ddd16f5aa01fd99a9d7f157885b2845374"}
Aug 05 18:09:05.787166 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:05.787021    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t"
Aug 05 18:09:05.788498 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:05.788477    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz" event={"ID":"f5e241b7-3f96-47e4-af63-e81a1c9e47d3","Type":"ContainerStarted","Data":"1a22b4d88d360550cd590c6972dc6337d801f217c8963c78a96e6729a64c0bd5"}
Aug 05 18:09:05.788592 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:05.788518    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz"
Aug 05 18:09:05.788592 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:05.788534    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz" event={"ID":"f5e241b7-3f96-47e4-af63-e81a1c9e47d3","Type":"ContainerStarted","Data":"6ba8bbc9f3c6a17655b52255b01349314174d9fbfa4300e893ebec90e1bdd7b1"}
Aug 05 18:09:05.804441 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:05.804398    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t" podStartSLOduration=2.463837281 podStartE2EDuration="2.804384804s" totalImagesPullingTime="340.547523ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:09:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:09:05.803502702 +0000 UTC m=+2852.785080132" watchObservedRunningTime="2026-08-05 18:09:05.804384804 +0000 UTC m=+2852.785962233"
Aug 05 18:09:05.821842 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:05.821785    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz" podStartSLOduration=1.821766077 podStartE2EDuration="1.821766077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:09:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:09:05.81994204 +0000 UTC m=+2852.801519471" watchObservedRunningTime="2026-08-05 18:09:05.821766077 +0000 UTC m=+2852.803343505"
Aug 05 18:09:11.799110 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:11.799077    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t"
Aug 05 18:09:16.795641 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:16.795609    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz"
Aug 05 18:09:16.849809 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:16.849777    2550 generic.go:356] "Generic (PLEG): container finished" podID="4de64bda-e7bf-422f-991f-238208f1cab8" containerID="31935cdd534616fa26b3a9e158fbe7f43e18c3b429ae2a6a80bc750ca95b7bfb" exitCode=0
Aug 05 18:09:16.849969 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:16.849847    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp" event={"ID":"4de64bda-e7bf-422f-991f-238208f1cab8","Type":"ContainerDied","Data":"31935cdd534616fa26b3a9e158fbe7f43e18c3b429ae2a6a80bc750ca95b7bfb"}
Aug 05 18:09:16.850230 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:16.850217    2550 scope.go:117] "RemoveContainer" containerID="31935cdd534616fa26b3a9e158fbe7f43e18c3b429ae2a6a80bc750ca95b7bfb"
Aug 05 18:09:16.851358 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:16.851330    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 18:09:16.851430 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:16.851414    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 18:09:16.851468 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:16.851454    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Aug 05 18:09:17.771254 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:17.771216    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:09:17.856766 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:17.856732    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp" event={"ID":"4de64bda-e7bf-422f-991f-238208f1cab8","Type":"ContainerStarted","Data":"8135aa59f60fcd4b7d335ee98f2f48d4004d71807c8a3ef156013fa6feea9944"}
Aug 05 18:09:17.929722 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:17.929659    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:09:17.931109 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:17.931073    2550 patch_prober.go:28] interesting pod/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Aug 05 18:09:17.931257 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:17.931140    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp" podUID="4de64bda-e7bf-422f-991f-238208f1cab8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Aug 05 18:09:18.929023 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:18.928980    2550 patch_prober.go:28] interesting pod/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Aug 05 18:09:18.929544 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:18.929054    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp" podUID="4de64bda-e7bf-422f-991f-238208f1cab8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Aug 05 18:09:19.929839 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:19.929797    2550 patch_prober.go:28] interesting pod/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Aug 05 18:09:19.930376 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:19.929868    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp" podUID="4de64bda-e7bf-422f-991f-238208f1cab8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Aug 05 18:09:20.929985 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:20.929481    2550 patch_prober.go:28] interesting pod/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Aug 05 18:09:20.929985 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:20.929553    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp" podUID="4de64bda-e7bf-422f-991f-238208f1cab8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Aug 05 18:09:21.930065 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:21.930012    2550 patch_prober.go:28] interesting pod/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Aug 05 18:09:21.930588 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:21.930089    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp" podUID="4de64bda-e7bf-422f-991f-238208f1cab8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Aug 05 18:09:22.932897 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:22.932866    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:09:22.933358 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:22.933158    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:09:22.933872 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:22.933857    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:09:32.880310 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:32.880265    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz"]
Aug 05 18:09:32.884166 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:32.884146    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz"
Aug 05 18:09:32.886754 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:32.886735    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-594b65-serving-cert\""
Aug 05 18:09:32.897265 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:32.897242    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz"]
Aug 05 18:09:32.956492 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:32.956455    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a-maas-api-tls\") pod \"maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz\" (UID: \"c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz"
Aug 05 18:09:32.956676 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:32.956503    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz\" (UID: \"c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz"
Aug 05 18:09:32.956676 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:32.956533    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vtlf\" (UniqueName: \"kubernetes.io/projected/c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a-kube-api-access-8vtlf\") pod \"maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz\" (UID: \"c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz"
Aug 05 18:09:33.057614 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:33.057573    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a-maas-api-tls\") pod \"maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz\" (UID: \"c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz"
Aug 05 18:09:33.057809 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:33.057626    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz\" (UID: \"c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz"
Aug 05 18:09:33.057809 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:33.057652    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vtlf\" (UniqueName: \"kubernetes.io/projected/c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a-kube-api-access-8vtlf\") pod \"maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz\" (UID: \"c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz"
Aug 05 18:09:33.057809 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:09:33.057756    2550 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-serving-cert: secret "maas-api-e2e-shared-b-594b65-serving-cert" not found
Aug 05 18:09:33.057963 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:09:33.057844    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a-maas-api-tls podName:c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a nodeName:}" failed. No retries permitted until 2026-08-05 18:09:33.557823577 +0000 UTC m=+2880.539400983 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a-maas-api-tls") pod "maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz" (UID: "c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a") : secret "maas-api-e2e-shared-b-594b65-serving-cert" not found
Aug 05 18:09:33.058357 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:33.058338    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz\" (UID: \"c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz"
Aug 05 18:09:33.066632 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:33.066611    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vtlf\" (UniqueName: \"kubernetes.io/projected/c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a-kube-api-access-8vtlf\") pod \"maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz\" (UID: \"c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz"
Aug 05 18:09:33.505941 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:33.505912    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb"]
Aug 05 18:09:33.509406 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:33.509385    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb"
Aug 05 18:09:33.511970 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:33.511941    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-594b65-dockercfg-zm52x\""
Aug 05 18:09:33.512086 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:33.512041    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-594b65\""
Aug 05 18:09:33.521213 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:33.521188    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb"]
Aug 05 18:09:33.563163 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:33.563140    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ns4x\" (UniqueName: \"kubernetes.io/projected/273d04d4-27b2-4c47-97bc-afb962cf168d-kube-api-access-8ns4x\") pod \"payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb\" (UID: \"273d04d4-27b2-4c47-97bc-afb962cf168d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb"
Aug 05 18:09:33.563278 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:33.563208    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/273d04d4-27b2-4c47-97bc-afb962cf168d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb\" (UID: \"273d04d4-27b2-4c47-97bc-afb962cf168d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb"
Aug 05 18:09:33.563278 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:33.563255    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a-maas-api-tls\") pod \"maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz\" (UID: \"c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz"
Aug 05 18:09:33.565436 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:33.565419    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a-maas-api-tls\") pod \"maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz\" (UID: \"c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz"
Aug 05 18:09:33.663802 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:33.663767    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/273d04d4-27b2-4c47-97bc-afb962cf168d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb\" (UID: \"273d04d4-27b2-4c47-97bc-afb962cf168d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb"
Aug 05 18:09:33.663967 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:33.663838    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ns4x\" (UniqueName: \"kubernetes.io/projected/273d04d4-27b2-4c47-97bc-afb962cf168d-kube-api-access-8ns4x\") pod \"payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb\" (UID: \"273d04d4-27b2-4c47-97bc-afb962cf168d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb"
Aug 05 18:09:33.664419 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:33.664399    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/273d04d4-27b2-4c47-97bc-afb962cf168d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb\" (UID: \"273d04d4-27b2-4c47-97bc-afb962cf168d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb"
Aug 05 18:09:33.674384 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:33.674356    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ns4x\" (UniqueName: \"kubernetes.io/projected/273d04d4-27b2-4c47-97bc-afb962cf168d-kube-api-access-8ns4x\") pod \"payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb\" (UID: \"273d04d4-27b2-4c47-97bc-afb962cf168d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb"
Aug 05 18:09:33.794224 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:33.794156    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz"
Aug 05 18:09:33.822084 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:33.822055    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb"
Aug 05 18:09:33.944775 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:33.944745    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz"]
Aug 05 18:09:33.948008 ip-10-0-134-46 kubenswrapper[2550]: W0805 18:09:33.947955    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5d5cec4_cff6_42e7_87fc_30e91b0c8c4a.slice/crio-80d20da7a81b31b2212d5ac26d83be7e9bc610cbb7abfa2e9bf7434bd519c318 WatchSource:0}: Error finding container 80d20da7a81b31b2212d5ac26d83be7e9bc610cbb7abfa2e9bf7434bd519c318: Status 404 returned error can't find the container with id 80d20da7a81b31b2212d5ac26d83be7e9bc610cbb7abfa2e9bf7434bd519c318
Aug 05 18:09:34.181465 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:34.181433    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb"]
Aug 05 18:09:34.183770 ip-10-0-134-46 kubenswrapper[2550]: W0805 18:09:34.183742    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod273d04d4_27b2_4c47_97bc_afb962cf168d.slice/crio-7c34aa28cad0278c5884465b59db2ab6c0dcbc7944bf0dbb34dae51e0167233d WatchSource:0}: Error finding container 7c34aa28cad0278c5884465b59db2ab6c0dcbc7944bf0dbb34dae51e0167233d: Status 404 returned error can't find the container with id 7c34aa28cad0278c5884465b59db2ab6c0dcbc7944bf0dbb34dae51e0167233d
Aug 05 18:09:34.941604 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:34.941563    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz" event={"ID":"c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a","Type":"ContainerStarted","Data":"bccd6aa65205cc0c4d80474eb708c0a992506b569104a91a026f3eeb564907c9"}
Aug 05 18:09:34.941825 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:34.941624    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz" event={"ID":"c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a","Type":"ContainerStarted","Data":"80d20da7a81b31b2212d5ac26d83be7e9bc610cbb7abfa2e9bf7434bd519c318"}
Aug 05 18:09:34.941825 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:34.941680    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz"
Aug 05 18:09:34.943133 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:34.943105    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb" event={"ID":"273d04d4-27b2-4c47-97bc-afb962cf168d","Type":"ContainerStarted","Data":"01b553b48cb27db0c78cdc80d151b40fcaecce07c913ecb863b63a3f97bb7b92"}
Aug 05 18:09:34.943258 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:34.943146    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb" event={"ID":"273d04d4-27b2-4c47-97bc-afb962cf168d","Type":"ContainerStarted","Data":"7c34aa28cad0278c5884465b59db2ab6c0dcbc7944bf0dbb34dae51e0167233d"}
Aug 05 18:09:34.943258 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:34.943188    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb"
Aug 05 18:09:34.961637 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:34.961583    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz" podStartSLOduration=2.523147657 podStartE2EDuration="2.961566789s" totalImagesPullingTime="438.419132ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:09:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:09:34.961100994 +0000 UTC m=+2881.942678426" watchObservedRunningTime="2026-08-05 18:09:34.961566789 +0000 UTC m=+2881.943144221"
Aug 05 18:09:34.979870 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:34.979818    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb" podStartSLOduration=1.979803085 podStartE2EDuration="1.979803085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:09:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:09:34.978905769 +0000 UTC m=+2881.960483220" watchObservedRunningTime="2026-08-05 18:09:34.979803085 +0000 UTC m=+2881.961380514"
Aug 05 18:09:40.954130 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:40.954098    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz"
Aug 05 18:09:45.950853 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:09:45.950814    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb"
Aug 05 18:11:33.703561 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:11:33.703536    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 18:11:33.842136 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:11:33.842109    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 18:15:38.287958 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:38.287925    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"]
Aug 05 18:15:38.291753 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:38.291734    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"
Aug 05 18:15:38.296209 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:38.296186    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ba69fc\"/\"default-dockercfg-xvpqv\""
Aug 05 18:15:38.296327 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:38.296186    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ba69fc\"/\"rate-test-model-ba69fc-kserve-self-signed-certs\""
Aug 05 18:15:38.296481 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:38.296467    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ba69fc\"/\"kube-root-ca.crt\""
Aug 05 18:15:38.296723 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:38.296692    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ba69fc\"/\"openshift-service-ca.crt\""
Aug 05 18:15:38.302657 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:38.302636    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"]
Aug 05 18:15:38.369869 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:38.369838    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/294e91df-d3de-4faa-816a-3ed9f7abef8d-tls-certs\") pod \"rate-test-model-ba69fc-kserve-5768788c5-s2gxk\" (UID: \"294e91df-d3de-4faa-816a-3ed9f7abef8d\") " pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"
Aug 05 18:15:38.369869 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:38.369871    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/294e91df-d3de-4faa-816a-3ed9f7abef8d-dshm\") pod \"rate-test-model-ba69fc-kserve-5768788c5-s2gxk\" (UID: \"294e91df-d3de-4faa-816a-3ed9f7abef8d\") " pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"
Aug 05 18:15:38.370066 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:38.369917    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/294e91df-d3de-4faa-816a-3ed9f7abef8d-model-cache\") pod \"rate-test-model-ba69fc-kserve-5768788c5-s2gxk\" (UID: \"294e91df-d3de-4faa-816a-3ed9f7abef8d\") " pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"
Aug 05 18:15:38.370066 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:38.369968    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgz99\" (UniqueName: \"kubernetes.io/projected/294e91df-d3de-4faa-816a-3ed9f7abef8d-kube-api-access-hgz99\") pod \"rate-test-model-ba69fc-kserve-5768788c5-s2gxk\" (UID: \"294e91df-d3de-4faa-816a-3ed9f7abef8d\") " pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"
Aug 05 18:15:38.370066 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:38.370032    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/294e91df-d3de-4faa-816a-3ed9f7abef8d-home\") pod \"rate-test-model-ba69fc-kserve-5768788c5-s2gxk\" (UID: \"294e91df-d3de-4faa-816a-3ed9f7abef8d\") " pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"
Aug 05 18:15:38.370168 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:38.370068    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/294e91df-d3de-4faa-816a-3ed9f7abef8d-tmp-dir\") pod \"rate-test-model-ba69fc-kserve-5768788c5-s2gxk\" (UID: \"294e91df-d3de-4faa-816a-3ed9f7abef8d\") " pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"
Aug 05 18:15:38.471229 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:38.471195    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgz99\" (UniqueName: \"kubernetes.io/projected/294e91df-d3de-4faa-816a-3ed9f7abef8d-kube-api-access-hgz99\") pod \"rate-test-model-ba69fc-kserve-5768788c5-s2gxk\" (UID: \"294e91df-d3de-4faa-816a-3ed9f7abef8d\") " pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"
Aug 05 18:15:38.471381 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:38.471236    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/294e91df-d3de-4faa-816a-3ed9f7abef8d-home\") pod \"rate-test-model-ba69fc-kserve-5768788c5-s2gxk\" (UID: \"294e91df-d3de-4faa-816a-3ed9f7abef8d\") " pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"
Aug 05 18:15:38.471381 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:38.471258    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/294e91df-d3de-4faa-816a-3ed9f7abef8d-tmp-dir\") pod \"rate-test-model-ba69fc-kserve-5768788c5-s2gxk\" (UID: \"294e91df-d3de-4faa-816a-3ed9f7abef8d\") " pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"
Aug 05 18:15:38.471381 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:38.471286    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/294e91df-d3de-4faa-816a-3ed9f7abef8d-tls-certs\") pod \"rate-test-model-ba69fc-kserve-5768788c5-s2gxk\" (UID: \"294e91df-d3de-4faa-816a-3ed9f7abef8d\") " pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"
Aug 05 18:15:38.471381 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:38.471310    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/294e91df-d3de-4faa-816a-3ed9f7abef8d-dshm\") pod \"rate-test-model-ba69fc-kserve-5768788c5-s2gxk\" (UID: \"294e91df-d3de-4faa-816a-3ed9f7abef8d\") " pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"
Aug 05 18:15:38.471381 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:38.471369    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/294e91df-d3de-4faa-816a-3ed9f7abef8d-model-cache\") pod \"rate-test-model-ba69fc-kserve-5768788c5-s2gxk\" (UID: \"294e91df-d3de-4faa-816a-3ed9f7abef8d\") " pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"
Aug 05 18:15:38.471646 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:38.471601    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/294e91df-d3de-4faa-816a-3ed9f7abef8d-tmp-dir\") pod \"rate-test-model-ba69fc-kserve-5768788c5-s2gxk\" (UID: \"294e91df-d3de-4faa-816a-3ed9f7abef8d\") " pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"
Aug 05 18:15:38.471734 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:38.471667    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/294e91df-d3de-4faa-816a-3ed9f7abef8d-home\") pod \"rate-test-model-ba69fc-kserve-5768788c5-s2gxk\" (UID: \"294e91df-d3de-4faa-816a-3ed9f7abef8d\") " pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"
Aug 05 18:15:38.471800 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:38.471737    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/294e91df-d3de-4faa-816a-3ed9f7abef8d-model-cache\") pod \"rate-test-model-ba69fc-kserve-5768788c5-s2gxk\" (UID: \"294e91df-d3de-4faa-816a-3ed9f7abef8d\") " pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"
Aug 05 18:15:38.473526 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:38.473504    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/294e91df-d3de-4faa-816a-3ed9f7abef8d-dshm\") pod \"rate-test-model-ba69fc-kserve-5768788c5-s2gxk\" (UID: \"294e91df-d3de-4faa-816a-3ed9f7abef8d\") " pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"
Aug 05 18:15:38.473712 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:38.473679    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/294e91df-d3de-4faa-816a-3ed9f7abef8d-tls-certs\") pod \"rate-test-model-ba69fc-kserve-5768788c5-s2gxk\" (UID: \"294e91df-d3de-4faa-816a-3ed9f7abef8d\") " pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"
Aug 05 18:15:38.479024 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:38.479000    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgz99\" (UniqueName: \"kubernetes.io/projected/294e91df-d3de-4faa-816a-3ed9f7abef8d-kube-api-access-hgz99\") pod \"rate-test-model-ba69fc-kserve-5768788c5-s2gxk\" (UID: \"294e91df-d3de-4faa-816a-3ed9f7abef8d\") " pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"
Aug 05 18:15:38.601173 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:38.601086    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"
Aug 05 18:15:38.735881 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:38.735857    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"]
Aug 05 18:15:38.737881 ip-10-0-134-46 kubenswrapper[2550]: W0805 18:15:38.737855    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod294e91df_d3de_4faa_816a_3ed9f7abef8d.slice/crio-7c879d0f9adfc7386f247a65d6f31b40e7e194c205388a450915f8f48c164573 WatchSource:0}: Error finding container 7c879d0f9adfc7386f247a65d6f31b40e7e194c205388a450915f8f48c164573: Status 404 returned error can't find the container with id 7c879d0f9adfc7386f247a65d6f31b40e7e194c205388a450915f8f48c164573
Aug 05 18:15:39.704882 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:39.704845    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk" event={"ID":"294e91df-d3de-4faa-816a-3ed9f7abef8d","Type":"ContainerStarted","Data":"474b10f08eec392a61752ca59d0033fc9635fed60c6cc6dda2691534f3c59f7b"}
Aug 05 18:15:39.705254 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:39.704890    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk" event={"ID":"294e91df-d3de-4faa-816a-3ed9f7abef8d","Type":"ContainerStarted","Data":"7c879d0f9adfc7386f247a65d6f31b40e7e194c205388a450915f8f48c164573"}
Aug 05 18:15:39.723493 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:39.723448    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk" podStartSLOduration=1.723432448 podStartE2EDuration="1.723432448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:15:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:15:39.722263039 +0000 UTC m=+3246.703840471" watchObservedRunningTime="2026-08-05 18:15:39.723432448 +0000 UTC m=+3246.705009877"
Aug 05 18:15:48.601906 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:48.601863    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"
Aug 05 18:15:48.602308 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:48.601921    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"
Aug 05 18:15:48.614934 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:48.614912    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"
Aug 05 18:15:48.755786 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:15:48.755760    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"
Aug 05 18:16:33.731962 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:16:33.731935    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 18:16:33.870204 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:16:33.870175    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 18:17:23.591779 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:23.588788    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"]
Aug 05 18:17:38.614416 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:38.614369    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk" podUID="294e91df-d3de-4faa-816a-3ed9f7abef8d" containerName="main" containerID="cri-o://474b10f08eec392a61752ca59d0033fc9635fed60c6cc6dda2691534f3c59f7b" gracePeriod=45
Aug 05 18:17:38.745147 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:38.745105    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk" podUID="294e91df-d3de-4faa-816a-3ed9f7abef8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": dial tcp 10.132.0.78:8000: connect: connection refused"
Aug 05 18:17:38.872158 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:38.872103    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"
Aug 05 18:17:38.979492 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:38.979460    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/294e91df-d3de-4faa-816a-3ed9f7abef8d-dshm\") pod \"294e91df-d3de-4faa-816a-3ed9f7abef8d\" (UID: \"294e91df-d3de-4faa-816a-3ed9f7abef8d\") "
Aug 05 18:17:38.979492 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:38.979506    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/294e91df-d3de-4faa-816a-3ed9f7abef8d-model-cache\") pod \"294e91df-d3de-4faa-816a-3ed9f7abef8d\" (UID: \"294e91df-d3de-4faa-816a-3ed9f7abef8d\") "
Aug 05 18:17:38.979808 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:38.979538    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/294e91df-d3de-4faa-816a-3ed9f7abef8d-home\") pod \"294e91df-d3de-4faa-816a-3ed9f7abef8d\" (UID: \"294e91df-d3de-4faa-816a-3ed9f7abef8d\") "
Aug 05 18:17:38.979808 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:38.979568    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgz99\" (UniqueName: \"kubernetes.io/projected/294e91df-d3de-4faa-816a-3ed9f7abef8d-kube-api-access-hgz99\") pod \"294e91df-d3de-4faa-816a-3ed9f7abef8d\" (UID: \"294e91df-d3de-4faa-816a-3ed9f7abef8d\") "
Aug 05 18:17:38.979808 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:38.979616    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/294e91df-d3de-4faa-816a-3ed9f7abef8d-tls-certs\") pod \"294e91df-d3de-4faa-816a-3ed9f7abef8d\" (UID: \"294e91df-d3de-4faa-816a-3ed9f7abef8d\") "
Aug 05 18:17:38.979808 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:38.979658    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/294e91df-d3de-4faa-816a-3ed9f7abef8d-tmp-dir\") pod \"294e91df-d3de-4faa-816a-3ed9f7abef8d\" (UID: \"294e91df-d3de-4faa-816a-3ed9f7abef8d\") "
Aug 05 18:17:38.980072 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:38.979803    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/294e91df-d3de-4faa-816a-3ed9f7abef8d-model-cache" (OuterVolumeSpecName: "model-cache") pod "294e91df-d3de-4faa-816a-3ed9f7abef8d" (UID: "294e91df-d3de-4faa-816a-3ed9f7abef8d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:17:38.980135 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:38.979893    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/294e91df-d3de-4faa-816a-3ed9f7abef8d-home" (OuterVolumeSpecName: "home") pod "294e91df-d3de-4faa-816a-3ed9f7abef8d" (UID: "294e91df-d3de-4faa-816a-3ed9f7abef8d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:17:38.980135 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:38.980035    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/294e91df-d3de-4faa-816a-3ed9f7abef8d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "294e91df-d3de-4faa-816a-3ed9f7abef8d" (UID: "294e91df-d3de-4faa-816a-3ed9f7abef8d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:17:38.981878 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:38.981853    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/294e91df-d3de-4faa-816a-3ed9f7abef8d-dshm" (OuterVolumeSpecName: "dshm") pod "294e91df-d3de-4faa-816a-3ed9f7abef8d" (UID: "294e91df-d3de-4faa-816a-3ed9f7abef8d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:17:38.981878 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:38.981870    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/294e91df-d3de-4faa-816a-3ed9f7abef8d-kube-api-access-hgz99" (OuterVolumeSpecName: "kube-api-access-hgz99") pod "294e91df-d3de-4faa-816a-3ed9f7abef8d" (UID: "294e91df-d3de-4faa-816a-3ed9f7abef8d"). InnerVolumeSpecName "kube-api-access-hgz99". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:17:38.982022 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:38.981888    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/294e91df-d3de-4faa-816a-3ed9f7abef8d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "294e91df-d3de-4faa-816a-3ed9f7abef8d" (UID: "294e91df-d3de-4faa-816a-3ed9f7abef8d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:17:39.080601 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:39.080561    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/294e91df-d3de-4faa-816a-3ed9f7abef8d-home\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:17:39.080601 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:39.080594    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgz99\" (UniqueName: \"kubernetes.io/projected/294e91df-d3de-4faa-816a-3ed9f7abef8d-kube-api-access-hgz99\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:17:39.080601 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:39.080605    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/294e91df-d3de-4faa-816a-3ed9f7abef8d-tls-certs\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:17:39.080882 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:39.080617    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/294e91df-d3de-4faa-816a-3ed9f7abef8d-tmp-dir\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:17:39.080882 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:39.080624    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/294e91df-d3de-4faa-816a-3ed9f7abef8d-dshm\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:17:39.080882 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:39.080632    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/294e91df-d3de-4faa-816a-3ed9f7abef8d-model-cache\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:17:39.277899 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:39.277863    2550 generic.go:356] "Generic (PLEG): container finished" podID="294e91df-d3de-4faa-816a-3ed9f7abef8d" containerID="474b10f08eec392a61752ca59d0033fc9635fed60c6cc6dda2691534f3c59f7b" exitCode=0
Aug 05 18:17:39.277899 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:39.277904    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"
Aug 05 18:17:39.278120 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:39.277955    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk" event={"ID":"294e91df-d3de-4faa-816a-3ed9f7abef8d","Type":"ContainerDied","Data":"474b10f08eec392a61752ca59d0033fc9635fed60c6cc6dda2691534f3c59f7b"}
Aug 05 18:17:39.278120 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:39.277993    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk" event={"ID":"294e91df-d3de-4faa-816a-3ed9f7abef8d","Type":"ContainerDied","Data":"7c879d0f9adfc7386f247a65d6f31b40e7e194c205388a450915f8f48c164573"}
Aug 05 18:17:39.278120 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:39.278013    2550 scope.go:117] "RemoveContainer" containerID="474b10f08eec392a61752ca59d0033fc9635fed60c6cc6dda2691534f3c59f7b"
Aug 05 18:17:39.286873 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:39.286856    2550 scope.go:117] "RemoveContainer" containerID="474b10f08eec392a61752ca59d0033fc9635fed60c6cc6dda2691534f3c59f7b"
Aug 05 18:17:39.287124 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:17:39.287107    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"474b10f08eec392a61752ca59d0033fc9635fed60c6cc6dda2691534f3c59f7b\": container with ID starting with 474b10f08eec392a61752ca59d0033fc9635fed60c6cc6dda2691534f3c59f7b not found: ID does not exist" containerID="474b10f08eec392a61752ca59d0033fc9635fed60c6cc6dda2691534f3c59f7b"
Aug 05 18:17:39.287161 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:39.287135    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"474b10f08eec392a61752ca59d0033fc9635fed60c6cc6dda2691534f3c59f7b"} err="failed to get container status \"474b10f08eec392a61752ca59d0033fc9635fed60c6cc6dda2691534f3c59f7b\": rpc error: code = NotFound desc = could not find container \"474b10f08eec392a61752ca59d0033fc9635fed60c6cc6dda2691534f3c59f7b\": container with ID starting with 474b10f08eec392a61752ca59d0033fc9635fed60c6cc6dda2691534f3c59f7b not found: ID does not exist"
Aug 05 18:17:39.299901 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:39.299875    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"]
Aug 05 18:17:39.303623 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:39.303601    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ba69fc/rate-test-model-ba69fc-kserve-5768788c5-s2gxk"]
Aug 05 18:17:39.700217 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:39.700181    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="294e91df-d3de-4faa-816a-3ed9f7abef8d" path="/var/lib/kubelet/pods/294e91df-d3de-4faa-816a-3ed9f7abef8d/volumes"
Aug 05 18:17:47.754022 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:47.753984    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg"]
Aug 05 18:17:47.754640 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:47.754614    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="294e91df-d3de-4faa-816a-3ed9f7abef8d" containerName="main"
Aug 05 18:17:47.754640 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:47.754639    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="294e91df-d3de-4faa-816a-3ed9f7abef8d" containerName="main"
Aug 05 18:17:47.754872 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:47.754747    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="294e91df-d3de-4faa-816a-3ed9f7abef8d" containerName="main"
Aug 05 18:17:47.758263 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:47.758240    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg"
Aug 05 18:17:47.761140 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:47.761116    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-6427fe-serving-cert\""
Aug 05 18:17:47.770092 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:47.770066    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg"]
Aug 05 18:17:47.859840 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:47.859796    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3589901b-34d7-4a50-9b4e-4dad090e3917-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg\" (UID: \"3589901b-34d7-4a50-9b4e-4dad090e3917\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg"
Aug 05 18:17:47.859840 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:47.859850    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj78k\" (UniqueName: \"kubernetes.io/projected/3589901b-34d7-4a50-9b4e-4dad090e3917-kube-api-access-jj78k\") pod \"maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg\" (UID: \"3589901b-34d7-4a50-9b4e-4dad090e3917\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg"
Aug 05 18:17:47.860246 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:47.859922    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3589901b-34d7-4a50-9b4e-4dad090e3917-maas-api-tls\") pod \"maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg\" (UID: \"3589901b-34d7-4a50-9b4e-4dad090e3917\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg"
Aug 05 18:17:47.961108 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:47.961070    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3589901b-34d7-4a50-9b4e-4dad090e3917-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg\" (UID: \"3589901b-34d7-4a50-9b4e-4dad090e3917\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg"
Aug 05 18:17:47.961108 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:47.961110    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj78k\" (UniqueName: \"kubernetes.io/projected/3589901b-34d7-4a50-9b4e-4dad090e3917-kube-api-access-jj78k\") pod \"maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg\" (UID: \"3589901b-34d7-4a50-9b4e-4dad090e3917\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg"
Aug 05 18:17:47.961336 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:47.961146    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3589901b-34d7-4a50-9b4e-4dad090e3917-maas-api-tls\") pod \"maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg\" (UID: \"3589901b-34d7-4a50-9b4e-4dad090e3917\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg"
Aug 05 18:17:47.961336 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:17:47.961268    2550 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-serving-cert: secret "maas-api-e2e-ipp-a-6427fe-serving-cert" not found
Aug 05 18:17:47.961336 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:17:47.961331    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3589901b-34d7-4a50-9b4e-4dad090e3917-maas-api-tls podName:3589901b-34d7-4a50-9b4e-4dad090e3917 nodeName:}" failed. No retries permitted until 2026-08-05 18:17:48.46131484 +0000 UTC m=+3375.442892247 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3589901b-34d7-4a50-9b4e-4dad090e3917-maas-api-tls") pod "maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg" (UID: "3589901b-34d7-4a50-9b4e-4dad090e3917") : secret "maas-api-e2e-ipp-a-6427fe-serving-cert" not found
Aug 05 18:17:47.961763 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:47.961741    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3589901b-34d7-4a50-9b4e-4dad090e3917-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg\" (UID: \"3589901b-34d7-4a50-9b4e-4dad090e3917\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg"
Aug 05 18:17:47.975384 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:47.975359    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj78k\" (UniqueName: \"kubernetes.io/projected/3589901b-34d7-4a50-9b4e-4dad090e3917-kube-api-access-jj78k\") pod \"maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg\" (UID: \"3589901b-34d7-4a50-9b4e-4dad090e3917\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg"
Aug 05 18:17:48.467333 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:48.467302    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3589901b-34d7-4a50-9b4e-4dad090e3917-maas-api-tls\") pod \"maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg\" (UID: \"3589901b-34d7-4a50-9b4e-4dad090e3917\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg"
Aug 05 18:17:48.467514 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:17:48.467441    2550 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-serving-cert: secret "maas-api-e2e-ipp-a-6427fe-serving-cert" not found
Aug 05 18:17:48.467514 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:17:48.467513    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3589901b-34d7-4a50-9b4e-4dad090e3917-maas-api-tls podName:3589901b-34d7-4a50-9b4e-4dad090e3917 nodeName:}" failed. No retries permitted until 2026-08-05 18:17:49.467497723 +0000 UTC m=+3376.449075131 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3589901b-34d7-4a50-9b4e-4dad090e3917-maas-api-tls") pod "maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg" (UID: "3589901b-34d7-4a50-9b4e-4dad090e3917") : secret "maas-api-e2e-ipp-a-6427fe-serving-cert" not found
Aug 05 18:17:48.528636 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:48.528600    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2"]
Aug 05 18:17:48.532254 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:48.532238    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2"
Aug 05 18:17:48.534684 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:48.534664    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-6427fe-dockercfg-2l769\""
Aug 05 18:17:48.534821 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:48.534666    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-6427fe\""
Aug 05 18:17:48.542354 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:48.542320    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2"]
Aug 05 18:17:48.669496 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:48.669456    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c048be46-95b0-4620-a80e-9a0ffdc56961-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2\" (UID: \"c048be46-95b0-4620-a80e-9a0ffdc56961\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2"
Aug 05 18:17:48.669682 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:48.669541    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77467\" (UniqueName: \"kubernetes.io/projected/c048be46-95b0-4620-a80e-9a0ffdc56961-kube-api-access-77467\") pod \"payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2\" (UID: \"c048be46-95b0-4620-a80e-9a0ffdc56961\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2"
Aug 05 18:17:48.770980 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:48.770890    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77467\" (UniqueName: \"kubernetes.io/projected/c048be46-95b0-4620-a80e-9a0ffdc56961-kube-api-access-77467\") pod \"payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2\" (UID: \"c048be46-95b0-4620-a80e-9a0ffdc56961\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2"
Aug 05 18:17:48.770980 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:48.770977    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c048be46-95b0-4620-a80e-9a0ffdc56961-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2\" (UID: \"c048be46-95b0-4620-a80e-9a0ffdc56961\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2"
Aug 05 18:17:48.771646 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:48.771625    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c048be46-95b0-4620-a80e-9a0ffdc56961-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2\" (UID: \"c048be46-95b0-4620-a80e-9a0ffdc56961\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2"
Aug 05 18:17:48.780638 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:48.780601    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77467\" (UniqueName: \"kubernetes.io/projected/c048be46-95b0-4620-a80e-9a0ffdc56961-kube-api-access-77467\") pod \"payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2\" (UID: \"c048be46-95b0-4620-a80e-9a0ffdc56961\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2"
Aug 05 18:17:48.843579 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:48.843545    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2"
Aug 05 18:17:48.982067 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:48.982038    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2"]
Aug 05 18:17:48.985523 ip-10-0-134-46 kubenswrapper[2550]: W0805 18:17:48.985497    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc048be46_95b0_4620_a80e_9a0ffdc56961.slice/crio-e6a8b6a1aadc9f902995e9cfd1597a961cbedda3e57929c3d365c1cc9b86985d WatchSource:0}: Error finding container e6a8b6a1aadc9f902995e9cfd1597a961cbedda3e57929c3d365c1cc9b86985d: Status 404 returned error can't find the container with id e6a8b6a1aadc9f902995e9cfd1597a961cbedda3e57929c3d365c1cc9b86985d
Aug 05 18:17:49.331413 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:49.331323    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2" event={"ID":"c048be46-95b0-4620-a80e-9a0ffdc56961","Type":"ContainerStarted","Data":"900e8058b36aa040ea073b46a04ceff05b2d147b8d94c61d5848b350c64277bf"}
Aug 05 18:17:49.331413 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:49.331377    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2" event={"ID":"c048be46-95b0-4620-a80e-9a0ffdc56961","Type":"ContainerStarted","Data":"e6a8b6a1aadc9f902995e9cfd1597a961cbedda3e57929c3d365c1cc9b86985d"}
Aug 05 18:17:49.331596 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:49.331496    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2"
Aug 05 18:17:49.349498 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:49.349449    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2" podStartSLOduration=1.34943497 podStartE2EDuration="1.34943497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:17:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:17:49.346817882 +0000 UTC m=+3376.328395314" watchObservedRunningTime="2026-08-05 18:17:49.34943497 +0000 UTC m=+3376.331012399"
Aug 05 18:17:49.478335 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:49.478298    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3589901b-34d7-4a50-9b4e-4dad090e3917-maas-api-tls\") pod \"maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg\" (UID: \"3589901b-34d7-4a50-9b4e-4dad090e3917\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg"
Aug 05 18:17:49.480850 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:49.480825    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3589901b-34d7-4a50-9b4e-4dad090e3917-maas-api-tls\") pod \"maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg\" (UID: \"3589901b-34d7-4a50-9b4e-4dad090e3917\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg"
Aug 05 18:17:49.570691 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:49.570656    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg"
Aug 05 18:17:49.712790 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:49.712761    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg"]
Aug 05 18:17:49.715245 ip-10-0-134-46 kubenswrapper[2550]: W0805 18:17:49.715211    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3589901b_34d7_4a50_9b4e_4dad090e3917.slice/crio-a70a696a097f4564f5e6e0a221f50f975c256c242386c1afb7e3a029f33eb00d WatchSource:0}: Error finding container a70a696a097f4564f5e6e0a221f50f975c256c242386c1afb7e3a029f33eb00d: Status 404 returned error can't find the container with id a70a696a097f4564f5e6e0a221f50f975c256c242386c1afb7e3a029f33eb00d
Aug 05 18:17:49.716731 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:49.716713    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 18:17:50.337996 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:50.337962    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg" event={"ID":"3589901b-34d7-4a50-9b4e-4dad090e3917","Type":"ContainerStarted","Data":"2e7ada5f70f06b151683f60f4b62d4160d9560f790208c92d9579f7d5838f6d4"}
Aug 05 18:17:50.338377 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:50.338018    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg" event={"ID":"3589901b-34d7-4a50-9b4e-4dad090e3917","Type":"ContainerStarted","Data":"a70a696a097f4564f5e6e0a221f50f975c256c242386c1afb7e3a029f33eb00d"}
Aug 05 18:17:50.338377 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:50.338107    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg"
Aug 05 18:17:50.357228 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:50.357175    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg" podStartSLOduration=2.845746458 podStartE2EDuration="3.357157915s" totalImagesPullingTime="511.411457ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:17:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:17:50.356200616 +0000 UTC m=+3377.337778045" watchObservedRunningTime="2026-08-05 18:17:50.357157915 +0000 UTC m=+3377.338735344"
Aug 05 18:17:56.350784 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:17:56.350744    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg"
Aug 05 18:18:00.340892 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:18:00.340857    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2"
Aug 05 18:18:47.476291 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:18:47.476255    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-crrck_b762d708-104b-4bbb-ba46-13a3aff27b83/payload-processing/0.log"
Aug 05 18:20:03.126505 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:03.126432    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-crrck_b762d708-104b-4bbb-ba46-13a3aff27b83/payload-processing/0.log"
Aug 05 18:20:07.632624 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:07.632586    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746"]
Aug 05 18:20:07.636811 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:07.636788    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746"
Aug 05 18:20:07.639342 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:07.639320    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-2696ea-serving-cert\""
Aug 05 18:20:07.647494 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:07.647472    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746"]
Aug 05 18:20:07.710062 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:07.710029    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d322eeed-d6f7-42a7-979e-7d9e55e62a69-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746\" (UID: \"d322eeed-d6f7-42a7-979e-7d9e55e62a69\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746"
Aug 05 18:20:07.710238 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:07.710071    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxqxr\" (UniqueName: \"kubernetes.io/projected/d322eeed-d6f7-42a7-979e-7d9e55e62a69-kube-api-access-qxqxr\") pod \"maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746\" (UID: \"d322eeed-d6f7-42a7-979e-7d9e55e62a69\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746"
Aug 05 18:20:07.710238 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:07.710092    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d322eeed-d6f7-42a7-979e-7d9e55e62a69-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746\" (UID: \"d322eeed-d6f7-42a7-979e-7d9e55e62a69\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746"
Aug 05 18:20:07.810688 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:07.810655    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxqxr\" (UniqueName: \"kubernetes.io/projected/d322eeed-d6f7-42a7-979e-7d9e55e62a69-kube-api-access-qxqxr\") pod \"maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746\" (UID: \"d322eeed-d6f7-42a7-979e-7d9e55e62a69\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746"
Aug 05 18:20:07.810913 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:07.810740    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d322eeed-d6f7-42a7-979e-7d9e55e62a69-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746\" (UID: \"d322eeed-d6f7-42a7-979e-7d9e55e62a69\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746"
Aug 05 18:20:07.810986 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:07.810927    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d322eeed-d6f7-42a7-979e-7d9e55e62a69-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746\" (UID: \"d322eeed-d6f7-42a7-979e-7d9e55e62a69\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746"
Aug 05 18:20:07.811535 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:07.811500    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d322eeed-d6f7-42a7-979e-7d9e55e62a69-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746\" (UID: \"d322eeed-d6f7-42a7-979e-7d9e55e62a69\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746"
Aug 05 18:20:07.813493 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:07.813472    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d322eeed-d6f7-42a7-979e-7d9e55e62a69-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746\" (UID: \"d322eeed-d6f7-42a7-979e-7d9e55e62a69\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746"
Aug 05 18:20:07.820064 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:07.820038    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxqxr\" (UniqueName: \"kubernetes.io/projected/d322eeed-d6f7-42a7-979e-7d9e55e62a69-kube-api-access-qxqxr\") pod \"maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746\" (UID: \"d322eeed-d6f7-42a7-979e-7d9e55e62a69\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746"
Aug 05 18:20:07.948556 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:07.948523    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746"
Aug 05 18:20:08.095775 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:08.095746    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746"]
Aug 05 18:20:08.098329 ip-10-0-134-46 kubenswrapper[2550]: W0805 18:20:08.098297    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd322eeed_d6f7_42a7_979e_7d9e55e62a69.slice/crio-476f72489ff4bddd58ab97a5154f970131c43ca7e065c5f621f55680f11d9faa WatchSource:0}: Error finding container 476f72489ff4bddd58ab97a5154f970131c43ca7e065c5f621f55680f11d9faa: Status 404 returned error can't find the container with id 476f72489ff4bddd58ab97a5154f970131c43ca7e065c5f621f55680f11d9faa
Aug 05 18:20:09.018001 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:09.017917    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746" event={"ID":"d322eeed-d6f7-42a7-979e-7d9e55e62a69","Type":"ContainerStarted","Data":"b005f56b7faaf4b9fe7807c30a4e99bc91e82ade1a958cd45c50efaa5e788539"}
Aug 05 18:20:09.018366 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:09.018085    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746"
Aug 05 18:20:09.018366 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:09.018104    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746" event={"ID":"d322eeed-d6f7-42a7-979e-7d9e55e62a69","Type":"ContainerStarted","Data":"476f72489ff4bddd58ab97a5154f970131c43ca7e065c5f621f55680f11d9faa"}
Aug 05 18:20:09.041851 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:09.041798    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746" podStartSLOduration=1.470462631 podStartE2EDuration="2.041783228s" totalImagesPullingTime="571.320597ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:20:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:20:09.040332029 +0000 UTC m=+3516.021909475" watchObservedRunningTime="2026-08-05 18:20:09.041783228 +0000 UTC m=+3516.023360656"
Aug 05 18:20:15.028877 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:15.028844    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746"
Aug 05 18:20:38.286397 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:38.286357    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746"]
Aug 05 18:20:38.287024 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:38.286658    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746" podUID="d322eeed-d6f7-42a7-979e-7d9e55e62a69" containerName="maas-api" containerID="cri-o://b005f56b7faaf4b9fe7807c30a4e99bc91e82ade1a958cd45c50efaa5e788539" gracePeriod=30
Aug 05 18:20:38.540170 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:38.540117    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746"
Aug 05 18:20:38.598149 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:38.598116    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d322eeed-d6f7-42a7-979e-7d9e55e62a69-maas-api-tls\") pod \"d322eeed-d6f7-42a7-979e-7d9e55e62a69\" (UID: \"d322eeed-d6f7-42a7-979e-7d9e55e62a69\") "
Aug 05 18:20:38.598346 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:38.598180    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxqxr\" (UniqueName: \"kubernetes.io/projected/d322eeed-d6f7-42a7-979e-7d9e55e62a69-kube-api-access-qxqxr\") pod \"d322eeed-d6f7-42a7-979e-7d9e55e62a69\" (UID: \"d322eeed-d6f7-42a7-979e-7d9e55e62a69\") "
Aug 05 18:20:38.598346 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:38.598259    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d322eeed-d6f7-42a7-979e-7d9e55e62a69-openshift-service-ca\") pod \"d322eeed-d6f7-42a7-979e-7d9e55e62a69\" (UID: \"d322eeed-d6f7-42a7-979e-7d9e55e62a69\") "
Aug 05 18:20:38.598575 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:38.598549    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d322eeed-d6f7-42a7-979e-7d9e55e62a69-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d322eeed-d6f7-42a7-979e-7d9e55e62a69" (UID: "d322eeed-d6f7-42a7-979e-7d9e55e62a69"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:20:38.600244 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:38.600210    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d322eeed-d6f7-42a7-979e-7d9e55e62a69-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d322eeed-d6f7-42a7-979e-7d9e55e62a69" (UID: "d322eeed-d6f7-42a7-979e-7d9e55e62a69"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:20:38.600336 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:38.600258    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d322eeed-d6f7-42a7-979e-7d9e55e62a69-kube-api-access-qxqxr" (OuterVolumeSpecName: "kube-api-access-qxqxr") pod "d322eeed-d6f7-42a7-979e-7d9e55e62a69" (UID: "d322eeed-d6f7-42a7-979e-7d9e55e62a69"). InnerVolumeSpecName "kube-api-access-qxqxr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:20:38.699899 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:38.699869    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d322eeed-d6f7-42a7-979e-7d9e55e62a69-maas-api-tls\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:20:38.699899 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:38.699898    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxqxr\" (UniqueName: \"kubernetes.io/projected/d322eeed-d6f7-42a7-979e-7d9e55e62a69-kube-api-access-qxqxr\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:20:38.700100 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:38.699910    2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d322eeed-d6f7-42a7-979e-7d9e55e62a69-openshift-service-ca\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:20:39.162739 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:39.162683    2550 generic.go:356] "Generic (PLEG): container finished" podID="d322eeed-d6f7-42a7-979e-7d9e55e62a69" containerID="b005f56b7faaf4b9fe7807c30a4e99bc91e82ade1a958cd45c50efaa5e788539" exitCode=0
Aug 05 18:20:39.162921 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:39.162759    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746"
Aug 05 18:20:39.162921 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:39.162780    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746" event={"ID":"d322eeed-d6f7-42a7-979e-7d9e55e62a69","Type":"ContainerDied","Data":"b005f56b7faaf4b9fe7807c30a4e99bc91e82ade1a958cd45c50efaa5e788539"}
Aug 05 18:20:39.162921 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:39.162840    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746" event={"ID":"d322eeed-d6f7-42a7-979e-7d9e55e62a69","Type":"ContainerDied","Data":"476f72489ff4bddd58ab97a5154f970131c43ca7e065c5f621f55680f11d9faa"}
Aug 05 18:20:39.162921 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:39.162871    2550 scope.go:117] "RemoveContainer" containerID="b005f56b7faaf4b9fe7807c30a4e99bc91e82ade1a958cd45c50efaa5e788539"
Aug 05 18:20:39.181532 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:39.181514    2550 scope.go:117] "RemoveContainer" containerID="b005f56b7faaf4b9fe7807c30a4e99bc91e82ade1a958cd45c50efaa5e788539"
Aug 05 18:20:39.181808 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:20:39.181789    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b005f56b7faaf4b9fe7807c30a4e99bc91e82ade1a958cd45c50efaa5e788539\": container with ID starting with b005f56b7faaf4b9fe7807c30a4e99bc91e82ade1a958cd45c50efaa5e788539 not found: ID does not exist" containerID="b005f56b7faaf4b9fe7807c30a4e99bc91e82ade1a958cd45c50efaa5e788539"
Aug 05 18:20:39.181884 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:39.181817    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b005f56b7faaf4b9fe7807c30a4e99bc91e82ade1a958cd45c50efaa5e788539"} err="failed to get container status \"b005f56b7faaf4b9fe7807c30a4e99bc91e82ade1a958cd45c50efaa5e788539\": rpc error: code = NotFound desc = could not find container \"b005f56b7faaf4b9fe7807c30a4e99bc91e82ade1a958cd45c50efaa5e788539\": container with ID starting with b005f56b7faaf4b9fe7807c30a4e99bc91e82ade1a958cd45c50efaa5e788539 not found: ID does not exist"
Aug 05 18:20:39.192956 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:39.192933    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746"]
Aug 05 18:20:39.201166 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:39.201146    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2696ea-69d56fdb57-fq746"]
Aug 05 18:20:39.700346 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:20:39.700317    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d322eeed-d6f7-42a7-979e-7d9e55e62a69" path="/var/lib/kubelet/pods/d322eeed-d6f7-42a7-979e-7d9e55e62a69/volumes"
Aug 05 18:21:05.021820 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:05.021787    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg"]
Aug 05 18:21:05.022335 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:05.022032    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg" podUID="3589901b-34d7-4a50-9b4e-4dad090e3917" containerName="maas-api" containerID="cri-o://2e7ada5f70f06b151683f60f4b62d4160d9560f790208c92d9579f7d5838f6d4" gracePeriod=30
Aug 05 18:21:05.085983 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:05.085954    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2"]
Aug 05 18:21:05.086221 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:05.086195    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2" podUID="c048be46-95b0-4620-a80e-9a0ffdc56961" containerName="payload-pre-processing" containerID="cri-o://900e8058b36aa040ea073b46a04ceff05b2d147b8d94c61d5848b350c64277bf" gracePeriod=30
Aug 05 18:21:05.300746 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:05.300642    2550 generic.go:356] "Generic (PLEG): container finished" podID="c048be46-95b0-4620-a80e-9a0ffdc56961" containerID="900e8058b36aa040ea073b46a04ceff05b2d147b8d94c61d5848b350c64277bf" exitCode=0
Aug 05 18:21:05.300746 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:05.300724    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2" event={"ID":"c048be46-95b0-4620-a80e-9a0ffdc56961","Type":"ContainerDied","Data":"900e8058b36aa040ea073b46a04ceff05b2d147b8d94c61d5848b350c64277bf"}
Aug 05 18:21:05.302485 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:05.302455    2550 generic.go:356] "Generic (PLEG): container finished" podID="3589901b-34d7-4a50-9b4e-4dad090e3917" containerID="2e7ada5f70f06b151683f60f4b62d4160d9560f790208c92d9579f7d5838f6d4" exitCode=0
Aug 05 18:21:05.302613 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:05.302525    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg" event={"ID":"3589901b-34d7-4a50-9b4e-4dad090e3917","Type":"ContainerDied","Data":"2e7ada5f70f06b151683f60f4b62d4160d9560f790208c92d9579f7d5838f6d4"}
Aug 05 18:21:05.351161 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:05.351139    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg"
Aug 05 18:21:05.391117 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:05.391098    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2"
Aug 05 18:21:05.424419 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:05.424394    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3589901b-34d7-4a50-9b4e-4dad090e3917-openshift-service-ca\") pod \"3589901b-34d7-4a50-9b4e-4dad090e3917\" (UID: \"3589901b-34d7-4a50-9b4e-4dad090e3917\") "
Aug 05 18:21:05.424532 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:05.424494    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jj78k\" (UniqueName: \"kubernetes.io/projected/3589901b-34d7-4a50-9b4e-4dad090e3917-kube-api-access-jj78k\") pod \"3589901b-34d7-4a50-9b4e-4dad090e3917\" (UID: \"3589901b-34d7-4a50-9b4e-4dad090e3917\") "
Aug 05 18:21:05.424608 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:05.424590    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3589901b-34d7-4a50-9b4e-4dad090e3917-maas-api-tls\") pod \"3589901b-34d7-4a50-9b4e-4dad090e3917\" (UID: \"3589901b-34d7-4a50-9b4e-4dad090e3917\") "
Aug 05 18:21:05.424883 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:05.424853    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3589901b-34d7-4a50-9b4e-4dad090e3917-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3589901b-34d7-4a50-9b4e-4dad090e3917" (UID: "3589901b-34d7-4a50-9b4e-4dad090e3917"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:21:05.426492 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:05.426468    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3589901b-34d7-4a50-9b4e-4dad090e3917-kube-api-access-jj78k" (OuterVolumeSpecName: "kube-api-access-jj78k") pod "3589901b-34d7-4a50-9b4e-4dad090e3917" (UID: "3589901b-34d7-4a50-9b4e-4dad090e3917"). InnerVolumeSpecName "kube-api-access-jj78k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:21:05.426559 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:05.426500    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3589901b-34d7-4a50-9b4e-4dad090e3917-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3589901b-34d7-4a50-9b4e-4dad090e3917" (UID: "3589901b-34d7-4a50-9b4e-4dad090e3917"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:21:05.525914 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:05.525888    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c048be46-95b0-4620-a80e-9a0ffdc56961-plugins-config-volume\") pod \"c048be46-95b0-4620-a80e-9a0ffdc56961\" (UID: \"c048be46-95b0-4620-a80e-9a0ffdc56961\") "
Aug 05 18:21:05.526040 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:05.525951    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77467\" (UniqueName: \"kubernetes.io/projected/c048be46-95b0-4620-a80e-9a0ffdc56961-kube-api-access-77467\") pod \"c048be46-95b0-4620-a80e-9a0ffdc56961\" (UID: \"c048be46-95b0-4620-a80e-9a0ffdc56961\") "
Aug 05 18:21:05.526208 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:05.526187    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c048be46-95b0-4620-a80e-9a0ffdc56961-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c048be46-95b0-4620-a80e-9a0ffdc56961" (UID: "c048be46-95b0-4620-a80e-9a0ffdc56961"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:21:05.526260 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:05.526227    2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3589901b-34d7-4a50-9b4e-4dad090e3917-openshift-service-ca\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:21:05.526260 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:05.526240    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jj78k\" (UniqueName: \"kubernetes.io/projected/3589901b-34d7-4a50-9b4e-4dad090e3917-kube-api-access-jj78k\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:21:05.526260 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:05.526251    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3589901b-34d7-4a50-9b4e-4dad090e3917-maas-api-tls\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:21:05.527769 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:05.527748    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c048be46-95b0-4620-a80e-9a0ffdc56961-kube-api-access-77467" (OuterVolumeSpecName: "kube-api-access-77467") pod "c048be46-95b0-4620-a80e-9a0ffdc56961" (UID: "c048be46-95b0-4620-a80e-9a0ffdc56961"). InnerVolumeSpecName "kube-api-access-77467". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:21:05.627029 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:05.626974    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c048be46-95b0-4620-a80e-9a0ffdc56961-plugins-config-volume\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:21:05.627029 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:05.626992    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77467\" (UniqueName: \"kubernetes.io/projected/c048be46-95b0-4620-a80e-9a0ffdc56961-kube-api-access-77467\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:21:06.311891 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:06.311803    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2"
Aug 05 18:21:06.312357 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:06.311802    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2" event={"ID":"c048be46-95b0-4620-a80e-9a0ffdc56961","Type":"ContainerDied","Data":"e6a8b6a1aadc9f902995e9cfd1597a961cbedda3e57929c3d365c1cc9b86985d"}
Aug 05 18:21:06.312357 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:06.311945    2550 scope.go:117] "RemoveContainer" containerID="900e8058b36aa040ea073b46a04ceff05b2d147b8d94c61d5848b350c64277bf"
Aug 05 18:21:06.313372 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:06.313275    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg"
Aug 05 18:21:06.313372 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:06.313293    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg" event={"ID":"3589901b-34d7-4a50-9b4e-4dad090e3917","Type":"ContainerDied","Data":"a70a696a097f4564f5e6e0a221f50f975c256c242386c1afb7e3a029f33eb00d"}
Aug 05 18:21:06.321031 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:06.321013    2550 scope.go:117] "RemoveContainer" containerID="2e7ada5f70f06b151683f60f4b62d4160d9560f790208c92d9579f7d5838f6d4"
Aug 05 18:21:06.330408 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:06.330379    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg"]
Aug 05 18:21:06.337054 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:06.337032    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6427fe-5c5685ff85-7hxvg"]
Aug 05 18:21:06.351443 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:06.351420    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2"]
Aug 05 18:21:06.355778 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:06.355759    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6427fe-694957c58d-6mdq2"]
Aug 05 18:21:07.701818 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:07.701787    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3589901b-34d7-4a50-9b4e-4dad090e3917" path="/var/lib/kubelet/pods/3589901b-34d7-4a50-9b4e-4dad090e3917/volumes"
Aug 05 18:21:07.702184 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:07.702114    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c048be46-95b0-4620-a80e-9a0ffdc56961" path="/var/lib/kubelet/pods/c048be46-95b0-4620-a80e-9a0ffdc56961/volumes"
Aug 05 18:21:33.762936 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:33.762910    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 18:21:33.897450 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:21:33.897370    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-thnnz_1e69a0d5-f55f-4ff7-bd66-82fcc68f8674/ovn-acl-logging/0.log"
Aug 05 18:22:47.284048 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.284011    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t"]
Aug 05 18:22:47.284589 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.284252    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t" podUID="75b1ba90-be69-4811-9ee5-9b686b19ad2d" containerName="maas-api" containerID="cri-o://a0f884edc389bdd02f60b8267982fcfde1c1af56d7384476c00b6a78fb1de7a5" gracePeriod=30
Aug 05 18:22:47.468548 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.468515    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz"]
Aug 05 18:22:47.468928 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.468780    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz" podUID="f5e241b7-3f96-47e4-af63-e81a1c9e47d3" containerName="payload-processing" containerID="cri-o://1a22b4d88d360550cd590c6972dc6337d801f217c8963c78a96e6729a64c0bd5" gracePeriod=30
Aug 05 18:22:47.597439 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.597408    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t"
Aug 05 18:22:47.724671 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.724641    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz"
Aug 05 18:22:47.742287 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.742223    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49rph\" (UniqueName: \"kubernetes.io/projected/75b1ba90-be69-4811-9ee5-9b686b19ad2d-kube-api-access-49rph\") pod \"75b1ba90-be69-4811-9ee5-9b686b19ad2d\" (UID: \"75b1ba90-be69-4811-9ee5-9b686b19ad2d\") "
Aug 05 18:22:47.742287 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.742291    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75b1ba90-be69-4811-9ee5-9b686b19ad2d-maas-api-tls\") pod \"75b1ba90-be69-4811-9ee5-9b686b19ad2d\" (UID: \"75b1ba90-be69-4811-9ee5-9b686b19ad2d\") "
Aug 05 18:22:47.742548 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.742360    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75b1ba90-be69-4811-9ee5-9b686b19ad2d-openshift-service-ca\") pod \"75b1ba90-be69-4811-9ee5-9b686b19ad2d\" (UID: \"75b1ba90-be69-4811-9ee5-9b686b19ad2d\") "
Aug 05 18:22:47.742861 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.742817    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75b1ba90-be69-4811-9ee5-9b686b19ad2d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "75b1ba90-be69-4811-9ee5-9b686b19ad2d" (UID: "75b1ba90-be69-4811-9ee5-9b686b19ad2d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:22:47.745227 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.745197    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75b1ba90-be69-4811-9ee5-9b686b19ad2d-kube-api-access-49rph" (OuterVolumeSpecName: "kube-api-access-49rph") pod "75b1ba90-be69-4811-9ee5-9b686b19ad2d" (UID: "75b1ba90-be69-4811-9ee5-9b686b19ad2d"). InnerVolumeSpecName "kube-api-access-49rph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:22:47.745502 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.745477    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75b1ba90-be69-4811-9ee5-9b686b19ad2d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "75b1ba90-be69-4811-9ee5-9b686b19ad2d" (UID: "75b1ba90-be69-4811-9ee5-9b686b19ad2d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:22:47.810126 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.810014    2550 generic.go:356] "Generic (PLEG): container finished" podID="75b1ba90-be69-4811-9ee5-9b686b19ad2d" containerID="a0f884edc389bdd02f60b8267982fcfde1c1af56d7384476c00b6a78fb1de7a5" exitCode=0
Aug 05 18:22:47.810126 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.810069    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t"
Aug 05 18:22:47.810126 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.810102    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t" event={"ID":"75b1ba90-be69-4811-9ee5-9b686b19ad2d","Type":"ContainerDied","Data":"a0f884edc389bdd02f60b8267982fcfde1c1af56d7384476c00b6a78fb1de7a5"}
Aug 05 18:22:47.810408 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.810159    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t" event={"ID":"75b1ba90-be69-4811-9ee5-9b686b19ad2d","Type":"ContainerDied","Data":"ac19adf5d9b71b45c7a78dd5f0bf75ddd16f5aa01fd99a9d7f157885b2845374"}
Aug 05 18:22:47.810408 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.810191    2550 scope.go:117] "RemoveContainer" containerID="a0f884edc389bdd02f60b8267982fcfde1c1af56d7384476c00b6a78fb1de7a5"
Aug 05 18:22:47.812147 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.811906    2550 generic.go:356] "Generic (PLEG): container finished" podID="f5e241b7-3f96-47e4-af63-e81a1c9e47d3" containerID="1a22b4d88d360550cd590c6972dc6337d801f217c8963c78a96e6729a64c0bd5" exitCode=0
Aug 05 18:22:47.812147 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.811971    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz" event={"ID":"f5e241b7-3f96-47e4-af63-e81a1c9e47d3","Type":"ContainerDied","Data":"1a22b4d88d360550cd590c6972dc6337d801f217c8963c78a96e6729a64c0bd5"}
Aug 05 18:22:47.812147 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.811984    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz"
Aug 05 18:22:47.812147 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.812015    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz" event={"ID":"f5e241b7-3f96-47e4-af63-e81a1c9e47d3","Type":"ContainerDied","Data":"6ba8bbc9f3c6a17655b52255b01349314174d9fbfa4300e893ebec90e1bdd7b1"}
Aug 05 18:22:47.820656 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.820638    2550 scope.go:117] "RemoveContainer" containerID="a0f884edc389bdd02f60b8267982fcfde1c1af56d7384476c00b6a78fb1de7a5"
Aug 05 18:22:47.820962 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:22:47.820945    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0f884edc389bdd02f60b8267982fcfde1c1af56d7384476c00b6a78fb1de7a5\": container with ID starting with a0f884edc389bdd02f60b8267982fcfde1c1af56d7384476c00b6a78fb1de7a5 not found: ID does not exist" containerID="a0f884edc389bdd02f60b8267982fcfde1c1af56d7384476c00b6a78fb1de7a5"
Aug 05 18:22:47.821015 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.820972    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0f884edc389bdd02f60b8267982fcfde1c1af56d7384476c00b6a78fb1de7a5"} err="failed to get container status \"a0f884edc389bdd02f60b8267982fcfde1c1af56d7384476c00b6a78fb1de7a5\": rpc error: code = NotFound desc = could not find container \"a0f884edc389bdd02f60b8267982fcfde1c1af56d7384476c00b6a78fb1de7a5\": container with ID starting with a0f884edc389bdd02f60b8267982fcfde1c1af56d7384476c00b6a78fb1de7a5 not found: ID does not exist"
Aug 05 18:22:47.821015 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.820992    2550 scope.go:117] "RemoveContainer" containerID="1a22b4d88d360550cd590c6972dc6337d801f217c8963c78a96e6729a64c0bd5"
Aug 05 18:22:47.828945 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.828928    2550 scope.go:117] "RemoveContainer" containerID="1a22b4d88d360550cd590c6972dc6337d801f217c8963c78a96e6729a64c0bd5"
Aug 05 18:22:47.829193 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:22:47.829173    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a22b4d88d360550cd590c6972dc6337d801f217c8963c78a96e6729a64c0bd5\": container with ID starting with 1a22b4d88d360550cd590c6972dc6337d801f217c8963c78a96e6729a64c0bd5 not found: ID does not exist" containerID="1a22b4d88d360550cd590c6972dc6337d801f217c8963c78a96e6729a64c0bd5"
Aug 05 18:22:47.829239 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.829200    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a22b4d88d360550cd590c6972dc6337d801f217c8963c78a96e6729a64c0bd5"} err="failed to get container status \"1a22b4d88d360550cd590c6972dc6337d801f217c8963c78a96e6729a64c0bd5\": rpc error: code = NotFound desc = could not find container \"1a22b4d88d360550cd590c6972dc6337d801f217c8963c78a96e6729a64c0bd5\": container with ID starting with 1a22b4d88d360550cd590c6972dc6337d801f217c8963c78a96e6729a64c0bd5 not found: ID does not exist"
Aug 05 18:22:47.833249 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.833230    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t"]
Aug 05 18:22:47.836883 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.836860    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ba69fc-5f589ff5c8-26g8t"]
Aug 05 18:22:47.843022 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.843003    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k42b7\" (UniqueName: \"kubernetes.io/projected/f5e241b7-3f96-47e4-af63-e81a1c9e47d3-kube-api-access-k42b7\") pod \"f5e241b7-3f96-47e4-af63-e81a1c9e47d3\" (UID: \"f5e241b7-3f96-47e4-af63-e81a1c9e47d3\") "
Aug 05 18:22:47.843140 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.843124    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5e241b7-3f96-47e4-af63-e81a1c9e47d3-plugins-config-volume\") pod \"f5e241b7-3f96-47e4-af63-e81a1c9e47d3\" (UID: \"f5e241b7-3f96-47e4-af63-e81a1c9e47d3\") "
Aug 05 18:22:47.843354 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.843340    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75b1ba90-be69-4811-9ee5-9b686b19ad2d-maas-api-tls\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:22:47.843415 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.843362    2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75b1ba90-be69-4811-9ee5-9b686b19ad2d-openshift-service-ca\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:22:47.843415 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.843377    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49rph\" (UniqueName: \"kubernetes.io/projected/75b1ba90-be69-4811-9ee5-9b686b19ad2d-kube-api-access-49rph\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:22:47.843484 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.843429    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5e241b7-3f96-47e4-af63-e81a1c9e47d3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f5e241b7-3f96-47e4-af63-e81a1c9e47d3" (UID: "f5e241b7-3f96-47e4-af63-e81a1c9e47d3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:22:47.844976 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.844956    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5e241b7-3f96-47e4-af63-e81a1c9e47d3-kube-api-access-k42b7" (OuterVolumeSpecName: "kube-api-access-k42b7") pod "f5e241b7-3f96-47e4-af63-e81a1c9e47d3" (UID: "f5e241b7-3f96-47e4-af63-e81a1c9e47d3"). InnerVolumeSpecName "kube-api-access-k42b7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:22:47.943983 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.943950    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5e241b7-3f96-47e4-af63-e81a1c9e47d3-plugins-config-volume\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:22:47.943983 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:47.943977    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k42b7\" (UniqueName: \"kubernetes.io/projected/f5e241b7-3f96-47e4-af63-e81a1c9e47d3-kube-api-access-k42b7\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:22:48.134990 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:48.134953    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz"]
Aug 05 18:22:48.139615 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:48.139586    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-ba69fc-65956d55db-4hptz"]
Aug 05 18:22:49.700402 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:49.700369    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75b1ba90-be69-4811-9ee5-9b686b19ad2d" path="/var/lib/kubelet/pods/75b1ba90-be69-4811-9ee5-9b686b19ad2d/volumes"
Aug 05 18:22:49.700808 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:49.700749    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5e241b7-3f96-47e4-af63-e81a1c9e47d3" path="/var/lib/kubelet/pods/f5e241b7-3f96-47e4-af63-e81a1c9e47d3/volumes"
Aug 05 18:22:54.585665 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:54.585629    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"]
Aug 05 18:22:54.586181 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:54.585892    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp" podUID="4de64bda-e7bf-422f-991f-238208f1cab8" containerName="istio-proxy" containerID="cri-o://8135aa59f60fcd4b7d335ee98f2f48d4004d71807c8a3ef156013fa6feea9944" gracePeriod=30
Aug 05 18:22:59.840496 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.840468    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:22:59.852591 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.852529    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-workload-certs\") pod \"4de64bda-e7bf-422f-991f-238208f1cab8\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") "
Aug 05 18:22:59.852591 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.852571    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-workload-socket\") pod \"4de64bda-e7bf-422f-991f-238208f1cab8\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") "
Aug 05 18:22:59.852780 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.852601    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4de64bda-e7bf-422f-991f-238208f1cab8-istio-token\") pod \"4de64bda-e7bf-422f-991f-238208f1cab8\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") "
Aug 05 18:22:59.852780 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.852618    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-istio-data\") pod \"4de64bda-e7bf-422f-991f-238208f1cab8\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") "
Aug 05 18:22:59.852780 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.852652    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4de64bda-e7bf-422f-991f-238208f1cab8-istio-podinfo\") pod \"4de64bda-e7bf-422f-991f-238208f1cab8\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") "
Aug 05 18:22:59.852780 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.852676    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4de64bda-e7bf-422f-991f-238208f1cab8-istiod-ca-cert\") pod \"4de64bda-e7bf-422f-991f-238208f1cab8\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") "
Aug 05 18:22:59.852780 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.852741    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9dct7\" (UniqueName: \"kubernetes.io/projected/4de64bda-e7bf-422f-991f-238208f1cab8-kube-api-access-9dct7\") pod \"4de64bda-e7bf-422f-991f-238208f1cab8\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") "
Aug 05 18:22:59.853029 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.852787    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-credential-socket\") pod \"4de64bda-e7bf-422f-991f-238208f1cab8\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") "
Aug 05 18:22:59.853029 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.852809    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-istio-envoy\") pod \"4de64bda-e7bf-422f-991f-238208f1cab8\" (UID: \"4de64bda-e7bf-422f-991f-238208f1cab8\") "
Aug 05 18:22:59.853029 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.852828    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4de64bda-e7bf-422f-991f-238208f1cab8" (UID: "4de64bda-e7bf-422f-991f-238208f1cab8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:22:59.853029 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.852910    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4de64bda-e7bf-422f-991f-238208f1cab8" (UID: "4de64bda-e7bf-422f-991f-238208f1cab8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:22:59.853029 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.853000    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-istio-data" (OuterVolumeSpecName: "istio-data") pod "4de64bda-e7bf-422f-991f-238208f1cab8" (UID: "4de64bda-e7bf-422f-991f-238208f1cab8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:22:59.853314 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.853132    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4de64bda-e7bf-422f-991f-238208f1cab8" (UID: "4de64bda-e7bf-422f-991f-238208f1cab8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:22:59.853374 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.853336    2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-istio-data\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:22:59.853374 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.853351    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4de64bda-e7bf-422f-991f-238208f1cab8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4de64bda-e7bf-422f-991f-238208f1cab8" (UID: "4de64bda-e7bf-422f-991f-238208f1cab8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:22:59.853570 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.853510    2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-workload-certs\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:22:59.853570 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.853542    2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-workload-socket\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:22:59.855072 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.855049    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4de64bda-e7bf-422f-991f-238208f1cab8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4de64bda-e7bf-422f-991f-238208f1cab8" (UID: "4de64bda-e7bf-422f-991f-238208f1cab8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 05 18:22:59.855288 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.855261    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4de64bda-e7bf-422f-991f-238208f1cab8-kube-api-access-9dct7" (OuterVolumeSpecName: "kube-api-access-9dct7") pod "4de64bda-e7bf-422f-991f-238208f1cab8" (UID: "4de64bda-e7bf-422f-991f-238208f1cab8"). InnerVolumeSpecName "kube-api-access-9dct7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:22:59.855288 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.855275    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4de64bda-e7bf-422f-991f-238208f1cab8-istio-token" (OuterVolumeSpecName: "istio-token") pod "4de64bda-e7bf-422f-991f-238208f1cab8" (UID: "4de64bda-e7bf-422f-991f-238208f1cab8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:22:59.855410 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.855308    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4de64bda-e7bf-422f-991f-238208f1cab8" (UID: "4de64bda-e7bf-422f-991f-238208f1cab8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:22:59.875380 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.875353    2550 generic.go:356] "Generic (PLEG): container finished" podID="4de64bda-e7bf-422f-991f-238208f1cab8" containerID="8135aa59f60fcd4b7d335ee98f2f48d4004d71807c8a3ef156013fa6feea9944" exitCode=0
Aug 05 18:22:59.875534 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.875416    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"
Aug 05 18:22:59.875534 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.875440    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp" event={"ID":"4de64bda-e7bf-422f-991f-238208f1cab8","Type":"ContainerDied","Data":"8135aa59f60fcd4b7d335ee98f2f48d4004d71807c8a3ef156013fa6feea9944"}
Aug 05 18:22:59.875534 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.875499    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp" event={"ID":"4de64bda-e7bf-422f-991f-238208f1cab8","Type":"ContainerDied","Data":"cb356721c8532c1d4a161eb9487ff94be7cbbf704cec74683310976e363b3cb3"}
Aug 05 18:22:59.875534 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.875525    2550 scope.go:117] "RemoveContainer" containerID="8135aa59f60fcd4b7d335ee98f2f48d4004d71807c8a3ef156013fa6feea9944"
Aug 05 18:22:59.883909 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.883889    2550 scope.go:117] "RemoveContainer" containerID="31935cdd534616fa26b3a9e158fbe7f43e18c3b429ae2a6a80bc750ca95b7bfb"
Aug 05 18:22:59.891762 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.891751    2550 scope.go:117] "RemoveContainer" containerID="8135aa59f60fcd4b7d335ee98f2f48d4004d71807c8a3ef156013fa6feea9944"
Aug 05 18:22:59.892072 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:22:59.892052    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8135aa59f60fcd4b7d335ee98f2f48d4004d71807c8a3ef156013fa6feea9944\": container with ID starting with 8135aa59f60fcd4b7d335ee98f2f48d4004d71807c8a3ef156013fa6feea9944 not found: ID does not exist" containerID="8135aa59f60fcd4b7d335ee98f2f48d4004d71807c8a3ef156013fa6feea9944"
Aug 05 18:22:59.892169 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.892080    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8135aa59f60fcd4b7d335ee98f2f48d4004d71807c8a3ef156013fa6feea9944"} err="failed to get container status \"8135aa59f60fcd4b7d335ee98f2f48d4004d71807c8a3ef156013fa6feea9944\": rpc error: code = NotFound desc = could not find container \"8135aa59f60fcd4b7d335ee98f2f48d4004d71807c8a3ef156013fa6feea9944\": container with ID starting with 8135aa59f60fcd4b7d335ee98f2f48d4004d71807c8a3ef156013fa6feea9944 not found: ID does not exist"
Aug 05 18:22:59.892169 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.892098    2550 scope.go:117] "RemoveContainer" containerID="31935cdd534616fa26b3a9e158fbe7f43e18c3b429ae2a6a80bc750ca95b7bfb"
Aug 05 18:22:59.892348 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:22:59.892328    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31935cdd534616fa26b3a9e158fbe7f43e18c3b429ae2a6a80bc750ca95b7bfb\": container with ID starting with 31935cdd534616fa26b3a9e158fbe7f43e18c3b429ae2a6a80bc750ca95b7bfb not found: ID does not exist" containerID="31935cdd534616fa26b3a9e158fbe7f43e18c3b429ae2a6a80bc750ca95b7bfb"
Aug 05 18:22:59.892414 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.892353    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31935cdd534616fa26b3a9e158fbe7f43e18c3b429ae2a6a80bc750ca95b7bfb"} err="failed to get container status \"31935cdd534616fa26b3a9e158fbe7f43e18c3b429ae2a6a80bc750ca95b7bfb\": rpc error: code = NotFound desc = could not find container \"31935cdd534616fa26b3a9e158fbe7f43e18c3b429ae2a6a80bc750ca95b7bfb\": container with ID starting with 31935cdd534616fa26b3a9e158fbe7f43e18c3b429ae2a6a80bc750ca95b7bfb not found: ID does not exist"
Aug 05 18:22:59.900933 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.900905    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"]
Aug 05 18:22:59.904475 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.904455    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-ba69fc-openshift-default-848b94c58d-v8kxp"]
Aug 05 18:22:59.954454 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.954422    2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4de64bda-e7bf-422f-991f-238208f1cab8-istio-podinfo\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:22:59.954454 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.954454    2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4de64bda-e7bf-422f-991f-238208f1cab8-istiod-ca-cert\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:22:59.954633 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.954468    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dct7\" (UniqueName: \"kubernetes.io/projected/4de64bda-e7bf-422f-991f-238208f1cab8-kube-api-access-9dct7\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:22:59.954633 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.954482    2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-credential-socket\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:22:59.954633 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.954495    2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4de64bda-e7bf-422f-991f-238208f1cab8-istio-envoy\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:22:59.954633 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:22:59.954509    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4de64bda-e7bf-422f-991f-238208f1cab8-istio-token\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:23:01.700865 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:01.700831    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4de64bda-e7bf-422f-991f-238208f1cab8" path="/var/lib/kubelet/pods/4de64bda-e7bf-422f-991f-238208f1cab8/volumes"
Aug 05 18:23:05.431407 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.431373    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz"]
Aug 05 18:23:05.431944 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.431575    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz" podUID="c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a" containerName="maas-api" containerID="cri-o://bccd6aa65205cc0c4d80474eb708c0a992506b569104a91a026f3eeb564907c9" gracePeriod=30
Aug 05 18:23:05.486315 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.486280    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb"]
Aug 05 18:23:05.486587 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.486557    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb" podUID="273d04d4-27b2-4c47-97bc-afb962cf168d" containerName="payload-processing" containerID="cri-o://01b553b48cb27db0c78cdc80d151b40fcaecce07c913ecb863b63a3f97bb7b92" gracePeriod=30
Aug 05 18:23:05.735693 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.735669    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz"
Aug 05 18:23:05.768352 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.768329    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb"
Aug 05 18:23:05.803365 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.803333    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vtlf\" (UniqueName: \"kubernetes.io/projected/c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a-kube-api-access-8vtlf\") pod \"c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a\" (UID: \"c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a\") "
Aug 05 18:23:05.803553 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.803391    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a-maas-api-tls\") pod \"c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a\" (UID: \"c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a\") "
Aug 05 18:23:05.803553 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.803424    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/273d04d4-27b2-4c47-97bc-afb962cf168d-plugins-config-volume\") pod \"273d04d4-27b2-4c47-97bc-afb962cf168d\" (UID: \"273d04d4-27b2-4c47-97bc-afb962cf168d\") "
Aug 05 18:23:05.803553 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.803454    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a-openshift-service-ca\") pod \"c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a\" (UID: \"c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a\") "
Aug 05 18:23:05.803775 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.803555    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ns4x\" (UniqueName: \"kubernetes.io/projected/273d04d4-27b2-4c47-97bc-afb962cf168d-kube-api-access-8ns4x\") pod \"273d04d4-27b2-4c47-97bc-afb962cf168d\" (UID: \"273d04d4-27b2-4c47-97bc-afb962cf168d\") "
Aug 05 18:23:05.804599 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.804558    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/273d04d4-27b2-4c47-97bc-afb962cf168d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "273d04d4-27b2-4c47-97bc-afb962cf168d" (UID: "273d04d4-27b2-4c47-97bc-afb962cf168d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:23:05.804956 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.804843    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a" (UID: "c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:23:05.806304 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.806267    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/273d04d4-27b2-4c47-97bc-afb962cf168d-kube-api-access-8ns4x" (OuterVolumeSpecName: "kube-api-access-8ns4x") pod "273d04d4-27b2-4c47-97bc-afb962cf168d" (UID: "273d04d4-27b2-4c47-97bc-afb962cf168d"). InnerVolumeSpecName "kube-api-access-8ns4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:23:05.806611 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.806592    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a" (UID: "c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:23:05.806787 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.806759    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a-kube-api-access-8vtlf" (OuterVolumeSpecName: "kube-api-access-8vtlf") pod "c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a" (UID: "c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a"). InnerVolumeSpecName "kube-api-access-8vtlf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:23:05.904144 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.904111    2550 generic.go:356] "Generic (PLEG): container finished" podID="c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a" containerID="bccd6aa65205cc0c4d80474eb708c0a992506b569104a91a026f3eeb564907c9" exitCode=0
Aug 05 18:23:05.904144 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.904139    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vtlf\" (UniqueName: \"kubernetes.io/projected/c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a-kube-api-access-8vtlf\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:23:05.904392 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.904159    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a-maas-api-tls\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:23:05.904392 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.904164    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz"
Aug 05 18:23:05.904392 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.904208    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz" event={"ID":"c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a","Type":"ContainerDied","Data":"bccd6aa65205cc0c4d80474eb708c0a992506b569104a91a026f3eeb564907c9"}
Aug 05 18:23:05.904392 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.904262    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz" event={"ID":"c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a","Type":"ContainerDied","Data":"80d20da7a81b31b2212d5ac26d83be7e9bc610cbb7abfa2e9bf7434bd519c318"}
Aug 05 18:23:05.904392 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.904168    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/273d04d4-27b2-4c47-97bc-afb962cf168d-plugins-config-volume\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:23:05.904392 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.904293    2550 scope.go:117] "RemoveContainer" containerID="bccd6aa65205cc0c4d80474eb708c0a992506b569104a91a026f3eeb564907c9"
Aug 05 18:23:05.904392 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.904321    2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a-openshift-service-ca\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:23:05.904747 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.904520    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ns4x\" (UniqueName: \"kubernetes.io/projected/273d04d4-27b2-4c47-97bc-afb962cf168d-kube-api-access-8ns4x\") on node \"ip-10-0-134-46.ec2.internal\" DevicePath \"\""
Aug 05 18:23:05.905539 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.905520    2550 generic.go:356] "Generic (PLEG): container finished" podID="273d04d4-27b2-4c47-97bc-afb962cf168d" containerID="01b553b48cb27db0c78cdc80d151b40fcaecce07c913ecb863b63a3f97bb7b92" exitCode=0
Aug 05 18:23:05.905641 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.905551    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb" event={"ID":"273d04d4-27b2-4c47-97bc-afb962cf168d","Type":"ContainerDied","Data":"01b553b48cb27db0c78cdc80d151b40fcaecce07c913ecb863b63a3f97bb7b92"}
Aug 05 18:23:05.905641 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.905565    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb"
Aug 05 18:23:05.905641 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.905585    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb" event={"ID":"273d04d4-27b2-4c47-97bc-afb962cf168d","Type":"ContainerDied","Data":"7c34aa28cad0278c5884465b59db2ab6c0dcbc7944bf0dbb34dae51e0167233d"}
Aug 05 18:23:05.914112 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.914091    2550 scope.go:117] "RemoveContainer" containerID="bccd6aa65205cc0c4d80474eb708c0a992506b569104a91a026f3eeb564907c9"
Aug 05 18:23:05.914423 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:23:05.914402    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bccd6aa65205cc0c4d80474eb708c0a992506b569104a91a026f3eeb564907c9\": container with ID starting with bccd6aa65205cc0c4d80474eb708c0a992506b569104a91a026f3eeb564907c9 not found: ID does not exist" containerID="bccd6aa65205cc0c4d80474eb708c0a992506b569104a91a026f3eeb564907c9"
Aug 05 18:23:05.914522 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.914430    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bccd6aa65205cc0c4d80474eb708c0a992506b569104a91a026f3eeb564907c9"} err="failed to get container status \"bccd6aa65205cc0c4d80474eb708c0a992506b569104a91a026f3eeb564907c9\": rpc error: code = NotFound desc = could not find container \"bccd6aa65205cc0c4d80474eb708c0a992506b569104a91a026f3eeb564907c9\": container with ID starting with bccd6aa65205cc0c4d80474eb708c0a992506b569104a91a026f3eeb564907c9 not found: ID does not exist"
Aug 05 18:23:05.914522 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.914446    2550 scope.go:117] "RemoveContainer" containerID="01b553b48cb27db0c78cdc80d151b40fcaecce07c913ecb863b63a3f97bb7b92"
Aug 05 18:23:05.922533 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.922512    2550 scope.go:117] "RemoveContainer" containerID="01b553b48cb27db0c78cdc80d151b40fcaecce07c913ecb863b63a3f97bb7b92"
Aug 05 18:23:05.922865 ip-10-0-134-46 kubenswrapper[2550]: E0805 18:23:05.922846    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01b553b48cb27db0c78cdc80d151b40fcaecce07c913ecb863b63a3f97bb7b92\": container with ID starting with 01b553b48cb27db0c78cdc80d151b40fcaecce07c913ecb863b63a3f97bb7b92 not found: ID does not exist" containerID="01b553b48cb27db0c78cdc80d151b40fcaecce07c913ecb863b63a3f97bb7b92"
Aug 05 18:23:05.922947 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.922874    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01b553b48cb27db0c78cdc80d151b40fcaecce07c913ecb863b63a3f97bb7b92"} err="failed to get container status \"01b553b48cb27db0c78cdc80d151b40fcaecce07c913ecb863b63a3f97bb7b92\": rpc error: code = NotFound desc = could not find container \"01b553b48cb27db0c78cdc80d151b40fcaecce07c913ecb863b63a3f97bb7b92\": container with ID starting with 01b553b48cb27db0c78cdc80d151b40fcaecce07c913ecb863b63a3f97bb7b92 not found: ID does not exist"
Aug 05 18:23:05.930005 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.929979    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb"]
Aug 05 18:23:05.933536 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.933513    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-594b65-cdd8786c8-wkgpb"]
Aug 05 18:23:05.944565 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.944430    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz"]
Aug 05 18:23:05.946092 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:05.946072    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-594b65-7f7f4dc859-g7cmz"]
Aug 05 18:23:07.700085 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:07.700054    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="273d04d4-27b2-4c47-97bc-afb962cf168d" path="/var/lib/kubelet/pods/273d04d4-27b2-4c47-97bc-afb962cf168d/volumes"
Aug 05 18:23:07.700461 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:07.700419    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a" path="/var/lib/kubelet/pods/c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a/volumes"
Aug 05 18:23:13.390306 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:13.390269    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-75db7cc4bd-5qrzc_92543e2a-899a-40b4-8c2f-5a25127e378f/authorino/0.log"
Aug 05 18:23:18.592942 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:18.592911    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-fbrn7_9409ab12-e7ca-44e9-84a5-4bc4d3f46916/manager/0.log"
Aug 05 18:23:18.727779 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:18.727746    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-2ldj9_56c44a81-44f1-451a-a9ed-add009f98ca7/manager/0.log"
Aug 05 18:23:18.849881 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:18.849805    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6bb68d8457-cqsfl_cf4d0eeb-01e0-4740-b068-1b1a838dec25/manager/0.log"
Aug 05 18:23:18.963606 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:18.963581    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-kxjqm_e24fda29-710a-4354-9a13-a35317191c15/server/0.log"
Aug 05 18:23:19.073063 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:19.073037    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-bfsdk_db99dc27-ca54-4ccb-ac40-eec6ddfbb1d5/manager/0.log"
Aug 05 18:23:19.312611 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:19.312588    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cc69f54b9-qlhch_4fd60b10-0d46-4be8-b0d6-73ad8455b81d/manager/0.log"
Aug 05 18:23:19.318407 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:19.318391    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cc69f54b9-qlhch_4fd60b10-0d46-4be8-b0d6-73ad8455b81d/copy-manifests/0.log"
Aug 05 18:23:19.797823 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:19.797795    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6475fbf58c-ptfhq_5bef8aee-313c-409f-879e-e453179a9f76/maas-api/0.log"
Aug 05 18:23:20.239027 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:20.238997    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-tdr8k_93e686c5-3d2a-4ffa-a3f1-0d143bdf724f/postgres/0.log"
Aug 05 18:23:20.987626 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:20.987602    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-75db7cc4bd-5qrzc_92543e2a-899a-40b4-8c2f-5a25127e378f/authorino/0.log"
Aug 05 18:23:21.103248 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:21.103225    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-qh78n_24913a9d-22fc-451b-b7e3-25cee6d65ce1/manager/0.log"
Aug 05 18:23:21.652859 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:21.652829    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-8qt45_e50c044b-cb40-4340-870e-2fac665bbdf7/manager/0.log"
Aug 05 18:23:22.010421 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:22.010389    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c_3f44bcc3-3f40-403f-bcc8-791f5472340b/istio-proxy/0.log"
Aug 05 18:23:22.337625 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:22.337550    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-746fd96bc4-btn27_8b6db924-1edc-415d-91c3-292040e1f654/kube-auth-proxy/0.log"
Aug 05 18:23:22.448153 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:22.448125    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6czx5_1b600ba3-1ae9-4de8-bffd-9bac75f659ad/istio-proxy/1.log"
Aug 05 18:23:22.687183 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:22.687154    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-crrck_b762d708-104b-4bbb-ba46-13a3aff27b83/payload-processing/0.log"
Aug 05 18:23:23.346667 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:23.346639    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-ks5mq_84b646e7-28f3-4a91-af8d-74a4305d8e92/main/0.log"
Aug 05 18:23:23.454157 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:23.454127    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvwkds_e287ca84-d2ca-4783-acc0-73229a00ff03/main/0.log"
Aug 05 18:23:29.349887 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.349851    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8dz5d/must-gather-v79fr"]
Aug 05 18:23:29.350295 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.350196    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d322eeed-d6f7-42a7-979e-7d9e55e62a69" containerName="maas-api"
Aug 05 18:23:29.350295 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.350207    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="d322eeed-d6f7-42a7-979e-7d9e55e62a69" containerName="maas-api"
Aug 05 18:23:29.350295 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.350221    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c048be46-95b0-4620-a80e-9a0ffdc56961" containerName="payload-pre-processing"
Aug 05 18:23:29.350295 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.350226    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="c048be46-95b0-4620-a80e-9a0ffdc56961" containerName="payload-pre-processing"
Aug 05 18:23:29.350295 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.350233    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5e241b7-3f96-47e4-af63-e81a1c9e47d3" containerName="payload-processing"
Aug 05 18:23:29.350295 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.350239    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5e241b7-3f96-47e4-af63-e81a1c9e47d3" containerName="payload-processing"
Aug 05 18:23:29.350295 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.350246    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4de64bda-e7bf-422f-991f-238208f1cab8" containerName="istio-proxy"
Aug 05 18:23:29.350295 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.350251    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="4de64bda-e7bf-422f-991f-238208f1cab8" containerName="istio-proxy"
Aug 05 18:23:29.350295 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.350257    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="273d04d4-27b2-4c47-97bc-afb962cf168d" containerName="payload-processing"
Aug 05 18:23:29.350295 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.350262    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="273d04d4-27b2-4c47-97bc-afb962cf168d" containerName="payload-processing"
Aug 05 18:23:29.350295 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.350269    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a" containerName="maas-api"
Aug 05 18:23:29.350295 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.350273    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a" containerName="maas-api"
Aug 05 18:23:29.350295 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.350281    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75b1ba90-be69-4811-9ee5-9b686b19ad2d" containerName="maas-api"
Aug 05 18:23:29.350295 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.350285    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="75b1ba90-be69-4811-9ee5-9b686b19ad2d" containerName="maas-api"
Aug 05 18:23:29.350295 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.350292    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4de64bda-e7bf-422f-991f-238208f1cab8" containerName="istio-proxy"
Aug 05 18:23:29.350295 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.350297    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="4de64bda-e7bf-422f-991f-238208f1cab8" containerName="istio-proxy"
Aug 05 18:23:29.350906 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.350306    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3589901b-34d7-4a50-9b4e-4dad090e3917" containerName="maas-api"
Aug 05 18:23:29.350906 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.350312    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="3589901b-34d7-4a50-9b4e-4dad090e3917" containerName="maas-api"
Aug 05 18:23:29.350906 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.350363    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="75b1ba90-be69-4811-9ee5-9b686b19ad2d" containerName="maas-api"
Aug 05 18:23:29.350906 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.350371    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="273d04d4-27b2-4c47-97bc-afb962cf168d" containerName="payload-processing"
Aug 05 18:23:29.350906 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.350379    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5e241b7-3f96-47e4-af63-e81a1c9e47d3" containerName="payload-processing"
Aug 05 18:23:29.350906 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.350385    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5d5cec4-cff6-42e7-87fc-30e91b0c8c4a" containerName="maas-api"
Aug 05 18:23:29.350906 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.350393    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="4de64bda-e7bf-422f-991f-238208f1cab8" containerName="istio-proxy"
Aug 05 18:23:29.350906 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.350398    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="c048be46-95b0-4620-a80e-9a0ffdc56961" containerName="payload-pre-processing"
Aug 05 18:23:29.350906 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.350405    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="4de64bda-e7bf-422f-991f-238208f1cab8" containerName="istio-proxy"
Aug 05 18:23:29.350906 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.350411    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="3589901b-34d7-4a50-9b4e-4dad090e3917" containerName="maas-api"
Aug 05 18:23:29.350906 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.350416    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="d322eeed-d6f7-42a7-979e-7d9e55e62a69" containerName="maas-api"
Aug 05 18:23:29.353532 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.353514    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8dz5d/must-gather-v79fr"
Aug 05 18:23:29.356218 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.356197    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8dz5d\"/\"kube-root-ca.crt\""
Aug 05 18:23:29.357063 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.357043    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8dz5d\"/\"openshift-service-ca.crt\""
Aug 05 18:23:29.357150 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.357045    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8dz5d\"/\"default-dockercfg-9zj9h\""
Aug 05 18:23:29.361613 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.361592    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8dz5d/must-gather-v79fr"]
Aug 05 18:23:29.497249 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.497212    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c0daed8a-7e59-42d9-a76a-d9f41246d14b-must-gather-output\") pod \"must-gather-v79fr\" (UID: \"c0daed8a-7e59-42d9-a76a-d9f41246d14b\") " pod="openshift-must-gather-8dz5d/must-gather-v79fr"
Aug 05 18:23:29.497437 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.497262    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55m9b\" (UniqueName: \"kubernetes.io/projected/c0daed8a-7e59-42d9-a76a-d9f41246d14b-kube-api-access-55m9b\") pod \"must-gather-v79fr\" (UID: \"c0daed8a-7e59-42d9-a76a-d9f41246d14b\") " pod="openshift-must-gather-8dz5d/must-gather-v79fr"
Aug 05 18:23:29.598410 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.598375    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55m9b\" (UniqueName: \"kubernetes.io/projected/c0daed8a-7e59-42d9-a76a-d9f41246d14b-kube-api-access-55m9b\") pod \"must-gather-v79fr\" (UID: \"c0daed8a-7e59-42d9-a76a-d9f41246d14b\") " pod="openshift-must-gather-8dz5d/must-gather-v79fr"
Aug 05 18:23:29.598588 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.598470    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c0daed8a-7e59-42d9-a76a-d9f41246d14b-must-gather-output\") pod \"must-gather-v79fr\" (UID: \"c0daed8a-7e59-42d9-a76a-d9f41246d14b\") " pod="openshift-must-gather-8dz5d/must-gather-v79fr"
Aug 05 18:23:29.598836 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.598816    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c0daed8a-7e59-42d9-a76a-d9f41246d14b-must-gather-output\") pod \"must-gather-v79fr\" (UID: \"c0daed8a-7e59-42d9-a76a-d9f41246d14b\") " pod="openshift-must-gather-8dz5d/must-gather-v79fr"
Aug 05 18:23:29.606765 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.606672    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55m9b\" (UniqueName: \"kubernetes.io/projected/c0daed8a-7e59-42d9-a76a-d9f41246d14b-kube-api-access-55m9b\") pod \"must-gather-v79fr\" (UID: \"c0daed8a-7e59-42d9-a76a-d9f41246d14b\") " pod="openshift-must-gather-8dz5d/must-gather-v79fr"
Aug 05 18:23:29.664052 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.664030    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8dz5d/must-gather-v79fr"
Aug 05 18:23:29.794395 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.794372    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8dz5d/must-gather-v79fr"]
Aug 05 18:23:29.796885 ip-10-0-134-46 kubenswrapper[2550]: W0805 18:23:29.796851    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0daed8a_7e59_42d9_a76a_d9f41246d14b.slice/crio-cac09662e7bc0a1bef2a879ef44827bc6f2f291143c4803608ba63c7ddfe6412 WatchSource:0}: Error finding container cac09662e7bc0a1bef2a879ef44827bc6f2f291143c4803608ba63c7ddfe6412: Status 404 returned error can't find the container with id cac09662e7bc0a1bef2a879ef44827bc6f2f291143c4803608ba63c7ddfe6412
Aug 05 18:23:29.798858 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:29.798841    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 18:23:30.020850 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:30.020818    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8dz5d/must-gather-v79fr" event={"ID":"c0daed8a-7e59-42d9-a76a-d9f41246d14b","Type":"ContainerStarted","Data":"cac09662e7bc0a1bef2a879ef44827bc6f2f291143c4803608ba63c7ddfe6412"}
Aug 05 18:23:31.027620 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:31.027528    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8dz5d/must-gather-v79fr" event={"ID":"c0daed8a-7e59-42d9-a76a-d9f41246d14b","Type":"ContainerStarted","Data":"347b3d6acdbc2dde2779b4f641e46498e0f57e90fcacefb1ffb79afa8ce81c95"}
Aug 05 18:23:31.027620 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:31.027581    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8dz5d/must-gather-v79fr" event={"ID":"c0daed8a-7e59-42d9-a76a-d9f41246d14b","Type":"ContainerStarted","Data":"c54e952ede77ea7afba3c3e39c6eaa80c657b5d4fc2cb37b991ca21c1a2a58c8"}
Aug 05 18:23:31.045321 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:31.045263    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8dz5d/must-gather-v79fr" podStartSLOduration=1.186666701 podStartE2EDuration="2.045242615s" totalImagesPullingTime="858.575914ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:23:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:23:31.044666626 +0000 UTC m=+3718.026244056" watchObservedRunningTime="2026-08-05 18:23:31.045242615 +0000 UTC m=+3718.026820045"
Aug 05 18:23:32.412037 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:32.412006    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-h8vbl_bfbd82d0-51a1-4b7f-ba2c-19f17ad3b35f/global-pull-secret-syncer/0.log"
Aug 05 18:23:32.556374 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:32.556340    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-g55gw_bf34f30e-7b6a-49fc-a27f-e1955f248d7c/konnectivity-agent/0.log"
Aug 05 18:23:32.651822 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:32.651788    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-46.ec2.internal_af6ae06b6895f6ad9d2e2bf995b0fd40/haproxy/0.log"
Aug 05 18:23:37.717721 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:37.717015    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-75db7cc4bd-5qrzc_92543e2a-899a-40b4-8c2f-5a25127e378f/authorino/0.log"
Aug 05 18:23:37.787719 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:37.787015    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-qh78n_24913a9d-22fc-451b-b7e3-25cee6d65ce1/manager/0.log"
Aug 05 18:23:38.082475 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:38.082390    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-8qt45_e50c044b-cb40-4340-870e-2fac665bbdf7/manager/0.log"
Aug 05 18:23:39.930692 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:39.930625    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2m9lf_0ece264d-044c-441c-8db0-c84744efa149/node-exporter/0.log"
Aug 05 18:23:39.977335 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:39.977300    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2m9lf_0ece264d-044c-441c-8db0-c84744efa149/kube-rbac-proxy/0.log"
Aug 05 18:23:40.021033 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:40.021001    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2m9lf_0ece264d-044c-441c-8db0-c84744efa149/init-textfile/0.log"
Aug 05 18:23:42.276095 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:42.276058    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8dz5d/perf-node-gather-daemonset-q5fsm"]
Aug 05 18:23:42.283902 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:42.283869    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8dz5d/perf-node-gather-daemonset-q5fsm"
Aug 05 18:23:42.290902 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:42.290869    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8dz5d/perf-node-gather-daemonset-q5fsm"]
Aug 05 18:23:42.353961 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:42.353924    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgr55\" (UniqueName: \"kubernetes.io/projected/d112a867-d048-458b-a250-cba58126a6b0-kube-api-access-jgr55\") pod \"perf-node-gather-daemonset-q5fsm\" (UID: \"d112a867-d048-458b-a250-cba58126a6b0\") " pod="openshift-must-gather-8dz5d/perf-node-gather-daemonset-q5fsm"
Aug 05 18:23:42.354224 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:42.354202    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d112a867-d048-458b-a250-cba58126a6b0-lib-modules\") pod \"perf-node-gather-daemonset-q5fsm\" (UID: \"d112a867-d048-458b-a250-cba58126a6b0\") " pod="openshift-must-gather-8dz5d/perf-node-gather-daemonset-q5fsm"
Aug 05 18:23:42.354363 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:42.354341    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d112a867-d048-458b-a250-cba58126a6b0-podres\") pod \"perf-node-gather-daemonset-q5fsm\" (UID: \"d112a867-d048-458b-a250-cba58126a6b0\") " pod="openshift-must-gather-8dz5d/perf-node-gather-daemonset-q5fsm"
Aug 05 18:23:42.354500 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:42.354485    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d112a867-d048-458b-a250-cba58126a6b0-sys\") pod \"perf-node-gather-daemonset-q5fsm\" (UID: \"d112a867-d048-458b-a250-cba58126a6b0\") " pod="openshift-must-gather-8dz5d/perf-node-gather-daemonset-q5fsm"
Aug 05 18:23:42.354653 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:42.354639    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d112a867-d048-458b-a250-cba58126a6b0-proc\") pod \"perf-node-gather-daemonset-q5fsm\" (UID: \"d112a867-d048-458b-a250-cba58126a6b0\") " pod="openshift-must-gather-8dz5d/perf-node-gather-daemonset-q5fsm"
Aug 05 18:23:42.455265 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:42.455229    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgr55\" (UniqueName: \"kubernetes.io/projected/d112a867-d048-458b-a250-cba58126a6b0-kube-api-access-jgr55\") pod \"perf-node-gather-daemonset-q5fsm\" (UID: \"d112a867-d048-458b-a250-cba58126a6b0\") " pod="openshift-must-gather-8dz5d/perf-node-gather-daemonset-q5fsm"
Aug 05 18:23:42.455265 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:42.455267    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d112a867-d048-458b-a250-cba58126a6b0-lib-modules\") pod \"perf-node-gather-daemonset-q5fsm\" (UID: \"d112a867-d048-458b-a250-cba58126a6b0\") " pod="openshift-must-gather-8dz5d/perf-node-gather-daemonset-q5fsm"
Aug 05 18:23:42.455519 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:42.455289    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d112a867-d048-458b-a250-cba58126a6b0-podres\") pod \"perf-node-gather-daemonset-q5fsm\" (UID: \"d112a867-d048-458b-a250-cba58126a6b0\") " pod="openshift-must-gather-8dz5d/perf-node-gather-daemonset-q5fsm"
Aug 05 18:23:42.455519 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:42.455317    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d112a867-d048-458b-a250-cba58126a6b0-sys\") pod \"perf-node-gather-daemonset-q5fsm\" (UID: \"d112a867-d048-458b-a250-cba58126a6b0\") " pod="openshift-must-gather-8dz5d/perf-node-gather-daemonset-q5fsm"
Aug 05 18:23:42.455519 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:42.455338    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d112a867-d048-458b-a250-cba58126a6b0-proc\") pod \"perf-node-gather-daemonset-q5fsm\" (UID: \"d112a867-d048-458b-a250-cba58126a6b0\") " pod="openshift-must-gather-8dz5d/perf-node-gather-daemonset-q5fsm"
Aug 05 18:23:42.455519 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:42.455445    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d112a867-d048-458b-a250-cba58126a6b0-proc\") pod \"perf-node-gather-daemonset-q5fsm\" (UID: \"d112a867-d048-458b-a250-cba58126a6b0\") " pod="openshift-must-gather-8dz5d/perf-node-gather-daemonset-q5fsm"
Aug 05 18:23:42.455519 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:42.455453    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d112a867-d048-458b-a250-cba58126a6b0-podres\") pod \"perf-node-gather-daemonset-q5fsm\" (UID: \"d112a867-d048-458b-a250-cba58126a6b0\") " pod="openshift-must-gather-8dz5d/perf-node-gather-daemonset-q5fsm"
Aug 05 18:23:42.455519 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:42.455456    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d112a867-d048-458b-a250-cba58126a6b0-lib-modules\") pod \"perf-node-gather-daemonset-q5fsm\" (UID: \"d112a867-d048-458b-a250-cba58126a6b0\") " pod="openshift-must-gather-8dz5d/perf-node-gather-daemonset-q5fsm"
Aug 05 18:23:42.455519 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:42.455444    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d112a867-d048-458b-a250-cba58126a6b0-sys\") pod \"perf-node-gather-daemonset-q5fsm\" (UID: \"d112a867-d048-458b-a250-cba58126a6b0\") " pod="openshift-must-gather-8dz5d/perf-node-gather-daemonset-q5fsm"
Aug 05 18:23:42.463366 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:42.463337    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgr55\" (UniqueName: \"kubernetes.io/projected/d112a867-d048-458b-a250-cba58126a6b0-kube-api-access-jgr55\") pod \"perf-node-gather-daemonset-q5fsm\" (UID: \"d112a867-d048-458b-a250-cba58126a6b0\") " pod="openshift-must-gather-8dz5d/perf-node-gather-daemonset-q5fsm"
Aug 05 18:23:42.599188 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:42.599102    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8dz5d/perf-node-gather-daemonset-q5fsm"
Aug 05 18:23:42.823665 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:42.823578    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8dz5d/perf-node-gather-daemonset-q5fsm"]
Aug 05 18:23:43.183106 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:43.182921    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8dz5d/perf-node-gather-daemonset-q5fsm" event={"ID":"d112a867-d048-458b-a250-cba58126a6b0","Type":"ContainerStarted","Data":"6512f650bb0a3eeca37fddf10e847bd8977eba5dd837aa14df227de4be9311ae"}
Aug 05 18:23:43.183106 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:43.182988    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8dz5d/perf-node-gather-daemonset-q5fsm" event={"ID":"d112a867-d048-458b-a250-cba58126a6b0","Type":"ContainerStarted","Data":"6ec6910a8cf3edbf9cd4e4eb772192fbc9314a2377de9fc89a01c55c6e65513f"}
Aug 05 18:23:43.183370 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:43.183202    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8dz5d/perf-node-gather-daemonset-q5fsm"
Aug 05 18:23:43.203737 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:43.203547    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8dz5d/perf-node-gather-daemonset-q5fsm" podStartSLOduration=1.203526507 podStartE2EDuration="1.203526507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 18:23:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:23:43.199888221 +0000 UTC m=+3730.181465648" watchObservedRunningTime="2026-08-05 18:23:43.203526507 +0000 UTC m=+3730.185103939"
Aug 05 18:23:45.154560 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:45.154532    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6xcf5_976b5f92-171d-43bc-9117-b2f42f830596/dns/0.log"
Aug 05 18:23:45.183801 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:45.183770    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6xcf5_976b5f92-171d-43bc-9117-b2f42f830596/kube-rbac-proxy/0.log"
Aug 05 18:23:45.335189 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:45.335146    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2f92r_b382a0b8-435d-430a-a8a2-faa3d0b3bdc1/dns-node-resolver/0.log"
Aug 05 18:23:45.994269 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:45.994235    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bpcrr_610a012b-bd9a-4627-8976-b1f785945d61/node-ca/0.log"
Aug 05 18:23:47.031904 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:47.031869    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c_3f44bcc3-3f40-403f-bcc8-791f5472340b/istio-proxy/0.log"
Aug 05 18:23:47.326876 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:47.326801    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-746fd96bc4-btn27_8b6db924-1edc-415d-91c3-292040e1f654/kube-auth-proxy/0.log"
Aug 05 18:23:47.361336 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:47.361306    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6czx5_1b600ba3-1ae9-4de8-bffd-9bac75f659ad/istio-proxy/0.log"
Aug 05 18:23:47.369071 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:47.369049    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6czx5_1b600ba3-1ae9-4de8-bffd-9bac75f659ad/istio-proxy/1.log"
Aug 05 18:23:47.440510 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:47.440476    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-crrck_b762d708-104b-4bbb-ba46-13a3aff27b83/payload-processing/0.log"
Aug 05 18:23:48.334815 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:48.334788    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gxsh5_8bdd9d59-2279-49b8-882d-210c0d146301/serve-healthcheck-canary/0.log"
Aug 05 18:23:49.017352 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:49.017310    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vgvsc_358a44e2-7356-4ac2-afee-afac2f16f107/kube-rbac-proxy/0.log"
Aug 05 18:23:49.042143 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:49.042117    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vgvsc_358a44e2-7356-4ac2-afee-afac2f16f107/exporter/0.log"
Aug 05 18:23:49.064998 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:49.064960    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vgvsc_358a44e2-7356-4ac2-afee-afac2f16f107/extractor/0.log"
Aug 05 18:23:49.200850 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:49.200125    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8dz5d/perf-node-gather-daemonset-q5fsm"
Aug 05 18:23:51.622509 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:51.622465    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-fbrn7_9409ab12-e7ca-44e9-84a5-4bc4d3f46916/manager/0.log"
Aug 05 18:23:51.921209 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:51.921183    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-2ldj9_56c44a81-44f1-451a-a9ed-add009f98ca7/manager/0.log"
Aug 05 18:23:52.146818 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:52.146782    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6bb68d8457-cqsfl_cf4d0eeb-01e0-4740-b068-1b1a838dec25/manager/0.log"
Aug 05 18:23:52.173837 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:52.173747    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-kxjqm_e24fda29-710a-4354-9a13-a35317191c15/server/0.log"
Aug 05 18:23:52.268488 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:52.268456    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-bfsdk_db99dc27-ca54-4ccb-ac40-eec6ddfbb1d5/manager/0.log"
Aug 05 18:23:52.420851 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:52.420820    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cc69f54b9-qlhch_4fd60b10-0d46-4be8-b0d6-73ad8455b81d/manager/0.log"
Aug 05 18:23:52.443290 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:52.443209    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cc69f54b9-qlhch_4fd60b10-0d46-4be8-b0d6-73ad8455b81d/copy-manifests/0.log"
Aug 05 18:23:52.980171 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:52.980137    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c_3f44bcc3-3f40-403f-bcc8-791f5472340b/istio-proxy/0.log"
Aug 05 18:23:53.211137 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:53.211106    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-746fd96bc4-btn27_8b6db924-1edc-415d-91c3-292040e1f654/kube-auth-proxy/0.log"
Aug 05 18:23:53.230280 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:53.230184    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6czx5_1b600ba3-1ae9-4de8-bffd-9bac75f659ad/istio-proxy/0.log"
Aug 05 18:23:53.238630 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:53.238606    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6czx5_1b600ba3-1ae9-4de8-bffd-9bac75f659ad/istio-proxy/1.log"
Aug 05 18:23:53.280792 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:53.280752    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-crrck_b762d708-104b-4bbb-ba46-13a3aff27b83/payload-processing/0.log"
Aug 05 18:23:54.646410 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:54.646372    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-hjd5c_3f44bcc3-3f40-403f-bcc8-791f5472340b/istio-proxy/0.log"
Aug 05 18:23:54.847962 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:54.847898    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-746fd96bc4-btn27_8b6db924-1edc-415d-91c3-292040e1f654/kube-auth-proxy/0.log"
Aug 05 18:23:54.869065 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:54.869033    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6czx5_1b600ba3-1ae9-4de8-bffd-9bac75f659ad/istio-proxy/0.log"
Aug 05 18:23:54.876746 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:54.876690    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6czx5_1b600ba3-1ae9-4de8-bffd-9bac75f659ad/istio-proxy/1.log"
Aug 05 18:23:54.924277 ip-10-0-134-46 kubenswrapper[2550]: I0805 18:23:54.924248    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-crrck_b762d708-104b-4bbb-ba46-13a3aff27b83/payload-processing/0.log"