Aug 04 17:51:56.154973 ip-10-0-128-172 systemd[1]: Starting Kubernetes Kubelet...
Aug 04 17:51:56.615951 ip-10-0-128-172 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 04 17:51:56.615951 ip-10-0-128-172 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Aug 04 17:51:56.615951 ip-10-0-128-172 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 04 17:51:56.615951 ip-10-0-128-172 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Aug 04 17:51:56.615951 ip-10-0-128-172 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 04 17:51:56.617025 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.616915    2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Aug 04 17:51:56.625241 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625221    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 17:51:56.625241 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625238    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 17:51:56.625241 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625243    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 17:51:56.625241 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625248    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 17:51:56.625492 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625252    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 17:51:56.625492 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625256    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 17:51:56.625492 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625259    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 17:51:56.625492 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625263    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 17:51:56.625492 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625267    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 17:51:56.625492 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625270    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 17:51:56.625492 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625277    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 17:51:56.625492 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625282    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 17:51:56.625492 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625287    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 17:51:56.625492 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625291    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 17:51:56.625492 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625295    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 17:51:56.625492 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625298    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 17:51:56.625492 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625302    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 17:51:56.625492 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625306    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 17:51:56.625492 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625310    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 17:51:56.625492 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625314    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 17:51:56.625492 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625317    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 17:51:56.625492 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625321    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 17:51:56.625492 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625325    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 17:51:56.626300 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625329    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 17:51:56.626300 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625333    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 17:51:56.626300 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625336    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 17:51:56.626300 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625340    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 17:51:56.626300 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625344    2560 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 17:51:56.626300 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625347    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 17:51:56.626300 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625351    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 17:51:56.626300 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625355    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 17:51:56.626300 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625359    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 17:51:56.626300 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625363    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 17:51:56.626300 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625367    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 17:51:56.626300 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625371    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 17:51:56.626300 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625375    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 17:51:56.626300 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625379    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 17:51:56.626300 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625388    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 17:51:56.626300 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625395    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 17:51:56.626300 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625400    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 17:51:56.626300 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625405    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 17:51:56.626300 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625410    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 17:51:56.626300 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625414    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 17:51:56.627198 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625419    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 17:51:56.627198 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625423    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 17:51:56.627198 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625427    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 17:51:56.627198 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625431    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 17:51:56.627198 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625435    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 17:51:56.627198 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625440    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 17:51:56.627198 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625446    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 17:51:56.627198 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625452    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 17:51:56.627198 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625457    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 17:51:56.627198 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625462    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 17:51:56.627198 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625466    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 17:51:56.627198 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625470    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 17:51:56.627198 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625474    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 17:51:56.627198 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625478    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 17:51:56.627198 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625482    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 17:51:56.627198 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625487    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 17:51:56.627198 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625490    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 17:51:56.627198 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625494    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 17:51:56.627198 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625498    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 17:51:56.627995 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625503    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 17:51:56.627995 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625507    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 17:51:56.627995 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625511    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 17:51:56.627995 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625515    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 17:51:56.627995 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625519    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 17:51:56.627995 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625523    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 17:51:56.627995 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625527    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 17:51:56.627995 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625533    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 17:51:56.627995 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625537    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 17:51:56.627995 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625547    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 17:51:56.627995 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625551    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 17:51:56.627995 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625557    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 17:51:56.627995 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625562    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 17:51:56.627995 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625566    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 17:51:56.627995 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625572    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 17:51:56.627995 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625576    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 17:51:56.627995 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625580    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 17:51:56.627995 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625584    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 17:51:56.627995 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625588    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 17:51:56.627995 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625593    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 17:51:56.628874 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625597    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 17:51:56.628874 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625601    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 17:51:56.628874 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625605    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 17:51:56.628874 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625609    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 17:51:56.628874 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625614    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 17:51:56.628874 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625618    2560 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 17:51:56.628874 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625622    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 17:51:56.628874 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625633    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 17:51:56.628874 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625638    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 17:51:56.628874 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625642    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 17:51:56.628874 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625646    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 17:51:56.628874 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625650    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 17:51:56.628874 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625655    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 17:51:56.628874 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625659    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 17:51:56.628874 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625663    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 17:51:56.628874 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625667    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 17:51:56.628874 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.625671    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 17:51:56.628874 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626835    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 17:51:56.628874 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626844    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 17:51:56.628874 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626850    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 17:51:56.629771 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626855    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 17:51:56.629771 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626859    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 17:51:56.629771 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626864    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 17:51:56.629771 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626868    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 17:51:56.629771 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626875    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 17:51:56.629771 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626880    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 17:51:56.629771 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626885    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 17:51:56.629771 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626892    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 17:51:56.629771 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626898    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 17:51:56.629771 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626903    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 17:51:56.629771 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626908    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 17:51:56.629771 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626912    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 17:51:56.629771 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626916    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 17:51:56.629771 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626921    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 17:51:56.629771 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626925    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 17:51:56.629771 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626929    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 17:51:56.629771 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626933    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 17:51:56.629771 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626938    2560 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 17:51:56.629771 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626942    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 17:51:56.630311 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626946    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 17:51:56.630311 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626950    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 17:51:56.630311 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626954    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 17:51:56.630311 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626959    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 17:51:56.630311 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626963    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 17:51:56.630311 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626966    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 17:51:56.630311 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626971    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 17:51:56.630311 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626974    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 17:51:56.630311 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626979    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 17:51:56.630311 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626983    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 17:51:56.630311 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626987    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 17:51:56.630311 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626991    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 17:51:56.630311 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.626996    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 17:51:56.630311 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627000    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 17:51:56.630311 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627004    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 17:51:56.630311 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627009    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 17:51:56.630311 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627014    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 17:51:56.630311 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627018    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 17:51:56.630311 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627022    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 17:51:56.630311 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627026    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 17:51:56.630813 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627030    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 17:51:56.630813 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627034    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 17:51:56.630813 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627038    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 17:51:56.630813 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627042    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 17:51:56.630813 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627048    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 17:51:56.630813 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627052    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 17:51:56.630813 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627056    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 17:51:56.630813 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627060    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 17:51:56.630813 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627064    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 17:51:56.630813 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627068    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 17:51:56.630813 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627072    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 17:51:56.630813 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627076    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 17:51:56.630813 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627080    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 17:51:56.630813 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627085    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 17:51:56.630813 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627089    2560 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 17:51:56.630813 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627093    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 17:51:56.630813 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627097    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 17:51:56.630813 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627101    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 17:51:56.630813 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627105    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 17:51:56.630813 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627109    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 17:51:56.631291 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627113    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 17:51:56.631291 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627118    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 17:51:56.631291 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627122    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 17:51:56.631291 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627126    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 17:51:56.631291 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627131    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 17:51:56.631291 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627135    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 17:51:56.631291 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627139    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 17:51:56.631291 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627144    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 17:51:56.631291 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627148    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 17:51:56.631291 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627153    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 17:51:56.631291 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627157    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 17:51:56.631291 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627161    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 17:51:56.631291 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627165    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 17:51:56.631291 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627169    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 17:51:56.631291 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627173    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 17:51:56.631291 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627177    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 17:51:56.631291 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627181    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 17:51:56.631291 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627185    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 17:51:56.631291 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627191    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 17:51:56.631291 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627195    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 17:51:56.631862 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627199    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 17:51:56.631862 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627203    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 17:51:56.631862 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627207    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 17:51:56.631862 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627211    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 17:51:56.631862 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627215    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 17:51:56.631862 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627218    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 17:51:56.631862 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627222    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 17:51:56.631862 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627226    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 17:51:56.631862 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627231    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 17:51:56.631862 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627235    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 17:51:56.631862 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627239    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 17:51:56.631862 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627243    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 17:51:56.631862 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627251    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 17:51:56.631862 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627255    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 17:51:56.631862 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627260    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 17:51:56.631862 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627267    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 17:51:56.631862 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627272    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 17:51:56.631862 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627864    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 17:51:56.631862 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627872    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 17:51:56.632514 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627877    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 17:51:56.632514 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627881    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 17:51:56.632514 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627887    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 17:51:56.632514 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627891    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 17:51:56.632514 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627895    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 17:51:56.632514 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627899    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 17:51:56.632514 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627904    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 17:51:56.632514 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627908    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 17:51:56.632514 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627912    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 17:51:56.632514 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627915    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 17:51:56.632514 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627920    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 17:51:56.632514 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627924    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 17:51:56.632514 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627928    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 17:51:56.632514 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627937    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 17:51:56.632514 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627942    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 17:51:56.632514 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627946    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 17:51:56.632514 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627950    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 17:51:56.632514 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627954    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 17:51:56.632514 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627958    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 17:51:56.632514 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627962    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 17:51:56.633154 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627966    2560 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 17:51:56.633154 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627971    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 17:51:56.633154 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627975    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 17:51:56.633154 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627979    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 17:51:56.633154 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627985    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 17:51:56.633154 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627992    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 17:51:56.633154 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.627997    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 17:51:56.633154 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628001    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 17:51:56.633154 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628005    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 17:51:56.633154 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628010    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 17:51:56.633154 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628014    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 17:51:56.633154 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628018    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 17:51:56.633154 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628022    2560 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 17:51:56.633154 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628026    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 17:51:56.633154 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628030    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 17:51:56.633154 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628035    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 17:51:56.633154 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628039    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 17:51:56.633154 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628045    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 17:51:56.633154 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628049    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 17:51:56.633154 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628054    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 17:51:56.633642 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628058    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 17:51:56.633642 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628062    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 17:51:56.633642 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628067    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 17:51:56.633642 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628071    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 17:51:56.633642 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628075    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 17:51:56.633642 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628080    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 17:51:56.633642 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628084    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 17:51:56.633642 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628088    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 17:51:56.633642 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628117    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 17:51:56.633642 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628124    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 17:51:56.633642 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628129    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 17:51:56.633642 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628134    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 17:51:56.633642 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628138    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 17:51:56.633642 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628143    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 17:51:56.633642 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628147    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 17:51:56.633642 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628151    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 17:51:56.633642 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628155    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 17:51:56.633642 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628161    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 17:51:56.633642 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628165    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 17:51:56.633642 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628169    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 17:51:56.634254 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628173    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 17:51:56.634254 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628177    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 17:51:56.634254 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628181    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 17:51:56.634254 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628185    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 17:51:56.634254 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628189    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 17:51:56.634254 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628194    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 17:51:56.634254 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628199    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 17:51:56.634254 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628203    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 17:51:56.634254 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628217    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 17:51:56.634254 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628221    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 17:51:56.634254 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628225    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 17:51:56.634254 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628229    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 17:51:56.634254 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628234    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 17:51:56.634254 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628238    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 17:51:56.634254 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628242    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 17:51:56.634254 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628246    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 17:51:56.634254 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628250    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 17:51:56.634254 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628255    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 17:51:56.634254 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628259    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 17:51:56.634254 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628263    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 17:51:56.634764 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628267    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 17:51:56.634764 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628271    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 17:51:56.634764 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628276    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 17:51:56.634764 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628280    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 17:51:56.634764 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628284    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 17:51:56.634764 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628291    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 17:51:56.634764 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628296    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 17:51:56.634764 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628301    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 17:51:56.634764 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628305    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 17:51:56.634764 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628312    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 17:51:56.634764 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628316    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 17:51:56.634764 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628320    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 17:51:56.634764 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628325    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 17:51:56.634764 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628329    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 17:51:56.634764 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628334    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 17:51:56.634764 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628338    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 17:51:56.634764 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.628342    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 17:51:56.634764 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628438    2560 flags.go:64] FLAG: --address="0.0.0.0"
Aug 04 17:51:56.634764 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628466    2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Aug 04 17:51:56.634764 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628475    2560 flags.go:64] FLAG: --anonymous-auth="true"
Aug 04 17:51:56.635248 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628482    2560 flags.go:64] FLAG: --application-metrics-count-limit="100"
Aug 04 17:51:56.635248 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628490    2560 flags.go:64] FLAG: --authentication-token-webhook="false"
Aug 04 17:51:56.635248 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628496    2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Aug 04 17:51:56.635248 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628502    2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Aug 04 17:51:56.635248 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628508    2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Aug 04 17:51:56.635248 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628513    2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Aug 04 17:51:56.635248 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628518    2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Aug 04 17:51:56.635248 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628523    2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Aug 04 17:51:56.635248 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628528    2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Aug 04 17:51:56.635248 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628534    2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Aug 04 17:51:56.635248 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628539    2560 flags.go:64] FLAG: --cgroup-root=""
Aug 04 17:51:56.635248 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628544    2560 flags.go:64] FLAG: --cgroups-per-qos="true"
Aug 04 17:51:56.635248 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628548    2560 flags.go:64] FLAG: --client-ca-file=""
Aug 04 17:51:56.635248 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628553    2560 flags.go:64] FLAG: --cloud-provider="external"
Aug 04 17:51:56.635248 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628557    2560 flags.go:64] FLAG: --cluster-dns="[]"
Aug 04 17:51:56.635248 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628568    2560 flags.go:64] FLAG: --cluster-domain=""
Aug 04 17:51:56.635248 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628573    2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Aug 04 17:51:56.635248 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628578    2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Aug 04 17:51:56.635248 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628583    2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Aug 04 17:51:56.635248 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628589    2560 flags.go:64] FLAG: --container-log-max-files="5"
Aug 04 17:51:56.635248 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628594    2560 flags.go:64] FLAG: --container-log-max-size="10Mi"
Aug 04 17:51:56.635248 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628601    2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Aug 04 17:51:56.635248 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628606    2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Aug 04 17:51:56.635851 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628611    2560 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Aug 04 17:51:56.635851 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628616    2560 flags.go:64] FLAG: --contention-profiling="false"
Aug 04 17:51:56.635851 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628621    2560 flags.go:64] FLAG: --cpu-cfs-quota="true"
Aug 04 17:51:56.635851 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628626    2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Aug 04 17:51:56.635851 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628631    2560 flags.go:64] FLAG: --cpu-manager-policy="none"
Aug 04 17:51:56.635851 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628635    2560 flags.go:64] FLAG: --cpu-manager-policy-options=""
Aug 04 17:51:56.635851 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628641    2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Aug 04 17:51:56.635851 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628645    2560 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Aug 04 17:51:56.635851 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628650    2560 flags.go:64] FLAG: --enable-debugging-handlers="true"
Aug 04 17:51:56.635851 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628655    2560 flags.go:64] FLAG: --enable-load-reader="false"
Aug 04 17:51:56.635851 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628660    2560 flags.go:64] FLAG: --enable-server="true"
Aug 04 17:51:56.635851 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628668    2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Aug 04 17:51:56.635851 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628678    2560 flags.go:64] FLAG: --event-burst="100"
Aug 04 17:51:56.635851 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628682    2560 flags.go:64] FLAG: --event-qps="50"
Aug 04 17:51:56.635851 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628687    2560 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Aug 04 17:51:56.635851 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628691    2560 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Aug 04 17:51:56.635851 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628696    2560 flags.go:64] FLAG: --eviction-hard=""
Aug 04 17:51:56.635851 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628702    2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Aug 04 17:51:56.635851 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628707    2560 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Aug 04 17:51:56.635851 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628732    2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Aug 04 17:51:56.635851 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628737    2560 flags.go:64] FLAG: --eviction-soft=""
Aug 04 17:51:56.635851 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628742    2560 flags.go:64] FLAG: --eviction-soft-grace-period=""
Aug 04 17:51:56.635851 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628746    2560 flags.go:64] FLAG: --exit-on-lock-contention="false"
Aug 04 17:51:56.635851 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628751    2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Aug 04 17:51:56.635851 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628755    2560 flags.go:64] FLAG: --experimental-mounter-path=""
Aug 04 17:51:56.636476 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628759    2560 flags.go:64] FLAG: --fail-cgroupv1="false"
Aug 04 17:51:56.636476 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628764    2560 flags.go:64] FLAG: --fail-swap-on="true"
Aug 04 17:51:56.636476 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628768    2560 flags.go:64] FLAG: --feature-gates=""
Aug 04 17:51:56.636476 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628774    2560 flags.go:64] FLAG: --file-check-frequency="20s"
Aug 04 17:51:56.636476 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628779    2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Aug 04 17:51:56.636476 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628786    2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Aug 04 17:51:56.636476 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628791    2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Aug 04 17:51:56.636476 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628796    2560 flags.go:64] FLAG: --healthz-port="10248"
Aug 04 17:51:56.636476 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628801    2560 flags.go:64] FLAG: --help="false"
Aug 04 17:51:56.636476 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628805    2560 flags.go:64] FLAG: --hostname-override="ip-10-0-128-172.ec2.internal"
Aug 04 17:51:56.636476 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628811    2560 flags.go:64] FLAG: --housekeeping-interval="10s"
Aug 04 17:51:56.636476 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628816    2560 flags.go:64] FLAG: --http-check-frequency="20s"
Aug 04 17:51:56.636476 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628820    2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Aug 04 17:51:56.636476 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628826    2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Aug 04 17:51:56.636476 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628831    2560 flags.go:64] FLAG: --image-gc-high-threshold="85"
Aug 04 17:51:56.636476 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628835    2560 flags.go:64] FLAG: --image-gc-low-threshold="80"
Aug 04 17:51:56.636476 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628840    2560 flags.go:64] FLAG: --image-service-endpoint=""
Aug 04 17:51:56.636476 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628845    2560 flags.go:64] FLAG: --kernel-memcg-notification="false"
Aug 04 17:51:56.636476 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628849    2560 flags.go:64] FLAG: --kube-api-burst="100"
Aug 04 17:51:56.636476 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628855    2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Aug 04 17:51:56.636476 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628860    2560 flags.go:64] FLAG: --kube-api-qps="50"
Aug 04 17:51:56.636476 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628864    2560 flags.go:64] FLAG: --kube-reserved=""
Aug 04 17:51:56.636476 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628869    2560 flags.go:64] FLAG: --kube-reserved-cgroup=""
Aug 04 17:51:56.636476 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628873    2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Aug 04 17:51:56.637160 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628878    2560 flags.go:64] FLAG: --kubelet-cgroups=""
Aug 04 17:51:56.637160 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628884    2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Aug 04 17:51:56.637160 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628888    2560 flags.go:64] FLAG: --lock-file=""
Aug 04 17:51:56.637160 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628892    2560 flags.go:64] FLAG: --log-cadvisor-usage="false"
Aug 04 17:51:56.637160 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628896    2560 flags.go:64] FLAG: --log-flush-frequency="5s"
Aug 04 17:51:56.637160 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628901    2560 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Aug 04 17:51:56.637160 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628909    2560 flags.go:64] FLAG: --log-json-split-stream="false"
Aug 04 17:51:56.637160 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628914    2560 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Aug 04 17:51:56.637160 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628919    2560 flags.go:64] FLAG: --log-text-split-stream="false"
Aug 04 17:51:56.637160 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628923    2560 flags.go:64] FLAG: --logging-format="text"
Aug 04 17:51:56.637160 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628928    2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Aug 04 17:51:56.637160 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628934    2560 flags.go:64] FLAG: --make-iptables-util-chains="true"
Aug 04 17:51:56.637160 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628938    2560 flags.go:64] FLAG: --manifest-url=""
Aug 04 17:51:56.637160 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628942    2560 flags.go:64] FLAG: --manifest-url-header=""
Aug 04 17:51:56.637160 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628949    2560 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Aug 04 17:51:56.637160 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628954    2560 flags.go:64] FLAG: --max-open-files="1000000"
Aug 04 17:51:56.637160 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628961    2560 flags.go:64] FLAG: --max-pods="110"
Aug 04 17:51:56.637160 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628965    2560 flags.go:64] FLAG: --maximum-dead-containers="-1"
Aug 04 17:51:56.637160 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628970    2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Aug 04 17:51:56.637160 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628974    2560 flags.go:64] FLAG: --memory-manager-policy="None"
Aug 04 17:51:56.637160 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628979    2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Aug 04 17:51:56.637160 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628983    2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Aug 04 17:51:56.637160 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628988    2560 flags.go:64] FLAG: --node-ip="0.0.0.0"
Aug 04 17:51:56.637160 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.628993    2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Aug 04 17:51:56.637160 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629003    2560 flags.go:64] FLAG: --node-status-max-images="50"
Aug 04 17:51:56.637796 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629008    2560 flags.go:64] FLAG: --node-status-update-frequency="10s"
Aug 04 17:51:56.637796 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629013    2560 flags.go:64] FLAG: --oom-score-adj="-999"
Aug 04 17:51:56.637796 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629017    2560 flags.go:64] FLAG: --pod-cidr=""
Aug 04 17:51:56.637796 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629022    2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8"
Aug 04 17:51:56.637796 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629030    2560 flags.go:64] FLAG: --pod-manifest-path=""
Aug 04 17:51:56.637796 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629034    2560 flags.go:64] FLAG: --pod-max-pids="-1"
Aug 04 17:51:56.637796 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629039    2560 flags.go:64] FLAG: --pods-per-core="0"
Aug 04 17:51:56.637796 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629044    2560 flags.go:64] FLAG: --port="10250"
Aug 04 17:51:56.637796 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629050    2560 flags.go:64] FLAG: --protect-kernel-defaults="false"
Aug 04 17:51:56.637796 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629054    2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05dfaf7eeaa55786c"
Aug 04 17:51:56.637796 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629059    2560 flags.go:64] FLAG: --qos-reserved=""
Aug 04 17:51:56.637796 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629064    2560 flags.go:64] FLAG: --read-only-port="10255"
Aug 04 17:51:56.637796 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629068    2560 flags.go:64] FLAG: --register-node="true"
Aug 04 17:51:56.637796 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629072    2560 flags.go:64] FLAG: --register-with-taints=""
Aug 04 17:51:56.637796 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629078    2560 flags.go:64] FLAG: --registry-burst="10"
Aug 04 17:51:56.637796 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629082    2560 flags.go:64] FLAG: --registry-qps="5"
Aug 04 17:51:56.637796 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629087    2560 flags.go:64] FLAG: --reserved-cpus=""
Aug 04 17:51:56.637796 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629092    2560 flags.go:64] FLAG: --reserved-memory=""
Aug 04 17:51:56.637796 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629097    2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Aug 04 17:51:56.637796 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629101    2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Aug 04 17:51:56.637796 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629106    2560 flags.go:64] FLAG: --rotate-certificates="false"
Aug 04 17:51:56.637796 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629111    2560 flags.go:64] FLAG: --rotate-server-certificates="false"
Aug 04 17:51:56.637796 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629115    2560 flags.go:64] FLAG: --runonce="false"
Aug 04 17:51:56.637796 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629120    2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Aug 04 17:51:56.637796 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629126    2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Aug 04 17:51:56.638405 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629131    2560 flags.go:64] FLAG: --seccomp-default="false"
Aug 04 17:51:56.638405 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629135    2560 flags.go:64] FLAG: --serialize-image-pulls="true"
Aug 04 17:51:56.638405 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629140    2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Aug 04 17:51:56.638405 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629144    2560 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Aug 04 17:51:56.638405 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629149    2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Aug 04 17:51:56.638405 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629154    2560 flags.go:64] FLAG: --storage-driver-password="root"
Aug 04 17:51:56.638405 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629158    2560 flags.go:64] FLAG: --storage-driver-secure="false"
Aug 04 17:51:56.638405 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629162    2560 flags.go:64] FLAG: --storage-driver-table="stats"
Aug 04 17:51:56.638405 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629167    2560 flags.go:64] FLAG: --storage-driver-user="root"
Aug 04 17:51:56.638405 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629172    2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Aug 04 17:51:56.638405 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629177    2560 flags.go:64] FLAG: --sync-frequency="1m0s"
Aug 04 17:51:56.638405 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629182    2560 flags.go:64] FLAG: --system-cgroups=""
Aug 04 17:51:56.638405 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629186    2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Aug 04 17:51:56.638405 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629195    2560 flags.go:64] FLAG: --system-reserved-cgroup=""
Aug 04 17:51:56.638405 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629199    2560 flags.go:64] FLAG: --tls-cert-file=""
Aug 04 17:51:56.638405 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629205    2560 flags.go:64] FLAG: --tls-cipher-suites="[]"
Aug 04 17:51:56.638405 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629212    2560 flags.go:64] FLAG: --tls-min-version=""
Aug 04 17:51:56.638405 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629217    2560 flags.go:64] FLAG: --tls-private-key-file=""
Aug 04 17:51:56.638405 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629221    2560 flags.go:64] FLAG: --topology-manager-policy="none"
Aug 04 17:51:56.638405 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629226    2560 flags.go:64] FLAG: --topology-manager-policy-options=""
Aug 04 17:51:56.638405 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629230    2560 flags.go:64] FLAG: --topology-manager-scope="container"
Aug 04 17:51:56.638405 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629235    2560 flags.go:64] FLAG: --v="2"
Aug 04 17:51:56.638405 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629241    2560 flags.go:64] FLAG: --version="false"
Aug 04 17:51:56.638405 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629247    2560 flags.go:64] FLAG: --vmodule=""
Aug 04 17:51:56.638405 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629253    2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Aug 04 17:51:56.639021 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.629258    2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Aug 04 17:51:56.639021 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629432    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 17:51:56.639021 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629438    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 17:51:56.639021 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629443    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 17:51:56.639021 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629447    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 17:51:56.639021 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629451    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 17:51:56.639021 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629455    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 17:51:56.639021 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629460    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 17:51:56.639021 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629465    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 17:51:56.639021 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629469    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 17:51:56.639021 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629473    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 17:51:56.639021 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629477    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 17:51:56.639021 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629482    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 17:51:56.639021 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629486    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 17:51:56.639021 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629490    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 17:51:56.639021 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629494    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 17:51:56.639021 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629498    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 17:51:56.639021 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629502    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 17:51:56.639021 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629506    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 17:51:56.639021 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629512    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 17:51:56.639537 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629517    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 17:51:56.639537 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629521    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 17:51:56.639537 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629526    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 17:51:56.639537 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629530    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 17:51:56.639537 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629534    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 17:51:56.639537 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629538    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 17:51:56.639537 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629542    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 17:51:56.639537 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629546    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 17:51:56.639537 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629550    2560 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 17:51:56.639537 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629554    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 17:51:56.639537 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629559    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 17:51:56.639537 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629563    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 17:51:56.639537 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629567    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 17:51:56.639537 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629570    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 17:51:56.639537 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629574    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 17:51:56.639537 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629578    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 17:51:56.639537 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629583    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 17:51:56.639537 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629587    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 17:51:56.639537 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629591    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 17:51:56.639537 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629595    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 17:51:56.640035 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629599    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 17:51:56.640035 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629603    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 17:51:56.640035 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629607    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 17:51:56.640035 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629611    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 17:51:56.640035 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629616    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 17:51:56.640035 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629620    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 17:51:56.640035 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629624    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 17:51:56.640035 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629637    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 17:51:56.640035 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629642    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 17:51:56.640035 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629646    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 17:51:56.640035 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629650    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 17:51:56.640035 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629654    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 17:51:56.640035 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629658    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 17:51:56.640035 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629662    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 17:51:56.640035 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629668    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 17:51:56.640035 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629672    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 17:51:56.640035 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629676    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 17:51:56.640035 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629680    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 17:51:56.640035 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629684    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 17:51:56.640035 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629688    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 17:51:56.640515 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629692    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 17:51:56.640515 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629696    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 17:51:56.640515 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629700    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 17:51:56.640515 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629707    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 17:51:56.640515 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629729    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 17:51:56.640515 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629734    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 17:51:56.640515 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629738    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 17:51:56.640515 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629745    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 17:51:56.640515 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629749    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 17:51:56.640515 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629754    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 17:51:56.640515 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629759    2560 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 17:51:56.640515 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629763    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 17:51:56.640515 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629767    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 17:51:56.640515 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629772    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 17:51:56.640515 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629775    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 17:51:56.640515 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629779    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 17:51:56.640515 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629783    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 17:51:56.640515 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629787    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 17:51:56.640515 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629793    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 17:51:56.640985 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629798    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 17:51:56.640985 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629802    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 17:51:56.640985 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629806    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 17:51:56.640985 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629811    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 17:51:56.640985 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629815    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 17:51:56.640985 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629819    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 17:51:56.640985 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629825    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 17:51:56.640985 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629830    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 17:51:56.640985 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629835    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 17:51:56.640985 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629839    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 17:51:56.640985 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629843    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 17:51:56.640985 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629846    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 17:51:56.640985 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629851    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 17:51:56.640985 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629855    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 17:51:56.640985 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629859    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 17:51:56.640985 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629864    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 17:51:56.640985 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629868    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 17:51:56.640985 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629872    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 17:51:56.640985 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629876    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 17:51:56.641433 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629883    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 17:51:56.641433 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.629891    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 17:51:56.641433 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.631358    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 04 17:51:56.641433 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.638851    2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Aug 04 17:51:56.641433 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.638869    2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Aug 04 17:51:56.641433 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.638885    2560 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Aug 04 17:51:56.641433 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.638891    2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Aug 04 17:51:56.641433 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.638947    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 17:51:56.641433 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.638952    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 17:51:56.641433 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.638955    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 17:51:56.641433 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.638958    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 17:51:56.641433 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.638962    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 17:51:56.641433 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.638966    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 17:51:56.641433 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.638969    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 17:51:56.641816 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.638972    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 17:51:56.641816 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.638974    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 17:51:56.641816 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.638977    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 17:51:56.641816 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.638980    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 17:51:56.641816 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.638982    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 17:51:56.641816 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.638985    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 17:51:56.641816 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.638987    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 17:51:56.641816 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.638990    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 17:51:56.641816 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.638993    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 17:51:56.641816 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.638995    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 17:51:56.641816 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.638998    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 17:51:56.641816 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639000    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 17:51:56.641816 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639003    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 17:51:56.641816 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639005    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 17:51:56.641816 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639008    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 17:51:56.641816 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639010    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 17:51:56.641816 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639013    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 17:51:56.641816 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639015    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 17:51:56.641816 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639017    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 17:51:56.641816 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639020    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 17:51:56.642289 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639023    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 17:51:56.642289 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639025    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 17:51:56.642289 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639048    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 17:51:56.642289 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639052    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 17:51:56.642289 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639055    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 17:51:56.642289 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639058    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 17:51:56.642289 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639061    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 17:51:56.642289 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639064    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 17:51:56.642289 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639066    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 17:51:56.642289 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639068    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 17:51:56.642289 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639071    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 17:51:56.642289 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639073    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 17:51:56.642289 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639076    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 17:51:56.642289 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639078    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 17:51:56.642289 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639081    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 17:51:56.642289 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639084    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 17:51:56.642289 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639086    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 17:51:56.642289 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639089    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 17:51:56.642289 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639091    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 17:51:56.642289 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639093    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 17:51:56.642821 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639096    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 17:51:56.642821 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639099    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 17:51:56.642821 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639101    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 17:51:56.642821 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639103    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 17:51:56.642821 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639106    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 17:51:56.642821 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639108    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 17:51:56.642821 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639112    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 17:51:56.642821 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639116    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 17:51:56.642821 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639119    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 17:51:56.642821 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639122    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 17:51:56.642821 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639124    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 17:51:56.642821 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639127    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 17:51:56.642821 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639130    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 17:51:56.642821 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639132    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 17:51:56.642821 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639135    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 17:51:56.642821 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639143    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 17:51:56.642821 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639146    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 17:51:56.642821 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639150    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 17:51:56.642821 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639153    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 17:51:56.643286 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639156    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 17:51:56.643286 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639158    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 17:51:56.643286 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639161    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 17:51:56.643286 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639163    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 17:51:56.643286 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639166    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 17:51:56.643286 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639168    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 17:51:56.643286 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639171    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 17:51:56.643286 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639173    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 17:51:56.643286 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639176    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 17:51:56.643286 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639178    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 17:51:56.643286 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639181    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 17:51:56.643286 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639184    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 17:51:56.643286 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639186    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 17:51:56.643286 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639189    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 17:51:56.643286 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639191    2560 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 17:51:56.643286 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639194    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 17:51:56.643286 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639196    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 17:51:56.643286 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639199    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 17:51:56.643286 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639202    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 17:51:56.643751 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639204    2560 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 17:51:56.643751 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639207    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 17:51:56.643751 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639209    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 17:51:56.643751 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639211    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 17:51:56.643751 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639214    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 17:51:56.643751 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639216    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 17:51:56.643751 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639219    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 17:51:56.643751 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639221    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 17:51:56.643751 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639225    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 17:51:56.643751 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639228    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 17:51:56.643751 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639235    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 17:51:56.643751 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639238    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 17:51:56.643751 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639240    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 17:51:56.643751 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639243    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 17:51:56.643751 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.639247    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 04 17:51:56.644115 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639402    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 17:51:56.644115 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639407    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 17:51:56.644115 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639410    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 17:51:56.644115 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639413    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 17:51:56.644115 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639416    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 17:51:56.644115 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639420    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 17:51:56.644115 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639423    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 17:51:56.644115 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639426    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 17:51:56.644115 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639429    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 17:51:56.644115 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639431    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 17:51:56.644115 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639434    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 17:51:56.644115 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639436    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 17:51:56.644115 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639439    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 17:51:56.644115 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639441    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 17:51:56.644115 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639443    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 17:51:56.644115 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639446    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 17:51:56.644115 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639448    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 17:51:56.644115 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639451    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 17:51:56.644115 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639453    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 17:51:56.644115 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639456    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 17:51:56.644594 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639458    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 17:51:56.644594 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639461    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 17:51:56.644594 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639463    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 17:51:56.644594 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639465    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 17:51:56.644594 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639468    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 17:51:56.644594 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639470    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 17:51:56.644594 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639473    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 17:51:56.644594 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639475    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 17:51:56.644594 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639484    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 17:51:56.644594 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639487    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 17:51:56.644594 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639490    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 17:51:56.644594 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639492    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 17:51:56.644594 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639495    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 17:51:56.644594 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639497    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 17:51:56.644594 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639500    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 17:51:56.644594 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639502    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 17:51:56.644594 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639504    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 17:51:56.644594 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639507    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 17:51:56.644594 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639509    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 17:51:56.644594 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639511    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 17:51:56.645094 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639514    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 17:51:56.645094 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639516    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 17:51:56.645094 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639519    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 17:51:56.645094 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639522    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 17:51:56.645094 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639525    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 17:51:56.645094 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639528    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 17:51:56.645094 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639531    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 17:51:56.645094 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639534    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 17:51:56.645094 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639537    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 17:51:56.645094 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639539    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 17:51:56.645094 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639542    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 17:51:56.645094 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639545    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 17:51:56.645094 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639548    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 17:51:56.645094 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639550    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 17:51:56.645094 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639553    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 17:51:56.645094 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639555    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 17:51:56.645094 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639558    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 17:51:56.645094 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639561    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 17:51:56.645094 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639563    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 17:51:56.645552 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639566    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 17:51:56.645552 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639568    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 17:51:56.645552 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639576    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 17:51:56.645552 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639578    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 17:51:56.645552 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639581    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 17:51:56.645552 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639583    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 17:51:56.645552 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639586    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 17:51:56.645552 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639588    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 17:51:56.645552 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639591    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 17:51:56.645552 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639593    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 17:51:56.645552 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639595    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 17:51:56.645552 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639597    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 17:51:56.645552 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639600    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 17:51:56.645552 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639602    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 17:51:56.645552 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639610    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 17:51:56.645552 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639613    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 17:51:56.645552 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639616    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 17:51:56.645552 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639618    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 17:51:56.645552 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639620    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 17:51:56.646071 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639623    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 17:51:56.646071 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639625    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 17:51:56.646071 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639627    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 17:51:56.646071 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639629    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 17:51:56.646071 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639633    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 17:51:56.646071 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639636    2560 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 17:51:56.646071 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639639    2560 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 17:51:56.646071 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639642    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 17:51:56.646071 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639644    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 17:51:56.646071 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639647    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 17:51:56.646071 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639649    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 17:51:56.646071 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639651    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 17:51:56.646071 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639654    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 17:51:56.646071 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639656    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 17:51:56.646071 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639659    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 17:51:56.646071 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639661    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 17:51:56.646071 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639669    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 17:51:56.646071 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639672    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 17:51:56.646071 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639674    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 17:51:56.646071 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639677    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 17:51:56.646804 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:56.639679    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 17:51:56.646804 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.639683    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 04 17:51:56.646804 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.639805    2560 server.go:962] "Client rotation is on, will bootstrap in background"
Aug 04 17:51:56.646804 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.642083    2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Aug 04 17:51:56.646804 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.643876    2560 server.go:1019] "Starting client certificate rotation"
Aug 04 17:51:56.646804 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.643998    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 04 17:51:56.646804 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.644037    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 04 17:51:56.649591 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.649572    2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 04 17:51:56.650998 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.650981    2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 04 17:51:56.660894 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.660633    2560 log.go:25] "Validated CRI v1 runtime API"
Aug 04 17:51:56.664671 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.664655    2560 log.go:25] "Validated CRI v1 image API"
Aug 04 17:51:56.665404 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.665388    2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Aug 04 17:51:56.668203 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.668178    2560 fs.go:135] Filesystem UUIDs: map[5bf46f63-fd7b-45b4-bf68-bd83f2bbbb15:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 92e0cecf-2425-4272-88f3-48c4d57c387f:/dev/nvme0n1p3]
Aug 04 17:51:56.668272 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.668203    2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Aug 04 17:51:56.674704 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.674591    2560 manager.go:216] Machine: {Timestamp:2026-08-04 17:51:56.672310186 +0000 UTC m=+0.384745228 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3117590 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b675cf85e440cff80e0039cfe087b SystemUUID:ec2b675c-f85e-440c-ff80-e0039cfe087b BootID:cdb89cc3-6d13-431f-9d12-bad6fb8845bd Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3a:94:a7:a1:dd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3a:94:a7:a1:dd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7e:d3:5d:c9:6e:71 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Aug 04 17:51:56.674704 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.674693    2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Aug 04 17:51:56.674878 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.674805    2560 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Aug 04 17:51:56.677772 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.677750    2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Aug 04 17:51:56.677900 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.677776    2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-172.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Aug 04 17:51:56.678538 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.678528    2560 topology_manager.go:138] "Creating topology manager with none policy"
Aug 04 17:51:56.678569 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.678541    2560 container_manager_linux.go:309] "Creating device plugin manager"
Aug 04 17:51:56.678569 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.678549    2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 04 17:51:56.679276 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.679266    2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 04 17:51:56.680316 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.680306    2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Aug 04 17:51:56.682600 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.682583    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 04 17:51:56.685060 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.685049    2560 state_mem.go:36] "Initialized new in-memory state store"
Aug 04 17:51:56.685175 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.685166    2560 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Aug 04 17:51:56.685208 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.685197    2560 kubelet.go:486] "Attempting to sync node with API server"
Aug 04 17:51:56.685238 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.685211    2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Aug 04 17:51:56.685238 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.685233    2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Aug 04 17:51:56.685295 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.685246    2560 kubelet.go:398] "Adding apiserver pod source"
Aug 04 17:51:56.685295 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.685259    2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Aug 04 17:51:56.685634 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.685615    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 04 17:51:56.685677 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.685648    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 04 17:51:56.687917 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.687900    2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Aug 04 17:51:56.691013 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.690998    2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Aug 04 17:51:56.691079 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.691029    2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Aug 04 17:51:56.695698 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.695678    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Aug 04 17:51:56.695698 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.695697    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Aug 04 17:51:56.695698 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.695703    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Aug 04 17:51:56.695831 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.695726    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Aug 04 17:51:56.695831 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.695738    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Aug 04 17:51:56.695831 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.695743    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Aug 04 17:51:56.695831 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.695751    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Aug 04 17:51:56.695831 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.695756    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Aug 04 17:51:56.695831 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.695762    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Aug 04 17:51:56.695831 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.695768    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Aug 04 17:51:56.695831 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.695776    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Aug 04 17:51:56.695831 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.695781    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Aug 04 17:51:56.695831 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.695804    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Aug 04 17:51:56.695831 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.695809    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Aug 04 17:51:56.698788 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.698773    2560 server.go:1268] "Started kubelet"
Aug 04 17:51:56.698917 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.698868    2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Aug 04 17:51:56.698972 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.698937    2560 server_v1.go:49] "podresources" method="list" useActivePods=true
Aug 04 17:51:56.699026 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.698970    2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Aug 04 17:51:56.699555 ip-10-0-128-172 systemd[1]: Started Kubernetes Kubelet.
Aug 04 17:51:56.700300 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.700190    2560 server.go:310] "Adding debug handlers to kubelet server"
Aug 04 17:51:56.702851 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.702835    2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Aug 04 17:51:56.704414 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:51:56.704397    2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Aug 04 17:51:56.705447 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.705434    2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-172.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Aug 04 17:51:56.705536 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:51:56.705502    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Aug 04 17:51:56.705536 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:51:56.705514    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-172.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Aug 04 17:51:56.707321 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:51:56.705578    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-172.ec2.internal.18c8ab96dd4e7a95  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-172.ec2.internal,UID:ip-10-0-128-172.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-172.ec2.internal,},FirstTimestamp:2026-08-04 17:51:56.698753685 +0000 UTC m=+0.411188727,LastTimestamp:2026-08-04 17:51:56.698753685 +0000 UTC m=+0.411188727,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-172.ec2.internal,}"
Aug 04 17:51:56.709567 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.709548    2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Aug 04 17:51:56.709567 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.709556    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Aug 04 17:51:56.709747 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:51:56.709625    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-172.ec2.internal\" not found"
Aug 04 17:51:56.710561 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.710543    2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Aug 04 17:51:56.710561 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.710543    2560 volume_manager.go:311] "The desired_state_of_world populator starts"
Aug 04 17:51:56.710700 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.710572    2560 volume_manager.go:313] "Starting Kubelet Volume Manager"
Aug 04 17:51:56.710700 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.710578    2560 factory.go:55] Registering systemd factory
Aug 04 17:51:56.710700 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.710604    2560 factory.go:223] Registration of the systemd container factory successfully
Aug 04 17:51:56.710700 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.710692    2560 reconstruct.go:97] "Volume reconstruction finished"
Aug 04 17:51:56.710910 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.710733    2560 reconciler.go:29] "Reconciler: start to sync state"
Aug 04 17:51:56.710968 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.710938    2560 factory.go:153] Registering CRI-O factory
Aug 04 17:51:56.710968 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.710952    2560 factory.go:223] Registration of the crio container factory successfully
Aug 04 17:51:56.711057 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.711004    2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Aug 04 17:51:56.711057 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.711037    2560 factory.go:103] Registering Raw factory
Aug 04 17:51:56.711057 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.711052    2560 manager.go:1196] Started watching for new ooms in manager
Aug 04 17:51:56.711492 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.711448    2560 manager.go:349] Starting recovery of all containers
Aug 04 17:51:56.718682 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:51:56.718659    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Aug 04 17:51:56.718893 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:51:56.718871    2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-172.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Aug 04 17:51:56.721352 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.721338    2560 manager.go:354] Recovery completed
Aug 04 17:51:56.723855 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:51:56.723815    2560 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Aug 04 17:51:56.726879 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.726783    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 17:51:56.729313 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.729296    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-172.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 17:51:56.729370 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.729331    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-172.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 17:51:56.729370 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.729348    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-172.ec2.internal" event="NodeHasSufficientPID"
Aug 04 17:51:56.729954 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.729938    2560 cpu_manager.go:222] "Starting CPU manager" policy="none"
Aug 04 17:51:56.729954 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.729952    2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Aug 04 17:51:56.730101 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.729961    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 17:51:56.730101 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.729968    2560 state_mem.go:36] "Initialized new in-memory state store"
Aug 04 17:51:56.730632 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.730617    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-172.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 17:51:56.730687 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.730642    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-172.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 17:51:56.730687 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.730652    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-172.ec2.internal" event="NodeHasSufficientPID"
Aug 04 17:51:56.732098 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.732086    2560 policy_none.go:49] "None policy: Start"
Aug 04 17:51:56.732098 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.732101    2560 memory_manager.go:187] "Starting memorymanager" policy="None"
Aug 04 17:51:56.732169 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.732110    2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Aug 04 17:51:56.734109 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.734099    2560 policy_none.go:47] "Start"
Aug 04 17:51:56.747613 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:51:56.747474    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-172.ec2.internal.18c8ab96df20ce4f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-172.ec2.internal,UID:ip-10-0-128-172.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-172.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-172.ec2.internal,},FirstTimestamp:2026-08-04 17:51:56.729314895 +0000 UTC m=+0.441749938,LastTimestamp:2026-08-04 17:51:56.729314895 +0000 UTC m=+0.441749938,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-172.ec2.internal,}"
Aug 04 17:51:56.755600 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.755580    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8929r"
Aug 04 17:51:56.765326 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:51:56.765269    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-172.ec2.internal.18c8ab96df212ccd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-172.ec2.internal,UID:ip-10-0-128-172.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-172.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-172.ec2.internal,},FirstTimestamp:2026-08-04 17:51:56.729339085 +0000 UTC m=+0.441774130,LastTimestamp:2026-08-04 17:51:56.729339085 +0000 UTC m=+0.441774130,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-172.ec2.internal,}"
Aug 04 17:51:56.765507 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.765485    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8929r"
Aug 04 17:51:56.776586 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.776573    2560 manager.go:337] "Starting Device Plugin manager"
Aug 04 17:51:56.795776 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:51:56.776601    2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Aug 04 17:51:56.795776 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.776610    2560 server.go:87] "Starting device plugin registration server"
Aug 04 17:51:56.795776 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.776877    2560 eviction_manager.go:189] "Eviction manager: starting control loop"
Aug 04 17:51:56.795776 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.776886    2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Aug 04 17:51:56.795776 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.776981    2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Aug 04 17:51:56.795776 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.777055    2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Aug 04 17:51:56.795776 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.777064    2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Aug 04 17:51:56.795776 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:51:56.777543    2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Aug 04 17:51:56.795776 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:51:56.777588    2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-172.ec2.internal\" not found"
Aug 04 17:51:56.816131 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.816107    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Aug 04 17:51:56.817414 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.817399    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Aug 04 17:51:56.817482 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.817434    2560 status_manager.go:244] "Starting to sync pod status with apiserver"
Aug 04 17:51:56.817482 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.817451    2560 kubelet.go:2448] "Starting kubelet main sync loop"
Aug 04 17:51:56.817586 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:51:56.817482    2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Aug 04 17:51:56.820203 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.820178    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 17:51:56.877588 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.877537    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 17:51:56.878240 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.878225    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-172.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 17:51:56.878307 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.878251    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-172.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 17:51:56.878307 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.878262    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-172.ec2.internal" event="NodeHasSufficientPID"
Aug 04 17:51:56.878307 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.878282    2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-172.ec2.internal"
Aug 04 17:51:56.891092 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.891072    2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-172.ec2.internal"
Aug 04 17:51:56.891092 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:51:56.891091    2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-172.ec2.internal\": node \"ip-10-0-128-172.ec2.internal\" not found"
Aug 04 17:51:56.918390 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.918361    2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-172.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-172.ec2.internal"]
Aug 04 17:51:56.918472 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.918426    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 17:51:56.919143 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.919129    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-172.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 17:51:56.919242 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.919155    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-172.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 17:51:56.919242 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.919169    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-172.ec2.internal" event="NodeHasSufficientPID"
Aug 04 17:51:56.921386 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.921371    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 17:51:56.921494 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.921479    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-172.ec2.internal"
Aug 04 17:51:56.921543 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.921506    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 17:51:56.922069 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.922043    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-172.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 17:51:56.922166 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.922072    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-172.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 17:51:56.922166 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.922087    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-172.ec2.internal" event="NodeHasSufficientPID"
Aug 04 17:51:56.922687 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.922671    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-172.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 17:51:56.922765 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.922695    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-172.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 17:51:56.922765 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.922704    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-172.ec2.internal" event="NodeHasSufficientPID"
Aug 04 17:51:56.924869 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.924856    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-172.ec2.internal"
Aug 04 17:51:56.924935 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.924878    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 17:51:56.925491 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.925476    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-172.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 17:51:56.925580 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.925507    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-172.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 17:51:56.925580 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:56.925520    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-172.ec2.internal" event="NodeHasSufficientPID"
Aug 04 17:51:56.938221 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:51:56.938199    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-172.ec2.internal\" not found" node="ip-10-0-128-172.ec2.internal"
Aug 04 17:51:56.941813 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:51:56.941797    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-172.ec2.internal\" not found" node="ip-10-0-128-172.ec2.internal"
Aug 04 17:51:56.947554 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:51:56.947542    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-172.ec2.internal\" not found"
Aug 04 17:51:57.011771 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:57.011751    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fbe04a653517381697f35b4a5ee6bb15-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-172.ec2.internal\" (UID: \"fbe04a653517381697f35b4a5ee6bb15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-172.ec2.internal"
Aug 04 17:51:57.011845 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:57.011776    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbe04a653517381697f35b4a5ee6bb15-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-172.ec2.internal\" (UID: \"fbe04a653517381697f35b4a5ee6bb15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-172.ec2.internal"
Aug 04 17:51:57.011845 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:57.011793    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3428ebcd2d197fe8677d9367742cfcaa-config\") pod \"kube-apiserver-proxy-ip-10-0-128-172.ec2.internal\" (UID: \"3428ebcd2d197fe8677d9367742cfcaa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-172.ec2.internal"
Aug 04 17:51:57.047948 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:51:57.047929    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-172.ec2.internal\" not found"
Aug 04 17:51:57.111954 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:57.111934    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fbe04a653517381697f35b4a5ee6bb15-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-172.ec2.internal\" (UID: \"fbe04a653517381697f35b4a5ee6bb15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-172.ec2.internal"
Aug 04 17:51:57.112023 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:57.111958    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbe04a653517381697f35b4a5ee6bb15-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-172.ec2.internal\" (UID: \"fbe04a653517381697f35b4a5ee6bb15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-172.ec2.internal"
Aug 04 17:51:57.112023 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:57.111976    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3428ebcd2d197fe8677d9367742cfcaa-config\") pod \"kube-apiserver-proxy-ip-10-0-128-172.ec2.internal\" (UID: \"3428ebcd2d197fe8677d9367742cfcaa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-172.ec2.internal"
Aug 04 17:51:57.112023 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:57.112011    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3428ebcd2d197fe8677d9367742cfcaa-config\") pod \"kube-apiserver-proxy-ip-10-0-128-172.ec2.internal\" (UID: \"3428ebcd2d197fe8677d9367742cfcaa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-172.ec2.internal"
Aug 04 17:51:57.112122 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:57.112037    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fbe04a653517381697f35b4a5ee6bb15-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-172.ec2.internal\" (UID: \"fbe04a653517381697f35b4a5ee6bb15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-172.ec2.internal"
Aug 04 17:51:57.112122 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:57.112042    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbe04a653517381697f35b4a5ee6bb15-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-172.ec2.internal\" (UID: \"fbe04a653517381697f35b4a5ee6bb15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-172.ec2.internal"
Aug 04 17:51:57.148036 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:51:57.147978    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-172.ec2.internal\" not found"
Aug 04 17:51:57.240757 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:57.240737    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-172.ec2.internal"
Aug 04 17:51:57.244489 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:57.244471    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-172.ec2.internal"
Aug 04 17:51:57.248980 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:51:57.248965    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-172.ec2.internal\" not found"
Aug 04 17:51:57.349491 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:51:57.349456    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-172.ec2.internal\" not found"
Aug 04 17:51:57.449934 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:51:57.449876    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-172.ec2.internal\" not found"
Aug 04 17:51:57.550432 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:51:57.550397    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-172.ec2.internal\" not found"
Aug 04 17:51:57.642926 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:57.642904    2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Aug 04 17:51:57.643290 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:57.643037    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 04 17:51:57.651131 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:51:57.651116    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-172.ec2.internal\" not found"
Aug 04 17:51:57.752221 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:51:57.752159    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-172.ec2.internal\" not found"
Aug 04 17:51:57.771332 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:57.771301    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-03 17:46:58 +0000 UTC" deadline="2028-02-22 07:40:09.137416165 +0000 UTC"
Aug 04 17:51:57.771332 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:57.771326    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13597h48m11.366093304s"
Aug 04 17:51:57.822214 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:57.822182    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 17:51:57.823656 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:57.823634    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-172.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 17:51:57.823761 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:57.823669    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-172.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 17:51:57.823761 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:57.823682    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-172.ec2.internal" event="NodeHasSufficientPID"
Aug 04 17:51:57.852508 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:51:57.852485    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-172.ec2.internal\" not found"
Aug 04 17:51:57.873152 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:57.873132    2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 17:51:57.910338 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:57.910311    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-172.ec2.internal"
Aug 04 17:51:57.926868 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:57.926850    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 04 17:51:57.929197 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:57.929177    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-172.ec2.internal"
Aug 04 17:51:57.948595 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:57.948579    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 04 17:51:57.980505 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:57.980475    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3428ebcd2d197fe8677d9367742cfcaa.slice/crio-1837e021c56dd856a107b6f81ac54cdabbb068ae10b8dfa4e160d31d7dd2d5f5 WatchSource:0}: Error finding container 1837e021c56dd856a107b6f81ac54cdabbb068ae10b8dfa4e160d31d7dd2d5f5: Status 404 returned error can't find the container with id 1837e021c56dd856a107b6f81ac54cdabbb068ae10b8dfa4e160d31d7dd2d5f5
Aug 04 17:51:57.980984 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:51:57.980963    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbe04a653517381697f35b4a5ee6bb15.slice/crio-b94219d22b02203687a54372f12c03c1683c794f01d1a8df81cc883f970f2295 WatchSource:0}: Error finding container b94219d22b02203687a54372f12c03c1683c794f01d1a8df81cc883f970f2295: Status 404 returned error can't find the container with id b94219d22b02203687a54372f12c03c1683c794f01d1a8df81cc883f970f2295
Aug 04 17:51:57.985900 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:57.985885    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 17:51:57.987880 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:57.987860    2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 17:51:57.991427 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:51:57.991406    2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 17:52:00.216340 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.216319    2560 apiserver.go:52] "Watching apiserver"
Aug 04 17:52:00.224877 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.224693    2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Aug 04 17:52:00.225151 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.225099    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-f2zqg","openshift-multus/network-metrics-daemon-sstpm","openshift-network-diagnostics/network-check-target-5x9t4","openshift-ovn-kubernetes/ovnkube-node-2wj5g","kube-system/konnectivity-agent-qrkjb","kube-system/kube-apiserver-proxy-ip-10-0-128-172.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9","openshift-cluster-node-tuning-operator/tuned-c8zhn","openshift-dns/node-resolver-lzkvj","openshift-image-registry/node-ca-mdtdv","openshift-multus/multus-additional-cni-plugins-fjnbn","openshift-network-operator/iptables-alerter-bq2vk","kube-system/global-pull-secret-syncer-jgm5s","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-172.ec2.internal"]
Aug 04 17:52:00.227653 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.227634    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9"
Aug 04 17:52:00.229801 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.229781    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:00.229888 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:00.229841    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jgm5s" podUID="5ee91876-14a5-4691-9f54-0d3d11cf70e3"
Aug 04 17:52:00.231896 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.231880    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:52:00.231981 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:00.231947    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5x9t4" podUID="fba423bd-0c4b-4b5c-851d-f604236048c3"
Aug 04 17:52:00.234142 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.234121    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Aug 04 17:52:00.234231 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.234126    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.234231 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.234144    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Aug 04 17:52:00.234231 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.234169    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-z2l9x\""
Aug 04 17:52:00.234380 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.234123    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Aug 04 17:52:00.236281 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.236266    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qrkjb"
Aug 04 17:52:00.237940 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.237924    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Aug 04 17:52:00.238568 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.238551    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.239321 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.239306    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Aug 04 17:52:00.239778 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.239666    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Aug 04 17:52:00.240462 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.240437    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fh62v\""
Aug 04 17:52:00.240555 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.240446    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Aug 04 17:52:00.240555 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.240539    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Aug 04 17:52:00.240666 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.240635    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Aug 04 17:52:00.240738 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.240721    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Aug 04 17:52:00.241314 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.240724    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-w96m9\""
Aug 04 17:52:00.241314 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.240804    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Aug 04 17:52:00.241314 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.240867    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Aug 04 17:52:00.241314 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.241113    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lzkvj"
Aug 04 17:52:00.241314 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.241180    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Aug 04 17:52:00.241314 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.241214    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Aug 04 17:52:00.241624 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.241609    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nbcb8\""
Aug 04 17:52:00.243283 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.243267    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.244841 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.244824    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Aug 04 17:52:00.244937 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.244911    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Aug 04 17:52:00.245881 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.245866    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:00.245965 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:00.245923    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sstpm" podUID="6b581e21-8a6f-4ed7-8751-87941a6cbac3"
Aug 04 17:52:00.248016 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.247983    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-kb4vj\""
Aug 04 17:52:00.248231 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.248218    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mdtdv"
Aug 04 17:52:00.249391 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.249365    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/436be5a7-f6ca-4507-94ca-d9c442879877-sys-fs\") pod \"aws-ebs-csi-driver-node-2j6h9\" (UID: \"436be5a7-f6ca-4507-94ca-d9c442879877\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9"
Aug 04 17:52:00.249590 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.249419    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmwgb\" (UniqueName: \"kubernetes.io/projected/fba423bd-0c4b-4b5c-851d-f604236048c3-kube-api-access-vmwgb\") pod \"network-check-target-5x9t4\" (UID: \"fba423bd-0c4b-4b5c-851d-f604236048c3\") " pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:52:00.249590 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.249591    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Aug 04 17:52:00.249767 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.249636    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-host-run-netns\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.249767 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.249684    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-run-systemd\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.249899 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.249812    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-etc-sysctl-conf\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.249899 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.249836    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Aug 04 17:52:00.249899 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.249842    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-var-lib-kubelet\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.250183 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.249933    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-host\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.250183 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.250072    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3159ec60-2d7d-4781-ad4a-f103dcbba781-tmp\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.250183 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.250153    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-etc-openvswitch\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.250323 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.250184    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-run-openvswitch\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.250323 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.250270    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Aug 04 17:52:00.250323 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.250305    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-run-ovn\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.250456 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.250341    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-host-cni-netd\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.250456 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.250396    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qbnb6\""
Aug 04 17:52:00.250560 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.250488    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-etc-sysctl-d\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.250637 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.250538    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5ee91876-14a5-4691-9f54-0d3d11cf70e3-dbus\") pod \"global-pull-secret-syncer-jgm5s\" (UID: \"5ee91876-14a5-4691-9f54-0d3d11cf70e3\") " pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:00.250691 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.250657    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5ee91876-14a5-4691-9f54-0d3d11cf70e3-original-pull-secret\") pod \"global-pull-secret-syncer-jgm5s\" (UID: \"5ee91876-14a5-4691-9f54-0d3d11cf70e3\") " pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:00.250743 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.250701    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-host-kubelet\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.250743 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.250725    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-host-run-ovn-kubernetes\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.250852 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.250753    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d5c319a9-83f7-4237-be47-bc28a033babe-ovn-node-metrics-cert\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.250852 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.250784    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d5c319a9-83f7-4237-be47-bc28a033babe-ovnkube-script-lib\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.250956 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.250913    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-systemd-units\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.251044 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.250976    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-host-slash\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.251092 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.251054    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-var-lib-openvswitch\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.251133 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.251094    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d5c319a9-83f7-4237-be47-bc28a033babe-env-overrides\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.251181 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.251141    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-etc-sysconfig\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.251181 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.251168    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-run\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.251282 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.251197    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/436be5a7-f6ca-4507-94ca-d9c442879877-socket-dir\") pod \"aws-ebs-csi-driver-node-2j6h9\" (UID: \"436be5a7-f6ca-4507-94ca-d9c442879877\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9"
Aug 04 17:52:00.251282 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.251255    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ac303603-7175-4c58-8989-bafad506bc15-agent-certs\") pod \"konnectivity-agent-qrkjb\" (UID: \"ac303603-7175-4c58-8989-bafad506bc15\") " pod="kube-system/konnectivity-agent-qrkjb"
Aug 04 17:52:00.251282 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.251278    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8j62\" (UniqueName: \"kubernetes.io/projected/3159ec60-2d7d-4781-ad4a-f103dcbba781-kube-api-access-q8j62\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.251439 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.251303    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0b696fd5-c22f-480a-a9bd-a2f8a19784ae-hosts-file\") pod \"node-resolver-lzkvj\" (UID: \"0b696fd5-c22f-480a-a9bd-a2f8a19784ae\") " pod="openshift-dns/node-resolver-lzkvj"
Aug 04 17:52:00.251439 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.251331    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/436be5a7-f6ca-4507-94ca-d9c442879877-device-dir\") pod \"aws-ebs-csi-driver-node-2j6h9\" (UID: \"436be5a7-f6ca-4507-94ca-d9c442879877\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9"
Aug 04 17:52:00.251439 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.251357    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d5c319a9-83f7-4237-be47-bc28a033babe-ovnkube-config\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.251439 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.251410    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-etc-modprobe-d\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.251439 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.251418    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Aug 04 17:52:00.251677 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.251440    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-sys\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.251677 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.251480    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b696fd5-c22f-480a-a9bd-a2f8a19784ae-tmp-dir\") pod \"node-resolver-lzkvj\" (UID: \"0b696fd5-c22f-480a-a9bd-a2f8a19784ae\") " pod="openshift-dns/node-resolver-lzkvj"
Aug 04 17:52:00.251677 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.251506    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/436be5a7-f6ca-4507-94ca-d9c442879877-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2j6h9\" (UID: \"436be5a7-f6ca-4507-94ca-d9c442879877\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9"
Aug 04 17:52:00.251677 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.251552    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/436be5a7-f6ca-4507-94ca-d9c442879877-etc-selinux\") pod \"aws-ebs-csi-driver-node-2j6h9\" (UID: \"436be5a7-f6ca-4507-94ca-d9c442879877\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9"
Aug 04 17:52:00.251677 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.251578    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4d2q\" (UniqueName: \"kubernetes.io/projected/436be5a7-f6ca-4507-94ca-d9c442879877-kube-api-access-n4d2q\") pod \"aws-ebs-csi-driver-node-2j6h9\" (UID: \"436be5a7-f6ca-4507-94ca-d9c442879877\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9"
Aug 04 17:52:00.251677 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.251598    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5ee91876-14a5-4691-9f54-0d3d11cf70e3-kubelet-config\") pod \"global-pull-secret-syncer-jgm5s\" (UID: \"5ee91876-14a5-4691-9f54-0d3d11cf70e3\") " pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:00.252275 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.251735    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ac303603-7175-4c58-8989-bafad506bc15-konnectivity-ca\") pod \"konnectivity-agent-qrkjb\" (UID: \"ac303603-7175-4c58-8989-bafad506bc15\") " pod="kube-system/konnectivity-agent-qrkjb"
Aug 04 17:52:00.252275 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.251757    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-node-log\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.252275 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.251802    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-log-socket\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.252275 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.251910    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-etc-kubernetes\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.252275 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.251928    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-etc-systemd\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.252275 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.251954    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-lib-modules\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.253082 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.252013    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3159ec60-2d7d-4781-ad4a-f103dcbba781-etc-tuned\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.253082 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.252667    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-host-cni-bin\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.253082 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.252724    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.253082 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.252385    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fjnbn"
Aug 04 17:52:00.253082 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.252761    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvd6j\" (UniqueName: \"kubernetes.io/projected/d5c319a9-83f7-4237-be47-bc28a033babe-kube-api-access-bvd6j\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.253082 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.252131    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Aug 04 17:52:00.253082 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.252828    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4ntj\" (UniqueName: \"kubernetes.io/projected/0b696fd5-c22f-480a-a9bd-a2f8a19784ae-kube-api-access-x4ntj\") pod \"node-resolver-lzkvj\" (UID: \"0b696fd5-c22f-480a-a9bd-a2f8a19784ae\") " pod="openshift-dns/node-resolver-lzkvj"
Aug 04 17:52:00.253434 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.253121    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Aug 04 17:52:00.253434 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.253250    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cxk77\""
Aug 04 17:52:00.253434 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.253200    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/436be5a7-f6ca-4507-94ca-d9c442879877-registration-dir\") pod \"aws-ebs-csi-driver-node-2j6h9\" (UID: \"436be5a7-f6ca-4507-94ca-d9c442879877\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9"
Aug 04 17:52:00.255274 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.255204    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bq2vk"
Aug 04 17:52:00.256234 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.255808    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Aug 04 17:52:00.256234 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.255849    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Aug 04 17:52:00.256234 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.255875    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fmq7d\""
Aug 04 17:52:00.256234 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.256104    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Aug 04 17:52:00.258390 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.258371    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Aug 04 17:52:00.258494 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.258443    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-82cwc\""
Aug 04 17:52:00.258649 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.258634    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Aug 04 17:52:00.258765 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.258672    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Aug 04 17:52:00.261719 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.261699    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 04 17:52:00.277079 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.277056    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-n8mtt"
Aug 04 17:52:00.288544 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.288526    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-n8mtt"
Aug 04 17:52:00.340898 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.340875    2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Aug 04 17:52:00.353578 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.353557    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45l97\" (UniqueName: \"kubernetes.io/projected/8941bb3a-0305-4d69-8a95-bee3e685acbb-kube-api-access-45l97\") pod \"multus-additional-cni-plugins-fjnbn\" (UID: \"8941bb3a-0305-4d69-8a95-bee3e685acbb\") " pod="openshift-multus/multus-additional-cni-plugins-fjnbn"
Aug 04 17:52:00.353690 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.353588    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-host\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.353690 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.353606    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3159ec60-2d7d-4781-ad4a-f103dcbba781-tmp\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.353690 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.353625    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2dnl\" (UniqueName: \"kubernetes.io/projected/86548434-a6dc-4663-a028-f2967885ba3e-kube-api-access-p2dnl\") pod \"iptables-alerter-bq2vk\" (UID: \"86548434-a6dc-4663-a028-f2967885ba3e\") " pod="openshift-network-operator/iptables-alerter-bq2vk"
Aug 04 17:52:00.353690 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.353652    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8941bb3a-0305-4d69-8a95-bee3e685acbb-cni-binary-copy\") pod \"multus-additional-cni-plugins-fjnbn\" (UID: \"8941bb3a-0305-4d69-8a95-bee3e685acbb\") " pod="openshift-multus/multus-additional-cni-plugins-fjnbn"
Aug 04 17:52:00.353690 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.353676    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-etc-openvswitch\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.353690 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.353683    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-host\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.354109 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.353700    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-run-openvswitch\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.354109 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.353751    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-etc-openvswitch\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.354109 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.353771    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-run-openvswitch\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.354109 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.353776    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-host-cni-netd\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.354109 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.353812    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-host-cni-netd\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.354109 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.353882    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8941bb3a-0305-4d69-8a95-bee3e685acbb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fjnbn\" (UID: \"8941bb3a-0305-4d69-8a95-bee3e685acbb\") " pod="openshift-multus/multus-additional-cni-plugins-fjnbn"
Aug 04 17:52:00.354109 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.353930    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5ee91876-14a5-4691-9f54-0d3d11cf70e3-dbus\") pod \"global-pull-secret-syncer-jgm5s\" (UID: \"5ee91876-14a5-4691-9f54-0d3d11cf70e3\") " pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:00.354109 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.353971    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-host-kubelet\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.354109 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354018    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d5c319a9-83f7-4237-be47-bc28a033babe-ovnkube-script-lib\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.354670 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354129    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-etc-sysconfig\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.354670 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354147    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-host-kubelet\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.354670 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354162    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-run\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.354670 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354192    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/436be5a7-f6ca-4507-94ca-d9c442879877-socket-dir\") pod \"aws-ebs-csi-driver-node-2j6h9\" (UID: \"436be5a7-f6ca-4507-94ca-d9c442879877\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9"
Aug 04 17:52:00.354670 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354198    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5ee91876-14a5-4691-9f54-0d3d11cf70e3-dbus\") pod \"global-pull-secret-syncer-jgm5s\" (UID: \"5ee91876-14a5-4691-9f54-0d3d11cf70e3\") " pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:00.354670 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354214    2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Aug 04 17:52:00.354670 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354315    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/436be5a7-f6ca-4507-94ca-d9c442879877-socket-dir\") pod \"aws-ebs-csi-driver-node-2j6h9\" (UID: \"436be5a7-f6ca-4507-94ca-d9c442879877\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9"
Aug 04 17:52:00.354670 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354384    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-run\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.354670 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354384    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-etc-sysconfig\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.354670 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354411    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-multus-cni-dir\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.354670 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354437    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-os-release\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.354670 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354475    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-systemd-units\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.354670 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354491    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-var-lib-openvswitch\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.354670 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354506    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-multus-daemon-config\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.354670 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354523    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a4441318-a230-45e7-8ab5-bf4049533e35-serviceca\") pod \"node-ca-mdtdv\" (UID: \"a4441318-a230-45e7-8ab5-bf4049533e35\") " pod="openshift-image-registry/node-ca-mdtdv"
Aug 04 17:52:00.354670 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354545    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8941bb3a-0305-4d69-8a95-bee3e685acbb-system-cni-dir\") pod \"multus-additional-cni-plugins-fjnbn\" (UID: \"8941bb3a-0305-4d69-8a95-bee3e685acbb\") " pod="openshift-multus/multus-additional-cni-plugins-fjnbn"
Aug 04 17:52:00.354670 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354639    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ac303603-7175-4c58-8989-bafad506bc15-agent-certs\") pod \"konnectivity-agent-qrkjb\" (UID: \"ac303603-7175-4c58-8989-bafad506bc15\") " pod="kube-system/konnectivity-agent-qrkjb"
Aug 04 17:52:00.354670 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354665    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8j62\" (UniqueName: \"kubernetes.io/projected/3159ec60-2d7d-4781-ad4a-f103dcbba781-kube-api-access-q8j62\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.355607 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354691    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0b696fd5-c22f-480a-a9bd-a2f8a19784ae-hosts-file\") pod \"node-resolver-lzkvj\" (UID: \"0b696fd5-c22f-480a-a9bd-a2f8a19784ae\") " pod="openshift-dns/node-resolver-lzkvj"
Aug 04 17:52:00.355607 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354714    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/436be5a7-f6ca-4507-94ca-d9c442879877-device-dir\") pod \"aws-ebs-csi-driver-node-2j6h9\" (UID: \"436be5a7-f6ca-4507-94ca-d9c442879877\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9"
Aug 04 17:52:00.355607 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354719    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-systemd-units\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.355607 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354769    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/436be5a7-f6ca-4507-94ca-d9c442879877-device-dir\") pod \"aws-ebs-csi-driver-node-2j6h9\" (UID: \"436be5a7-f6ca-4507-94ca-d9c442879877\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9"
Aug 04 17:52:00.355607 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354763    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-var-lib-openvswitch\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.355607 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354796    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-host-var-lib-kubelet\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.355607 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354841    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a4441318-a230-45e7-8ab5-bf4049533e35-host\") pod \"node-ca-mdtdv\" (UID: \"a4441318-a230-45e7-8ab5-bf4049533e35\") " pod="openshift-image-registry/node-ca-mdtdv"
Aug 04 17:52:00.355607 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354839    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d5c319a9-83f7-4237-be47-bc28a033babe-ovnkube-script-lib\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.355607 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354878    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b581e21-8a6f-4ed7-8751-87941a6cbac3-metrics-certs\") pod \"network-metrics-daemon-sstpm\" (UID: \"6b581e21-8a6f-4ed7-8751-87941a6cbac3\") " pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:00.355607 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354913    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8941bb3a-0305-4d69-8a95-bee3e685acbb-cnibin\") pod \"multus-additional-cni-plugins-fjnbn\" (UID: \"8941bb3a-0305-4d69-8a95-bee3e685acbb\") " pod="openshift-multus/multus-additional-cni-plugins-fjnbn"
Aug 04 17:52:00.355607 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354938    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d5c319a9-83f7-4237-be47-bc28a033babe-ovnkube-config\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.355607 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.354979    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-etc-modprobe-d\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.355607 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.355017    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b696fd5-c22f-480a-a9bd-a2f8a19784ae-tmp-dir\") pod \"node-resolver-lzkvj\" (UID: \"0b696fd5-c22f-480a-a9bd-a2f8a19784ae\") " pod="openshift-dns/node-resolver-lzkvj"
Aug 04 17:52:00.355607 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.355059    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/436be5a7-f6ca-4507-94ca-d9c442879877-etc-selinux\") pod \"aws-ebs-csi-driver-node-2j6h9\" (UID: \"436be5a7-f6ca-4507-94ca-d9c442879877\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9"
Aug 04 17:52:00.355607 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.355083    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4d2q\" (UniqueName: \"kubernetes.io/projected/436be5a7-f6ca-4507-94ca-d9c442879877-kube-api-access-n4d2q\") pod \"aws-ebs-csi-driver-node-2j6h9\" (UID: \"436be5a7-f6ca-4507-94ca-d9c442879877\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9"
Aug 04 17:52:00.355607 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.355108    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5ee91876-14a5-4691-9f54-0d3d11cf70e3-kubelet-config\") pod \"global-pull-secret-syncer-jgm5s\" (UID: \"5ee91876-14a5-4691-9f54-0d3d11cf70e3\") " pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:00.355607 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.355182    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0b696fd5-c22f-480a-a9bd-a2f8a19784ae-hosts-file\") pod \"node-resolver-lzkvj\" (UID: \"0b696fd5-c22f-480a-a9bd-a2f8a19784ae\") " pod="openshift-dns/node-resolver-lzkvj"
Aug 04 17:52:00.356420 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.355160    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ac303603-7175-4c58-8989-bafad506bc15-konnectivity-ca\") pod \"konnectivity-agent-qrkjb\" (UID: \"ac303603-7175-4c58-8989-bafad506bc15\") " pod="kube-system/konnectivity-agent-qrkjb"
Aug 04 17:52:00.356420 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.355333    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-etc-systemd\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.356420 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.355437    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-lib-modules\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.356420 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.355465    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/436be5a7-f6ca-4507-94ca-d9c442879877-registration-dir\") pod \"aws-ebs-csi-driver-node-2j6h9\" (UID: \"436be5a7-f6ca-4507-94ca-d9c442879877\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9"
Aug 04 17:52:00.356420 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.355602    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-host-var-lib-cni-multus\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.356420 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.355690    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-etc-modprobe-d\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.356420 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.355704    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-etc-kubernetes\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.356420 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.355730    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgddm\" (UniqueName: \"kubernetes.io/projected/a4441318-a230-45e7-8ab5-bf4049533e35-kube-api-access-fgddm\") pod \"node-ca-mdtdv\" (UID: \"a4441318-a230-45e7-8ab5-bf4049533e35\") " pod="openshift-image-registry/node-ca-mdtdv"
Aug 04 17:52:00.356420 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.355755    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-host-cni-bin\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.356420 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.355776    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/436be5a7-f6ca-4507-94ca-d9c442879877-etc-selinux\") pod \"aws-ebs-csi-driver-node-2j6h9\" (UID: \"436be5a7-f6ca-4507-94ca-d9c442879877\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9"
Aug 04 17:52:00.356420 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.355780    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.356420 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.355845    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-etc-systemd\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.356420 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.355885    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-system-cni-dir\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.356420 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.355894    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ac303603-7175-4c58-8989-bafad506bc15-konnectivity-ca\") pod \"konnectivity-agent-qrkjb\" (UID: \"ac303603-7175-4c58-8989-bafad506bc15\") " pod="kube-system/konnectivity-agent-qrkjb"
Aug 04 17:52:00.356420 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.355910    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tbwh\" (UniqueName: \"kubernetes.io/projected/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-kube-api-access-7tbwh\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.356420 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.355936    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8941bb3a-0305-4d69-8a95-bee3e685acbb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fjnbn\" (UID: \"8941bb3a-0305-4d69-8a95-bee3e685acbb\") " pod="openshift-multus/multus-additional-cni-plugins-fjnbn"
Aug 04 17:52:00.356420 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.355967    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmwgb\" (UniqueName: \"kubernetes.io/projected/fba423bd-0c4b-4b5c-851d-f604236048c3-kube-api-access-vmwgb\") pod \"network-check-target-5x9t4\" (UID: \"fba423bd-0c4b-4b5c-851d-f604236048c3\") " pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:52:00.357027 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356015    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-host-run-netns\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.357027 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356041    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-run-systemd\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.357027 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356055    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-lib-modules\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.357027 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356081    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-etc-sysctl-conf\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.357027 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.355936    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.357027 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356145    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/436be5a7-f6ca-4507-94ca-d9c442879877-registration-dir\") pod \"aws-ebs-csi-driver-node-2j6h9\" (UID: \"436be5a7-f6ca-4507-94ca-d9c442879877\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9"
Aug 04 17:52:00.357027 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356197    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5ee91876-14a5-4691-9f54-0d3d11cf70e3-kubelet-config\") pod \"global-pull-secret-syncer-jgm5s\" (UID: \"5ee91876-14a5-4691-9f54-0d3d11cf70e3\") " pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:00.357027 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356248    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-host-run-netns\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.357027 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356293    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-host-cni-bin\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.357027 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356338    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-run-systemd\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.357027 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356392    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-var-lib-kubelet\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.357027 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356465    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrqb9\" (UniqueName: \"kubernetes.io/projected/6b581e21-8a6f-4ed7-8751-87941a6cbac3-kube-api-access-wrqb9\") pod \"network-metrics-daemon-sstpm\" (UID: \"6b581e21-8a6f-4ed7-8751-87941a6cbac3\") " pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:00.357027 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356472    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-etc-sysctl-conf\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.357027 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356508    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8941bb3a-0305-4d69-8a95-bee3e685acbb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fjnbn\" (UID: \"8941bb3a-0305-4d69-8a95-bee3e685acbb\") " pod="openshift-multus/multus-additional-cni-plugins-fjnbn"
Aug 04 17:52:00.357027 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356555    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-run-ovn\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.357027 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356594    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-var-lib-kubelet\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.357027 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356618    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-etc-sysctl-d\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.357733 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356660    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-multus-socket-dir-parent\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.357733 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356700    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-host-run-netns\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.357733 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356727    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-run-ovn\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.357733 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356736    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d5c319a9-83f7-4237-be47-bc28a033babe-ovnkube-config\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.357733 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356742    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/86548434-a6dc-4663-a028-f2967885ba3e-iptables-alerter-script\") pod \"iptables-alerter-bq2vk\" (UID: \"86548434-a6dc-4663-a028-f2967885ba3e\") " pod="openshift-network-operator/iptables-alerter-bq2vk"
Aug 04 17:52:00.357733 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356788    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5ee91876-14a5-4691-9f54-0d3d11cf70e3-original-pull-secret\") pod \"global-pull-secret-syncer-jgm5s\" (UID: \"5ee91876-14a5-4691-9f54-0d3d11cf70e3\") " pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:00.357733 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356812    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-etc-sysctl-d\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.357733 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356815    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-host-run-ovn-kubernetes\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.357733 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356855    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d5c319a9-83f7-4237-be47-bc28a033babe-ovn-node-metrics-cert\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.357733 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356885    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-cni-binary-copy\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.357733 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356887    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-host-run-ovn-kubernetes\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.357733 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356911    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-host-slash\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.357733 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356950    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d5c319a9-83f7-4237-be47-bc28a033babe-env-overrides\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.357733 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.356971    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b696fd5-c22f-480a-a9bd-a2f8a19784ae-tmp-dir\") pod \"node-resolver-lzkvj\" (UID: \"0b696fd5-c22f-480a-a9bd-a2f8a19784ae\") " pod="openshift-dns/node-resolver-lzkvj"
Aug 04 17:52:00.357733 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.357032    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-multus-conf-dir\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.357733 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.357095    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-host-run-multus-certs\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.357733 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.357122    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/86548434-a6dc-4663-a028-f2967885ba3e-host-slash\") pod \"iptables-alerter-bq2vk\" (UID: \"86548434-a6dc-4663-a028-f2967885ba3e\") " pod="openshift-network-operator/iptables-alerter-bq2vk"
Aug 04 17:52:00.358698 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.357147    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-sys\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.358698 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.357193    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/436be5a7-f6ca-4507-94ca-d9c442879877-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2j6h9\" (UID: \"436be5a7-f6ca-4507-94ca-d9c442879877\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9"
Aug 04 17:52:00.358698 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.357236    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-hostroot\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.358698 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.357271    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-node-log\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.358698 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.357299    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-log-socket\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.358698 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.357352    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-etc-kubernetes\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.358698 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.357379    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d5c319a9-83f7-4237-be47-bc28a033babe-env-overrides\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.358698 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.357384    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-sys\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.358698 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.357411    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3159ec60-2d7d-4781-ad4a-f103dcbba781-etc-tuned\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.358698 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.357431    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3159ec60-2d7d-4781-ad4a-f103dcbba781-etc-kubernetes\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.358698 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.357446    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-host-run-k8s-cni-cncf-io\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.358698 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.357508    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-host-var-lib-cni-bin\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.358698 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.357538    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvd6j\" (UniqueName: \"kubernetes.io/projected/d5c319a9-83f7-4237-be47-bc28a033babe-kube-api-access-bvd6j\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.358698 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.357596    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-172.ec2.internal" event={"ID":"3428ebcd2d197fe8677d9367742cfcaa","Type":"ContainerStarted","Data":"1837e021c56dd856a107b6f81ac54cdabbb068ae10b8dfa4e160d31d7dd2d5f5"}
Aug 04 17:52:00.358698 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.357697    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-log-socket\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.358698 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.357770    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/436be5a7-f6ca-4507-94ca-d9c442879877-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2j6h9\" (UID: \"436be5a7-f6ca-4507-94ca-d9c442879877\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9"
Aug 04 17:52:00.358698 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.357803    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-node-log\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.358698 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.357568    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4ntj\" (UniqueName: \"kubernetes.io/projected/0b696fd5-c22f-480a-a9bd-a2f8a19784ae-kube-api-access-x4ntj\") pod \"node-resolver-lzkvj\" (UID: \"0b696fd5-c22f-480a-a9bd-a2f8a19784ae\") " pod="openshift-dns/node-resolver-lzkvj"
Aug 04 17:52:00.359610 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.358065    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/436be5a7-f6ca-4507-94ca-d9c442879877-sys-fs\") pod \"aws-ebs-csi-driver-node-2j6h9\" (UID: \"436be5a7-f6ca-4507-94ca-d9c442879877\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9"
Aug 04 17:52:00.359610 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.358111    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-cnibin\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.359610 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.358217    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8941bb3a-0305-4d69-8a95-bee3e685acbb-os-release\") pod \"multus-additional-cni-plugins-fjnbn\" (UID: \"8941bb3a-0305-4d69-8a95-bee3e685acbb\") " pod="openshift-multus/multus-additional-cni-plugins-fjnbn"
Aug 04 17:52:00.359610 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.358289    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/436be5a7-f6ca-4507-94ca-d9c442879877-sys-fs\") pod \"aws-ebs-csi-driver-node-2j6h9\" (UID: \"436be5a7-f6ca-4507-94ca-d9c442879877\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9"
Aug 04 17:52:00.359610 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.358345    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d5c319a9-83f7-4237-be47-bc28a033babe-host-slash\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.359610 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.358375    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3159ec60-2d7d-4781-ad4a-f103dcbba781-tmp\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.359610 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:00.358477    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 17:52:00.359610 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:00.358540    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ee91876-14a5-4691-9f54-0d3d11cf70e3-original-pull-secret podName:5ee91876-14a5-4691-9f54-0d3d11cf70e3 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:00.85851884 +0000 UTC m=+3.041679007 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5ee91876-14a5-4691-9f54-0d3d11cf70e3-original-pull-secret") pod "global-pull-secret-syncer-jgm5s" (UID: "5ee91876-14a5-4691-9f54-0d3d11cf70e3") : object "kube-system"/"original-pull-secret" not registered
Aug 04 17:52:00.359610 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.359177    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ac303603-7175-4c58-8989-bafad506bc15-agent-certs\") pod \"konnectivity-agent-qrkjb\" (UID: \"ac303603-7175-4c58-8989-bafad506bc15\") " pod="kube-system/konnectivity-agent-qrkjb"
Aug 04 17:52:00.360451 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.360430    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-172.ec2.internal" event={"ID":"fbe04a653517381697f35b4a5ee6bb15","Type":"ContainerStarted","Data":"b94219d22b02203687a54372f12c03c1683c794f01d1a8df81cc883f970f2295"}
Aug 04 17:52:00.360532 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.360449    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3159ec60-2d7d-4781-ad4a-f103dcbba781-etc-tuned\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.360652 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.360539    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d5c319a9-83f7-4237-be47-bc28a033babe-ovn-node-metrics-cert\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.365687 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.365667    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8j62\" (UniqueName: \"kubernetes.io/projected/3159ec60-2d7d-4781-ad4a-f103dcbba781-kube-api-access-q8j62\") pod \"tuned-c8zhn\" (UID: \"3159ec60-2d7d-4781-ad4a-f103dcbba781\") " pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.370202 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:00.370126    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 17:52:00.370202 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:00.370151    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 17:52:00.370202 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:00.370165    2560 projected.go:196] Error preparing data for projected volume kube-api-access-vmwgb for pod openshift-network-diagnostics/network-check-target-5x9t4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 17:52:00.370399 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:00.370334    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fba423bd-0c4b-4b5c-851d-f604236048c3-kube-api-access-vmwgb podName:fba423bd-0c4b-4b5c-851d-f604236048c3 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:00.870277142 +0000 UTC m=+3.053437319 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vmwgb" (UniqueName: "kubernetes.io/projected/fba423bd-0c4b-4b5c-851d-f604236048c3-kube-api-access-vmwgb") pod "network-check-target-5x9t4" (UID: "fba423bd-0c4b-4b5c-851d-f604236048c3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 17:52:00.371056 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.371038    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4d2q\" (UniqueName: \"kubernetes.io/projected/436be5a7-f6ca-4507-94ca-d9c442879877-kube-api-access-n4d2q\") pod \"aws-ebs-csi-driver-node-2j6h9\" (UID: \"436be5a7-f6ca-4507-94ca-d9c442879877\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9"
Aug 04 17:52:00.372176 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.372151    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4ntj\" (UniqueName: \"kubernetes.io/projected/0b696fd5-c22f-480a-a9bd-a2f8a19784ae-kube-api-access-x4ntj\") pod \"node-resolver-lzkvj\" (UID: \"0b696fd5-c22f-480a-a9bd-a2f8a19784ae\") " pod="openshift-dns/node-resolver-lzkvj"
Aug 04 17:52:00.372266 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.372155    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvd6j\" (UniqueName: \"kubernetes.io/projected/d5c319a9-83f7-4237-be47-bc28a033babe-kube-api-access-bvd6j\") pod \"ovnkube-node-2wj5g\" (UID: \"d5c319a9-83f7-4237-be47-bc28a033babe\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.459214 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459181    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-multus-conf-dir\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.459328 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459222    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-host-run-multus-certs\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.459328 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459243    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/86548434-a6dc-4663-a028-f2967885ba3e-host-slash\") pod \"iptables-alerter-bq2vk\" (UID: \"86548434-a6dc-4663-a028-f2967885ba3e\") " pod="openshift-network-operator/iptables-alerter-bq2vk"
Aug 04 17:52:00.459328 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459265    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-hostroot\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.459328 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459288    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-host-run-k8s-cni-cncf-io\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.459328 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459307    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-host-var-lib-cni-bin\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.459590 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459331    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-cnibin\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.459590 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459352    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8941bb3a-0305-4d69-8a95-bee3e685acbb-os-release\") pod \"multus-additional-cni-plugins-fjnbn\" (UID: \"8941bb3a-0305-4d69-8a95-bee3e685acbb\") " pod="openshift-multus/multus-additional-cni-plugins-fjnbn"
Aug 04 17:52:00.459590 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459375    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45l97\" (UniqueName: \"kubernetes.io/projected/8941bb3a-0305-4d69-8a95-bee3e685acbb-kube-api-access-45l97\") pod \"multus-additional-cni-plugins-fjnbn\" (UID: \"8941bb3a-0305-4d69-8a95-bee3e685acbb\") " pod="openshift-multus/multus-additional-cni-plugins-fjnbn"
Aug 04 17:52:00.459590 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459400    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2dnl\" (UniqueName: \"kubernetes.io/projected/86548434-a6dc-4663-a028-f2967885ba3e-kube-api-access-p2dnl\") pod \"iptables-alerter-bq2vk\" (UID: \"86548434-a6dc-4663-a028-f2967885ba3e\") " pod="openshift-network-operator/iptables-alerter-bq2vk"
Aug 04 17:52:00.459590 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459423    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8941bb3a-0305-4d69-8a95-bee3e685acbb-cni-binary-copy\") pod \"multus-additional-cni-plugins-fjnbn\" (UID: \"8941bb3a-0305-4d69-8a95-bee3e685acbb\") " pod="openshift-multus/multus-additional-cni-plugins-fjnbn"
Aug 04 17:52:00.459590 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459450    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8941bb3a-0305-4d69-8a95-bee3e685acbb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fjnbn\" (UID: \"8941bb3a-0305-4d69-8a95-bee3e685acbb\") " pod="openshift-multus/multus-additional-cni-plugins-fjnbn"
Aug 04 17:52:00.459590 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459481    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-multus-cni-dir\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.459590 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459503    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-os-release\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.459590 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459547    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-multus-daemon-config\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.459590 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459571    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a4441318-a230-45e7-8ab5-bf4049533e35-serviceca\") pod \"node-ca-mdtdv\" (UID: \"a4441318-a230-45e7-8ab5-bf4049533e35\") " pod="openshift-image-registry/node-ca-mdtdv"
Aug 04 17:52:00.460090 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459594    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8941bb3a-0305-4d69-8a95-bee3e685acbb-system-cni-dir\") pod \"multus-additional-cni-plugins-fjnbn\" (UID: \"8941bb3a-0305-4d69-8a95-bee3e685acbb\") " pod="openshift-multus/multus-additional-cni-plugins-fjnbn"
Aug 04 17:52:00.460090 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459622    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-host-var-lib-kubelet\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.460090 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459645    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a4441318-a230-45e7-8ab5-bf4049533e35-host\") pod \"node-ca-mdtdv\" (UID: \"a4441318-a230-45e7-8ab5-bf4049533e35\") " pod="openshift-image-registry/node-ca-mdtdv"
Aug 04 17:52:00.460090 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459668    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b581e21-8a6f-4ed7-8751-87941a6cbac3-metrics-certs\") pod \"network-metrics-daemon-sstpm\" (UID: \"6b581e21-8a6f-4ed7-8751-87941a6cbac3\") " pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:00.460090 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459692    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8941bb3a-0305-4d69-8a95-bee3e685acbb-cnibin\") pod \"multus-additional-cni-plugins-fjnbn\" (UID: \"8941bb3a-0305-4d69-8a95-bee3e685acbb\") " pod="openshift-multus/multus-additional-cni-plugins-fjnbn"
Aug 04 17:52:00.460090 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459725    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-host-var-lib-cni-multus\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.460090 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459749    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-etc-kubernetes\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.460090 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459774    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgddm\" (UniqueName: \"kubernetes.io/projected/a4441318-a230-45e7-8ab5-bf4049533e35-kube-api-access-fgddm\") pod \"node-ca-mdtdv\" (UID: \"a4441318-a230-45e7-8ab5-bf4049533e35\") " pod="openshift-image-registry/node-ca-mdtdv"
Aug 04 17:52:00.460090 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459799    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-system-cni-dir\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.460090 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459823    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tbwh\" (UniqueName: \"kubernetes.io/projected/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-kube-api-access-7tbwh\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.460090 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459848    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8941bb3a-0305-4d69-8a95-bee3e685acbb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fjnbn\" (UID: \"8941bb3a-0305-4d69-8a95-bee3e685acbb\") " pod="openshift-multus/multus-additional-cni-plugins-fjnbn"
Aug 04 17:52:00.460090 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459892    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrqb9\" (UniqueName: \"kubernetes.io/projected/6b581e21-8a6f-4ed7-8751-87941a6cbac3-kube-api-access-wrqb9\") pod \"network-metrics-daemon-sstpm\" (UID: \"6b581e21-8a6f-4ed7-8751-87941a6cbac3\") " pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:00.460090 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459928    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8941bb3a-0305-4d69-8a95-bee3e685acbb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fjnbn\" (UID: \"8941bb3a-0305-4d69-8a95-bee3e685acbb\") " pod="openshift-multus/multus-additional-cni-plugins-fjnbn"
Aug 04 17:52:00.460090 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459956    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-multus-socket-dir-parent\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.460090 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.459981    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-host-run-netns\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.460090 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.460025    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/86548434-a6dc-4663-a028-f2967885ba3e-iptables-alerter-script\") pod \"iptables-alerter-bq2vk\" (UID: \"86548434-a6dc-4663-a028-f2967885ba3e\") " pod="openshift-network-operator/iptables-alerter-bq2vk"
Aug 04 17:52:00.460090 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.460067    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-cni-binary-copy\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.460883 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.460659    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-cni-binary-copy\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.460883 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.460726    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-multus-conf-dir\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.460883 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.460765    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-host-run-multus-certs\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.460883 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.460803    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/86548434-a6dc-4663-a028-f2967885ba3e-host-slash\") pod \"iptables-alerter-bq2vk\" (UID: \"86548434-a6dc-4663-a028-f2967885ba3e\") " pod="openshift-network-operator/iptables-alerter-bq2vk"
Aug 04 17:52:00.460883 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.460839    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-hostroot\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.460883 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.460876    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-host-run-k8s-cni-cncf-io\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.461193 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.460910    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-host-var-lib-cni-bin\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.461193 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.460955    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-cnibin\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.461193 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.461030    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8941bb3a-0305-4d69-8a95-bee3e685acbb-os-release\") pod \"multus-additional-cni-plugins-fjnbn\" (UID: \"8941bb3a-0305-4d69-8a95-bee3e685acbb\") " pod="openshift-multus/multus-additional-cni-plugins-fjnbn"
Aug 04 17:52:00.461762 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.461742    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8941bb3a-0305-4d69-8a95-bee3e685acbb-cni-binary-copy\") pod \"multus-additional-cni-plugins-fjnbn\" (UID: \"8941bb3a-0305-4d69-8a95-bee3e685acbb\") " pod="openshift-multus/multus-additional-cni-plugins-fjnbn"
Aug 04 17:52:00.464132 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.462163    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8941bb3a-0305-4d69-8a95-bee3e685acbb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fjnbn\" (UID: \"8941bb3a-0305-4d69-8a95-bee3e685acbb\") " pod="openshift-multus/multus-additional-cni-plugins-fjnbn"
Aug 04 17:52:00.464132 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.462238    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-multus-cni-dir\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.464132 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.462305    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-os-release\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.464132 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.462763    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-multus-daemon-config\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.464132 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.462841    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-system-cni-dir\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.464132 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.463069    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a4441318-a230-45e7-8ab5-bf4049533e35-serviceca\") pod \"node-ca-mdtdv\" (UID: \"a4441318-a230-45e7-8ab5-bf4049533e35\") " pod="openshift-image-registry/node-ca-mdtdv"
Aug 04 17:52:00.464132 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.463126    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8941bb3a-0305-4d69-8a95-bee3e685acbb-system-cni-dir\") pod \"multus-additional-cni-plugins-fjnbn\" (UID: \"8941bb3a-0305-4d69-8a95-bee3e685acbb\") " pod="openshift-multus/multus-additional-cni-plugins-fjnbn"
Aug 04 17:52:00.464132 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.463184    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-host-var-lib-kubelet\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.464132 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.463184    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8941bb3a-0305-4d69-8a95-bee3e685acbb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fjnbn\" (UID: \"8941bb3a-0305-4d69-8a95-bee3e685acbb\") " pod="openshift-multus/multus-additional-cni-plugins-fjnbn"
Aug 04 17:52:00.464132 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.463235    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a4441318-a230-45e7-8ab5-bf4049533e35-host\") pod \"node-ca-mdtdv\" (UID: \"a4441318-a230-45e7-8ab5-bf4049533e35\") " pod="openshift-image-registry/node-ca-mdtdv"
Aug 04 17:52:00.464132 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.463250    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-multus-socket-dir-parent\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.464132 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.463303    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-host-run-netns\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.464132 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:00.463459    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 17:52:00.464132 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:00.463508    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b581e21-8a6f-4ed7-8751-87941a6cbac3-metrics-certs podName:6b581e21-8a6f-4ed7-8751-87941a6cbac3 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:00.963492395 +0000 UTC m=+3.146652551 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6b581e21-8a6f-4ed7-8751-87941a6cbac3-metrics-certs") pod "network-metrics-daemon-sstpm" (UID: "6b581e21-8a6f-4ed7-8751-87941a6cbac3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 17:52:00.464132 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.463667    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8941bb3a-0305-4d69-8a95-bee3e685acbb-cnibin\") pod \"multus-additional-cni-plugins-fjnbn\" (UID: \"8941bb3a-0305-4d69-8a95-bee3e685acbb\") " pod="openshift-multus/multus-additional-cni-plugins-fjnbn"
Aug 04 17:52:00.464132 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.463715    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-host-var-lib-cni-multus\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.464132 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.463759    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-etc-kubernetes\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.464715 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.463768    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/86548434-a6dc-4663-a028-f2967885ba3e-iptables-alerter-script\") pod \"iptables-alerter-bq2vk\" (UID: \"86548434-a6dc-4663-a028-f2967885ba3e\") " pod="openshift-network-operator/iptables-alerter-bq2vk"
Aug 04 17:52:00.464715 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.464107    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8941bb3a-0305-4d69-8a95-bee3e685acbb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fjnbn\" (UID: \"8941bb3a-0305-4d69-8a95-bee3e685acbb\") " pod="openshift-multus/multus-additional-cni-plugins-fjnbn"
Aug 04 17:52:00.472495 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.472443    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45l97\" (UniqueName: \"kubernetes.io/projected/8941bb3a-0305-4d69-8a95-bee3e685acbb-kube-api-access-45l97\") pod \"multus-additional-cni-plugins-fjnbn\" (UID: \"8941bb3a-0305-4d69-8a95-bee3e685acbb\") " pod="openshift-multus/multus-additional-cni-plugins-fjnbn"
Aug 04 17:52:00.472669 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.472645    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2dnl\" (UniqueName: \"kubernetes.io/projected/86548434-a6dc-4663-a028-f2967885ba3e-kube-api-access-p2dnl\") pod \"iptables-alerter-bq2vk\" (UID: \"86548434-a6dc-4663-a028-f2967885ba3e\") " pod="openshift-network-operator/iptables-alerter-bq2vk"
Aug 04 17:52:00.472749 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.472665    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgddm\" (UniqueName: \"kubernetes.io/projected/a4441318-a230-45e7-8ab5-bf4049533e35-kube-api-access-fgddm\") pod \"node-ca-mdtdv\" (UID: \"a4441318-a230-45e7-8ab5-bf4049533e35\") " pod="openshift-image-registry/node-ca-mdtdv"
Aug 04 17:52:00.472909 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.472870    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrqb9\" (UniqueName: \"kubernetes.io/projected/6b581e21-8a6f-4ed7-8751-87941a6cbac3-kube-api-access-wrqb9\") pod \"network-metrics-daemon-sstpm\" (UID: \"6b581e21-8a6f-4ed7-8751-87941a6cbac3\") " pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:00.474043 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.474023    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tbwh\" (UniqueName: \"kubernetes.io/projected/cb7cece1-6af0-4189-b5e0-3b5db5c0ca56-kube-api-access-7tbwh\") pod \"multus-f2zqg\" (UID: \"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56\") " pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.535936 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.535911    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9"
Aug 04 17:52:00.543385 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.543220    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:00.554834 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.554815    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qrkjb"
Aug 04 17:52:00.558678 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.558516    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-c8zhn"
Aug 04 17:52:00.564276 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.564132    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lzkvj"
Aug 04 17:52:00.569605 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.569583    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f2zqg"
Aug 04 17:52:00.574560 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:52:00.574514    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b696fd5_c22f_480a_a9bd_a2f8a19784ae.slice/crio-d5a96885491ed3007eafe4b3ed9ccce2ed7710a3eabb4d03e6293297c854747f WatchSource:0}: Error finding container d5a96885491ed3007eafe4b3ed9ccce2ed7710a3eabb4d03e6293297c854747f: Status 404 returned error can't find the container with id d5a96885491ed3007eafe4b3ed9ccce2ed7710a3eabb4d03e6293297c854747f
Aug 04 17:52:00.575435 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.575416    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mdtdv"
Aug 04 17:52:00.575543 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:52:00.575454    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac303603_7175_4c58_8989_bafad506bc15.slice/crio-c0a47c5dee42b70d96a29da12f775208cc6d0a0062c288a806ee5628b50ee9fc WatchSource:0}: Error finding container c0a47c5dee42b70d96a29da12f775208cc6d0a0062c288a806ee5628b50ee9fc: Status 404 returned error can't find the container with id c0a47c5dee42b70d96a29da12f775208cc6d0a0062c288a806ee5628b50ee9fc
Aug 04 17:52:00.577713 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:52:00.577674    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3159ec60_2d7d_4781_ad4a_f103dcbba781.slice/crio-9ad9554e4d58cea3b6682c0b6c51fdb45d4db5525b23dc6165be99f39566cd15 WatchSource:0}: Error finding container 9ad9554e4d58cea3b6682c0b6c51fdb45d4db5525b23dc6165be99f39566cd15: Status 404 returned error can't find the container with id 9ad9554e4d58cea3b6682c0b6c51fdb45d4db5525b23dc6165be99f39566cd15
Aug 04 17:52:00.582439 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:52:00.582420    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb7cece1_6af0_4189_b5e0_3b5db5c0ca56.slice/crio-e20725e70d7ac49b9de8b58126863f0a0d9724eb3a0c552cbddb64d94f1f9abe WatchSource:0}: Error finding container e20725e70d7ac49b9de8b58126863f0a0d9724eb3a0c552cbddb64d94f1f9abe: Status 404 returned error can't find the container with id e20725e70d7ac49b9de8b58126863f0a0d9724eb3a0c552cbddb64d94f1f9abe
Aug 04 17:52:00.584271 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.584250    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fjnbn"
Aug 04 17:52:00.587754 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:52:00.587734    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4441318_a230_45e7_8ab5_bf4049533e35.slice/crio-f2c2eb18869a74b0fdc9d5e67bbff989b64204cded76ccd82635fa5085cd7c75 WatchSource:0}: Error finding container f2c2eb18869a74b0fdc9d5e67bbff989b64204cded76ccd82635fa5085cd7c75: Status 404 returned error can't find the container with id f2c2eb18869a74b0fdc9d5e67bbff989b64204cded76ccd82635fa5085cd7c75
Aug 04 17:52:00.588608 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.588586    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bq2vk"
Aug 04 17:52:00.597144 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:52:00.597120    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8941bb3a_0305_4d69_8a95_bee3e685acbb.slice/crio-c66257ea592b791fa2bc866f388f9278f9e0a7990cac0c86d71050932f47190a WatchSource:0}: Error finding container c66257ea592b791fa2bc866f388f9278f9e0a7990cac0c86d71050932f47190a: Status 404 returned error can't find the container with id c66257ea592b791fa2bc866f388f9278f9e0a7990cac0c86d71050932f47190a
Aug 04 17:52:00.601697 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.601540    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 17:52:00.602867 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:52:00.602845    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86548434_a6dc_4663_a028_f2967885ba3e.slice/crio-733131dde6469e1ccd7ea497697887ab495070dfbec69bccfcad4a08751f07ee WatchSource:0}: Error finding container 733131dde6469e1ccd7ea497697887ab495070dfbec69bccfcad4a08751f07ee: Status 404 returned error can't find the container with id 733131dde6469e1ccd7ea497697887ab495070dfbec69bccfcad4a08751f07ee
Aug 04 17:52:00.862748 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.862672    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5ee91876-14a5-4691-9f54-0d3d11cf70e3-original-pull-secret\") pod \"global-pull-secret-syncer-jgm5s\" (UID: \"5ee91876-14a5-4691-9f54-0d3d11cf70e3\") " pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:00.862899 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:00.862820    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 17:52:00.862899 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:00.862890    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ee91876-14a5-4691-9f54-0d3d11cf70e3-original-pull-secret podName:5ee91876-14a5-4691-9f54-0d3d11cf70e3 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:01.862870223 +0000 UTC m=+4.046030389 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5ee91876-14a5-4691-9f54-0d3d11cf70e3-original-pull-secret") pod "global-pull-secret-syncer-jgm5s" (UID: "5ee91876-14a5-4691-9f54-0d3d11cf70e3") : object "kube-system"/"original-pull-secret" not registered
Aug 04 17:52:00.963048 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:00.963006    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmwgb\" (UniqueName: \"kubernetes.io/projected/fba423bd-0c4b-4b5c-851d-f604236048c3-kube-api-access-vmwgb\") pod \"network-check-target-5x9t4\" (UID: \"fba423bd-0c4b-4b5c-851d-f604236048c3\") " pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:52:00.963199 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:00.963163    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 17:52:00.963199 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:00.963181    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 17:52:00.963199 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:00.963192    2560 projected.go:196] Error preparing data for projected volume kube-api-access-vmwgb for pod openshift-network-diagnostics/network-check-target-5x9t4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 17:52:00.963352 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:00.963254    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fba423bd-0c4b-4b5c-851d-f604236048c3-kube-api-access-vmwgb podName:fba423bd-0c4b-4b5c-851d-f604236048c3 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:01.963235717 +0000 UTC m=+4.146395890 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vmwgb" (UniqueName: "kubernetes.io/projected/fba423bd-0c4b-4b5c-851d-f604236048c3-kube-api-access-vmwgb") pod "network-check-target-5x9t4" (UID: "fba423bd-0c4b-4b5c-851d-f604236048c3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 17:52:01.063690 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:01.063657    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b581e21-8a6f-4ed7-8751-87941a6cbac3-metrics-certs\") pod \"network-metrics-daemon-sstpm\" (UID: \"6b581e21-8a6f-4ed7-8751-87941a6cbac3\") " pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:01.063847 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:01.063783    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 17:52:01.063847 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:01.063837    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b581e21-8a6f-4ed7-8751-87941a6cbac3-metrics-certs podName:6b581e21-8a6f-4ed7-8751-87941a6cbac3 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:02.063823495 +0000 UTC m=+4.246983648 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6b581e21-8a6f-4ed7-8751-87941a6cbac3-metrics-certs") pod "network-metrics-daemon-sstpm" (UID: "6b581e21-8a6f-4ed7-8751-87941a6cbac3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 17:52:01.292209 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:01.292109    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 17:47:00 +0000 UTC" deadline="2028-01-31 22:31:15.447748949 +0000 UTC"
Aug 04 17:52:01.292209 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:01.292149    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13084h39m14.155605628s"
Aug 04 17:52:01.377226 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:01.377188    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fjnbn" event={"ID":"8941bb3a-0305-4d69-8a95-bee3e685acbb","Type":"ContainerStarted","Data":"c66257ea592b791fa2bc866f388f9278f9e0a7990cac0c86d71050932f47190a"}
Aug 04 17:52:01.380858 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:01.380800    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lzkvj" event={"ID":"0b696fd5-c22f-480a-a9bd-a2f8a19784ae","Type":"ContainerStarted","Data":"d5a96885491ed3007eafe4b3ed9ccce2ed7710a3eabb4d03e6293297c854747f"}
Aug 04 17:52:01.382136 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:01.382108    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9" event={"ID":"436be5a7-f6ca-4507-94ca-d9c442879877","Type":"ContainerStarted","Data":"ba173063f04d54faa62aa94ec98f736c06e8b8e37edc1bf76528dbde5c7e1788"}
Aug 04 17:52:01.383514 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:01.383483    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bq2vk" event={"ID":"86548434-a6dc-4663-a028-f2967885ba3e","Type":"ContainerStarted","Data":"733131dde6469e1ccd7ea497697887ab495070dfbec69bccfcad4a08751f07ee"}
Aug 04 17:52:01.384947 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:01.384928    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mdtdv" event={"ID":"a4441318-a230-45e7-8ab5-bf4049533e35","Type":"ContainerStarted","Data":"f2c2eb18869a74b0fdc9d5e67bbff989b64204cded76ccd82635fa5085cd7c75"}
Aug 04 17:52:01.386348 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:01.386321    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f2zqg" event={"ID":"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56","Type":"ContainerStarted","Data":"e20725e70d7ac49b9de8b58126863f0a0d9724eb3a0c552cbddb64d94f1f9abe"}
Aug 04 17:52:01.387870 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:01.387847    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-c8zhn" event={"ID":"3159ec60-2d7d-4781-ad4a-f103dcbba781","Type":"ContainerStarted","Data":"9ad9554e4d58cea3b6682c0b6c51fdb45d4db5525b23dc6165be99f39566cd15"}
Aug 04 17:52:01.389874 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:01.389847    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qrkjb" event={"ID":"ac303603-7175-4c58-8989-bafad506bc15","Type":"ContainerStarted","Data":"c0a47c5dee42b70d96a29da12f775208cc6d0a0062c288a806ee5628b50ee9fc"}
Aug 04 17:52:01.391980 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:01.391956    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g" event={"ID":"d5c319a9-83f7-4237-be47-bc28a033babe","Type":"ContainerStarted","Data":"ddb81822ef8d0f261d1b1f1d1d03fd5cef1a89379bb8441643af0bbf62508072"}
Aug 04 17:52:01.873418 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:01.872768    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5ee91876-14a5-4691-9f54-0d3d11cf70e3-original-pull-secret\") pod \"global-pull-secret-syncer-jgm5s\" (UID: \"5ee91876-14a5-4691-9f54-0d3d11cf70e3\") " pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:01.873418 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:01.872977    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 17:52:01.873418 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:01.873065    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ee91876-14a5-4691-9f54-0d3d11cf70e3-original-pull-secret podName:5ee91876-14a5-4691-9f54-0d3d11cf70e3 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:03.873046194 +0000 UTC m=+6.056206365 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5ee91876-14a5-4691-9f54-0d3d11cf70e3-original-pull-secret") pod "global-pull-secret-syncer-jgm5s" (UID: "5ee91876-14a5-4691-9f54-0d3d11cf70e3") : object "kube-system"/"original-pull-secret" not registered
Aug 04 17:52:01.974104 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:01.974067    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmwgb\" (UniqueName: \"kubernetes.io/projected/fba423bd-0c4b-4b5c-851d-f604236048c3-kube-api-access-vmwgb\") pod \"network-check-target-5x9t4\" (UID: \"fba423bd-0c4b-4b5c-851d-f604236048c3\") " pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:52:01.974323 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:01.974308    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 17:52:01.974392 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:01.974334    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 17:52:01.974392 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:01.974346    2560 projected.go:196] Error preparing data for projected volume kube-api-access-vmwgb for pod openshift-network-diagnostics/network-check-target-5x9t4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 17:52:01.974500 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:01.974402    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fba423bd-0c4b-4b5c-851d-f604236048c3-kube-api-access-vmwgb podName:fba423bd-0c4b-4b5c-851d-f604236048c3 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:03.974384486 +0000 UTC m=+6.157544661 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vmwgb" (UniqueName: "kubernetes.io/projected/fba423bd-0c4b-4b5c-851d-f604236048c3-kube-api-access-vmwgb") pod "network-check-target-5x9t4" (UID: "fba423bd-0c4b-4b5c-851d-f604236048c3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 17:52:02.074659 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:02.074624    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b581e21-8a6f-4ed7-8751-87941a6cbac3-metrics-certs\") pod \"network-metrics-daemon-sstpm\" (UID: \"6b581e21-8a6f-4ed7-8751-87941a6cbac3\") " pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:02.074832 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:02.074806    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 17:52:02.074891 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:02.074867    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b581e21-8a6f-4ed7-8751-87941a6cbac3-metrics-certs podName:6b581e21-8a6f-4ed7-8751-87941a6cbac3 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:04.074848424 +0000 UTC m=+6.258008597 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6b581e21-8a6f-4ed7-8751-87941a6cbac3-metrics-certs") pod "network-metrics-daemon-sstpm" (UID: "6b581e21-8a6f-4ed7-8751-87941a6cbac3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 17:52:02.292946 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:02.292845    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 17:47:00 +0000 UTC" deadline="2028-02-22 20:37:54.831268291 +0000 UTC"
Aug 04 17:52:02.292946 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:02.292901    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13610h45m52.538370643s"
Aug 04 17:52:02.347315 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:02.347288    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:02.347446 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:02.347421    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jgm5s" podUID="5ee91876-14a5-4691-9f54-0d3d11cf70e3"
Aug 04 17:52:02.347834 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:02.347815    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:52:02.347932 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:02.347911    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5x9t4" podUID="fba423bd-0c4b-4b5c-851d-f604236048c3"
Aug 04 17:52:02.348041 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:02.348005    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:02.348144 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:02.348126    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sstpm" podUID="6b581e21-8a6f-4ed7-8751-87941a6cbac3"
Aug 04 17:52:03.418585 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:03.418502    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-172.ec2.internal" event={"ID":"fbe04a653517381697f35b4a5ee6bb15","Type":"ContainerStarted","Data":"adf5e5447c7dbb73758787fb66db274e78e4a3492e738a4fadf38621abd1a895"}
Aug 04 17:52:03.423368 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:03.423340    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-172.ec2.internal" event={"ID":"3428ebcd2d197fe8677d9367742cfcaa","Type":"ContainerStarted","Data":"30ea098ee82a5e9250b9bf0cb961ea6b315d340902d4f5c126e21be9c750d942"}
Aug 04 17:52:03.892965 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:03.892377    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5ee91876-14a5-4691-9f54-0d3d11cf70e3-original-pull-secret\") pod \"global-pull-secret-syncer-jgm5s\" (UID: \"5ee91876-14a5-4691-9f54-0d3d11cf70e3\") " pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:03.892965 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:03.892539    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 17:52:03.892965 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:03.892597    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ee91876-14a5-4691-9f54-0d3d11cf70e3-original-pull-secret podName:5ee91876-14a5-4691-9f54-0d3d11cf70e3 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:07.892580669 +0000 UTC m=+10.075740829 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5ee91876-14a5-4691-9f54-0d3d11cf70e3-original-pull-secret") pod "global-pull-secret-syncer-jgm5s" (UID: "5ee91876-14a5-4691-9f54-0d3d11cf70e3") : object "kube-system"/"original-pull-secret" not registered
Aug 04 17:52:03.993108 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:03.993069    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmwgb\" (UniqueName: \"kubernetes.io/projected/fba423bd-0c4b-4b5c-851d-f604236048c3-kube-api-access-vmwgb\") pod \"network-check-target-5x9t4\" (UID: \"fba423bd-0c4b-4b5c-851d-f604236048c3\") " pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:52:03.993347 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:03.993329    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 17:52:03.993347 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:03.993354    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 17:52:03.993487 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:03.993367    2560 projected.go:196] Error preparing data for projected volume kube-api-access-vmwgb for pod openshift-network-diagnostics/network-check-target-5x9t4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 17:52:03.993487 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:03.993433    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fba423bd-0c4b-4b5c-851d-f604236048c3-kube-api-access-vmwgb podName:fba423bd-0c4b-4b5c-851d-f604236048c3 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:07.993414944 +0000 UTC m=+10.176575118 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vmwgb" (UniqueName: "kubernetes.io/projected/fba423bd-0c4b-4b5c-851d-f604236048c3-kube-api-access-vmwgb") pod "network-check-target-5x9t4" (UID: "fba423bd-0c4b-4b5c-851d-f604236048c3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 17:52:04.093948 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:04.093911    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b581e21-8a6f-4ed7-8751-87941a6cbac3-metrics-certs\") pod \"network-metrics-daemon-sstpm\" (UID: \"6b581e21-8a6f-4ed7-8751-87941a6cbac3\") " pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:04.094139 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:04.094114    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 17:52:04.094217 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:04.094204    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b581e21-8a6f-4ed7-8751-87941a6cbac3-metrics-certs podName:6b581e21-8a6f-4ed7-8751-87941a6cbac3 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:08.094184678 +0000 UTC m=+10.277344838 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6b581e21-8a6f-4ed7-8751-87941a6cbac3-metrics-certs") pod "network-metrics-daemon-sstpm" (UID: "6b581e21-8a6f-4ed7-8751-87941a6cbac3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 17:52:04.348415 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:04.348332    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:04.348564 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:04.348461    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jgm5s" podUID="5ee91876-14a5-4691-9f54-0d3d11cf70e3"
Aug 04 17:52:04.348826 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:04.348805    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:04.348899 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:04.348830    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:52:04.348956 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:04.348936    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sstpm" podUID="6b581e21-8a6f-4ed7-8751-87941a6cbac3"
Aug 04 17:52:04.349077 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:04.349034    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5x9t4" podUID="fba423bd-0c4b-4b5c-851d-f604236048c3"
Aug 04 17:52:06.349177 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:06.349142    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:06.349687 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:06.349277    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jgm5s" podUID="5ee91876-14a5-4691-9f54-0d3d11cf70e3"
Aug 04 17:52:06.349687 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:06.349321    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:06.349687 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:06.349476    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sstpm" podUID="6b581e21-8a6f-4ed7-8751-87941a6cbac3"
Aug 04 17:52:06.349687 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:06.349543    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:52:06.349687 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:06.349623    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5x9t4" podUID="fba423bd-0c4b-4b5c-851d-f604236048c3"
Aug 04 17:52:07.929790 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:07.929753    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5ee91876-14a5-4691-9f54-0d3d11cf70e3-original-pull-secret\") pod \"global-pull-secret-syncer-jgm5s\" (UID: \"5ee91876-14a5-4691-9f54-0d3d11cf70e3\") " pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:07.930274 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:07.929935    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 17:52:07.930274 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:07.929980    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ee91876-14a5-4691-9f54-0d3d11cf70e3-original-pull-secret podName:5ee91876-14a5-4691-9f54-0d3d11cf70e3 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:15.92996681 +0000 UTC m=+18.113126965 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5ee91876-14a5-4691-9f54-0d3d11cf70e3-original-pull-secret") pod "global-pull-secret-syncer-jgm5s" (UID: "5ee91876-14a5-4691-9f54-0d3d11cf70e3") : object "kube-system"/"original-pull-secret" not registered
Aug 04 17:52:08.030648 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:08.030611    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmwgb\" (UniqueName: \"kubernetes.io/projected/fba423bd-0c4b-4b5c-851d-f604236048c3-kube-api-access-vmwgb\") pod \"network-check-target-5x9t4\" (UID: \"fba423bd-0c4b-4b5c-851d-f604236048c3\") " pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:52:08.030828 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:08.030797    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 17:52:08.030828 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:08.030819    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 17:52:08.030955 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:08.030832    2560 projected.go:196] Error preparing data for projected volume kube-api-access-vmwgb for pod openshift-network-diagnostics/network-check-target-5x9t4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 17:52:08.030955 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:08.030890    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fba423bd-0c4b-4b5c-851d-f604236048c3-kube-api-access-vmwgb podName:fba423bd-0c4b-4b5c-851d-f604236048c3 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:16.030870703 +0000 UTC m=+18.214030858 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vmwgb" (UniqueName: "kubernetes.io/projected/fba423bd-0c4b-4b5c-851d-f604236048c3-kube-api-access-vmwgb") pod "network-check-target-5x9t4" (UID: "fba423bd-0c4b-4b5c-851d-f604236048c3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 17:52:08.131257 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:08.131221    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b581e21-8a6f-4ed7-8751-87941a6cbac3-metrics-certs\") pod \"network-metrics-daemon-sstpm\" (UID: \"6b581e21-8a6f-4ed7-8751-87941a6cbac3\") " pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:08.131445 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:08.131418    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 17:52:08.131574 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:08.131486    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b581e21-8a6f-4ed7-8751-87941a6cbac3-metrics-certs podName:6b581e21-8a6f-4ed7-8751-87941a6cbac3 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:16.131466721 +0000 UTC m=+18.314626888 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6b581e21-8a6f-4ed7-8751-87941a6cbac3-metrics-certs") pod "network-metrics-daemon-sstpm" (UID: "6b581e21-8a6f-4ed7-8751-87941a6cbac3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 17:52:08.351046 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:08.349260    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:08.351046 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:08.349498    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jgm5s" podUID="5ee91876-14a5-4691-9f54-0d3d11cf70e3"
Aug 04 17:52:08.351046 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:08.350173    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:52:08.351046 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:08.350211    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:08.351046 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:08.350666    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sstpm" podUID="6b581e21-8a6f-4ed7-8751-87941a6cbac3"
Aug 04 17:52:08.351394 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:08.351217    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5x9t4" podUID="fba423bd-0c4b-4b5c-851d-f604236048c3"
Aug 04 17:52:09.436763 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:09.436726    2560 generic.go:356] "Generic (PLEG): container finished" podID="fbe04a653517381697f35b4a5ee6bb15" containerID="adf5e5447c7dbb73758787fb66db274e78e4a3492e738a4fadf38621abd1a895" exitCode=0
Aug 04 17:52:09.437225 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:09.436779    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-172.ec2.internal" event={"ID":"fbe04a653517381697f35b4a5ee6bb15","Type":"ContainerDied","Data":"adf5e5447c7dbb73758787fb66db274e78e4a3492e738a4fadf38621abd1a895"}
Aug 04 17:52:09.456639 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:09.456589    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-172.ec2.internal" podStartSLOduration=10.456570767 podStartE2EDuration="10.456570767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 17:51:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:52:03.460869879 +0000 UTC m=+5.644030056" watchObservedRunningTime="2026-08-04 17:52:09.456570767 +0000 UTC m=+11.639730944"
Aug 04 17:52:10.347632 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:10.347598    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:10.347822 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:10.347725    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jgm5s" podUID="5ee91876-14a5-4691-9f54-0d3d11cf70e3"
Aug 04 17:52:10.348136 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:10.348116    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:52:10.348246 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:10.348225    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5x9t4" podUID="fba423bd-0c4b-4b5c-851d-f604236048c3"
Aug 04 17:52:10.348640 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:10.348610    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:10.348752 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:10.348734    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sstpm" podUID="6b581e21-8a6f-4ed7-8751-87941a6cbac3"
Aug 04 17:52:12.347415 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:12.347339    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:12.347819 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:12.347463    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jgm5s" podUID="5ee91876-14a5-4691-9f54-0d3d11cf70e3"
Aug 04 17:52:12.347819 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:12.347340    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:52:12.347819 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:12.347503    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:12.347819 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:12.347609    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5x9t4" podUID="fba423bd-0c4b-4b5c-851d-f604236048c3"
Aug 04 17:52:12.347819 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:12.347708    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sstpm" podUID="6b581e21-8a6f-4ed7-8751-87941a6cbac3"
Aug 04 17:52:14.347431 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:14.347391    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:14.347431 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:14.347418    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:14.347865 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:14.347421    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:52:14.347865 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:14.347515    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jgm5s" podUID="5ee91876-14a5-4691-9f54-0d3d11cf70e3"
Aug 04 17:52:14.347865 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:14.347603    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sstpm" podUID="6b581e21-8a6f-4ed7-8751-87941a6cbac3"
Aug 04 17:52:14.347865 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:14.347678    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5x9t4" podUID="fba423bd-0c4b-4b5c-851d-f604236048c3"
Aug 04 17:52:15.991791 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:15.991753    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5ee91876-14a5-4691-9f54-0d3d11cf70e3-original-pull-secret\") pod \"global-pull-secret-syncer-jgm5s\" (UID: \"5ee91876-14a5-4691-9f54-0d3d11cf70e3\") " pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:15.992207 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:15.991917    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 17:52:15.992207 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:15.992011    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ee91876-14a5-4691-9f54-0d3d11cf70e3-original-pull-secret podName:5ee91876-14a5-4691-9f54-0d3d11cf70e3 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:31.991972697 +0000 UTC m=+34.175132867 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5ee91876-14a5-4691-9f54-0d3d11cf70e3-original-pull-secret") pod "global-pull-secret-syncer-jgm5s" (UID: "5ee91876-14a5-4691-9f54-0d3d11cf70e3") : object "kube-system"/"original-pull-secret" not registered
Aug 04 17:52:16.093067 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:16.093034    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmwgb\" (UniqueName: \"kubernetes.io/projected/fba423bd-0c4b-4b5c-851d-f604236048c3-kube-api-access-vmwgb\") pod \"network-check-target-5x9t4\" (UID: \"fba423bd-0c4b-4b5c-851d-f604236048c3\") " pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:52:16.093244 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:16.093196    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 17:52:16.093244 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:16.093216    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 17:52:16.093244 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:16.093230    2560 projected.go:196] Error preparing data for projected volume kube-api-access-vmwgb for pod openshift-network-diagnostics/network-check-target-5x9t4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 17:52:16.093408 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:16.093297    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fba423bd-0c4b-4b5c-851d-f604236048c3-kube-api-access-vmwgb podName:fba423bd-0c4b-4b5c-851d-f604236048c3 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:32.093278586 +0000 UTC m=+34.276438783 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vmwgb" (UniqueName: "kubernetes.io/projected/fba423bd-0c4b-4b5c-851d-f604236048c3-kube-api-access-vmwgb") pod "network-check-target-5x9t4" (UID: "fba423bd-0c4b-4b5c-851d-f604236048c3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 17:52:16.194280 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:16.194238    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b581e21-8a6f-4ed7-8751-87941a6cbac3-metrics-certs\") pod \"network-metrics-daemon-sstpm\" (UID: \"6b581e21-8a6f-4ed7-8751-87941a6cbac3\") " pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:16.194449 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:16.194413    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 17:52:16.194515 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:16.194477    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b581e21-8a6f-4ed7-8751-87941a6cbac3-metrics-certs podName:6b581e21-8a6f-4ed7-8751-87941a6cbac3 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:32.194461908 +0000 UTC m=+34.377622066 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6b581e21-8a6f-4ed7-8751-87941a6cbac3-metrics-certs") pod "network-metrics-daemon-sstpm" (UID: "6b581e21-8a6f-4ed7-8751-87941a6cbac3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 17:52:16.347676 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:16.347601    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:52:16.347676 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:16.347644    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:16.347884 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:16.347706    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:16.347884 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:16.347715    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5x9t4" podUID="fba423bd-0c4b-4b5c-851d-f604236048c3"
Aug 04 17:52:16.347884 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:16.347818    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jgm5s" podUID="5ee91876-14a5-4691-9f54-0d3d11cf70e3"
Aug 04 17:52:16.348054 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:16.347923    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sstpm" podUID="6b581e21-8a6f-4ed7-8751-87941a6cbac3"
Aug 04 17:52:18.348199 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:18.348172    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:18.348688 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:18.348269    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jgm5s" podUID="5ee91876-14a5-4691-9f54-0d3d11cf70e3"
Aug 04 17:52:18.348688 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:18.348309    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:52:18.348688 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:18.348368    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5x9t4" podUID="fba423bd-0c4b-4b5c-851d-f604236048c3"
Aug 04 17:52:18.348688 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:18.348403    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:18.348688 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:18.348470    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sstpm" podUID="6b581e21-8a6f-4ed7-8751-87941a6cbac3"
Aug 04 17:52:20.347829 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:20.347626    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:52:20.348365 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:20.347640    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:20.348365 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:20.348064    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5x9t4" podUID="fba423bd-0c4b-4b5c-851d-f604236048c3"
Aug 04 17:52:20.348365 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:20.348159    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jgm5s" podUID="5ee91876-14a5-4691-9f54-0d3d11cf70e3"
Aug 04 17:52:20.348365 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:20.347653    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:20.348365 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:20.348255    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sstpm" podUID="6b581e21-8a6f-4ed7-8751-87941a6cbac3"
Aug 04 17:52:20.457936 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:20.457681    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-c8zhn" event={"ID":"3159ec60-2d7d-4781-ad4a-f103dcbba781","Type":"ContainerStarted","Data":"b26cb63096d8e2ea9642ba5bfa651086216f76ee7b802dbd055ada678b9d438a"}
Aug 04 17:52:20.459629 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:20.459603    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qrkjb" event={"ID":"ac303603-7175-4c58-8989-bafad506bc15","Type":"ContainerStarted","Data":"a54d4e3909fd2fcde357a04252ee050b830cdaca4cefdf9b1f8af03f787c839e"}
Aug 04 17:52:20.461471 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:20.461446    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 17:52:20.461836 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:20.461816    2560 generic.go:356] "Generic (PLEG): container finished" podID="d5c319a9-83f7-4237-be47-bc28a033babe" containerID="930daf0a3accf72b3ac0b40b3348673b040dc9847e4317ecb3ea2db6755a6098" exitCode=1
Aug 04 17:52:20.461906 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:20.461882    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g" event={"ID":"d5c319a9-83f7-4237-be47-bc28a033babe","Type":"ContainerStarted","Data":"0d2bfd79f36ed15f3c7e439bb6975b59e49f499de3535bd800cc000abc4d9fe2"}
Aug 04 17:52:20.461982 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:20.461966    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g" event={"ID":"d5c319a9-83f7-4237-be47-bc28a033babe","Type":"ContainerDied","Data":"930daf0a3accf72b3ac0b40b3348673b040dc9847e4317ecb3ea2db6755a6098"}
Aug 04 17:52:20.462066 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:20.462051    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g" event={"ID":"d5c319a9-83f7-4237-be47-bc28a033babe","Type":"ContainerStarted","Data":"158f3db8922b263fc3e0eaf9e846381d06c0a878c49c4fc49dc6a9c9752b6589"}
Aug 04 17:52:20.463804 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:20.463781    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-172.ec2.internal" event={"ID":"fbe04a653517381697f35b4a5ee6bb15","Type":"ContainerStarted","Data":"f3835267ff69807ad4a8621faeaff5672df8a05b9882bff757b3d8e6ca7b6e3f"}
Aug 04 17:52:20.465110 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:20.465087    2560 generic.go:356] "Generic (PLEG): container finished" podID="8941bb3a-0305-4d69-8a95-bee3e685acbb" containerID="b073a8e14bf10ce51c223d314d6e95760a7767c726a6988df9c420445c6dbf08" exitCode=0
Aug 04 17:52:20.465197 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:20.465155    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fjnbn" event={"ID":"8941bb3a-0305-4d69-8a95-bee3e685acbb","Type":"ContainerDied","Data":"b073a8e14bf10ce51c223d314d6e95760a7767c726a6988df9c420445c6dbf08"}
Aug 04 17:52:20.466609 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:20.466563    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lzkvj" event={"ID":"0b696fd5-c22f-480a-a9bd-a2f8a19784ae","Type":"ContainerStarted","Data":"accc6b7752aa8a8b7935d1c11705ac23184e980abd4d929711af80617841b52c"}
Aug 04 17:52:20.467974 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:20.467956    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9" event={"ID":"436be5a7-f6ca-4507-94ca-d9c442879877","Type":"ContainerStarted","Data":"20859b57be202d2ed48ed27f4ba6ecc4cd7e431e2c60c2dd5f573ea5d535ea82"}
Aug 04 17:52:20.469462 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:20.469440    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mdtdv" event={"ID":"a4441318-a230-45e7-8ab5-bf4049533e35","Type":"ContainerStarted","Data":"0acae7eefaa58f31f974528610ab0a0958a87e66f1259663c440447e4f3aa8d4"}
Aug 04 17:52:20.470851 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:20.470833    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f2zqg" event={"ID":"cb7cece1-6af0-4189-b5e0-3b5db5c0ca56","Type":"ContainerStarted","Data":"ccde71927295cdc140af1979656d06e0bb05d2aafe5c6d3e7c2d9f2434e49452"}
Aug 04 17:52:20.479290 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:20.479252    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-c8zhn" podStartSLOduration=3.204364856 podStartE2EDuration="22.47923992s" totalImagesPullingTime="19.274875064s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 17:51:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:52:20.478605539 +0000 UTC m=+22.661765732" watchObservedRunningTime="2026-08-04 17:52:20.47923992 +0000 UTC m=+22.662400097"
Aug 04 17:52:20.505091 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:20.505050    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lzkvj" podStartSLOduration=3.239023929 podStartE2EDuration="22.505034799s" totalImagesPullingTime="19.26601087s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 17:51:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:52:20.504965798 +0000 UTC m=+22.688125973" watchObservedRunningTime="2026-08-04 17:52:20.505034799 +0000 UTC m=+22.688194974"
Aug 04 17:52:20.559315 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:20.559251    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mdtdv" podStartSLOduration=3.617697768 podStartE2EDuration="22.559228609s" totalImagesPullingTime="18.941530841s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 17:51:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:52:20.529200416 +0000 UTC m=+22.712360591" watchObservedRunningTime="2026-08-04 17:52:20.559228609 +0000 UTC m=+22.742388786"
Aug 04 17:52:20.580091 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:20.580033    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-172.ec2.internal" podStartSLOduration=21.579967804 podStartE2EDuration="21.579967804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 17:51:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:52:20.578426834 +0000 UTC m=+22.761587011" watchObservedRunningTime="2026-08-04 17:52:20.579967804 +0000 UTC m=+22.763127982"
Aug 04 17:52:20.602231 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:20.602115    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-f2zqg" podStartSLOduration=2.792459322 podStartE2EDuration="22.602095065s" totalImagesPullingTime="19.809635743s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 17:51:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:52:20.600655961 +0000 UTC m=+22.783816138" watchObservedRunningTime="2026-08-04 17:52:20.602095065 +0000 UTC m=+22.785255243"
Aug 04 17:52:20.620047 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:20.619976    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qrkjb" podStartSLOduration=3.341969639 podStartE2EDuration="22.619960998s" totalImagesPullingTime="19.277991359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 17:51:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:52:20.619320084 +0000 UTC m=+22.802480260" watchObservedRunningTime="2026-08-04 17:52:20.619960998 +0000 UTC m=+22.803121175"
Aug 04 17:52:21.476278 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:21.476251    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 17:52:21.476932 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:21.476632    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g" event={"ID":"d5c319a9-83f7-4237-be47-bc28a033babe","Type":"ContainerStarted","Data":"44ecbfdb08e4c4ad45e9894759a2be36b1f1226659483091da84274b7edab3ef"}
Aug 04 17:52:21.476932 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:21.476751    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g" event={"ID":"d5c319a9-83f7-4237-be47-bc28a033babe","Type":"ContainerStarted","Data":"3d37bca970df4c7498a4bf943c80f0c596e95bd7e30f5d0883f029a69e6891a5"}
Aug 04 17:52:21.476932 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:21.476803    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g" event={"ID":"d5c319a9-83f7-4237-be47-bc28a033babe","Type":"ContainerStarted","Data":"4369007670dc8588e42604c3bd3332982672769a7d1c2a42e1bc34716cf0c595"}
Aug 04 17:52:21.478180 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:21.478105    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bq2vk" event={"ID":"86548434-a6dc-4663-a028-f2967885ba3e","Type":"ContainerStarted","Data":"2549e6c5c7050bf92ca0eec34440ec9a88f288d5eb6e22e44ac46c64a5bc9d77"}
Aug 04 17:52:21.498254 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:21.498214    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bq2vk" podStartSLOduration=4.246101587 podStartE2EDuration="23.498200131s" totalImagesPullingTime="19.252098544s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 17:51:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:52:21.498070856 +0000 UTC m=+23.681231033" watchObservedRunningTime="2026-08-04 17:52:21.498200131 +0000 UTC m=+23.681360307"
Aug 04 17:52:21.651832 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:21.651809    2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Aug 04 17:52:22.319349 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:22.319262    2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-04T17:52:21.651829555Z","UUID":"ebee8f55-df7d-4c54-9851-a07bacd855e7","Handler":null,"Name":"","Endpoint":""}
Aug 04 17:52:22.320696 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:22.320679    2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Aug 04 17:52:22.320803 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:22.320715    2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Aug 04 17:52:22.347770 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:22.347746    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:52:22.347903 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:22.347746    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:22.347979 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:22.347856    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5x9t4" podUID="fba423bd-0c4b-4b5c-851d-f604236048c3"
Aug 04 17:52:22.347979 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:22.347947    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:22.348121 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:22.348097    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jgm5s" podUID="5ee91876-14a5-4691-9f54-0d3d11cf70e3"
Aug 04 17:52:22.348215 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:22.348193    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sstpm" podUID="6b581e21-8a6f-4ed7-8751-87941a6cbac3"
Aug 04 17:52:22.483043 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:22.482940    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9" event={"ID":"436be5a7-f6ca-4507-94ca-d9c442879877","Type":"ContainerStarted","Data":"f6785ee674b703ba57d29f7fb625a6389551ddd4fa519eb0d3ca76536e1f69e0"}
Aug 04 17:52:23.488748 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:23.488559    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 17:52:23.489161 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:23.489140    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g" event={"ID":"d5c319a9-83f7-4237-be47-bc28a033babe","Type":"ContainerStarted","Data":"d3ea70353aa2d1169ee2b43b3a7a3b3bae3da18bad5d64fc3a047102723c3d21"}
Aug 04 17:52:23.491469 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:23.491435    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9" event={"ID":"436be5a7-f6ca-4507-94ca-d9c442879877","Type":"ContainerStarted","Data":"4e2b9d8ae6da5c0a638ec66ee56a230cb739d04bf5cbb397456e552f235a867a"}
Aug 04 17:52:23.524120 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:23.524078    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j6h9" podStartSLOduration=3.156629739 podStartE2EDuration="25.524064491s" totalImagesPullingTime="22.367434752s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 17:51:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:52:23.523800137 +0000 UTC m=+25.706960312" watchObservedRunningTime="2026-08-04 17:52:23.524064491 +0000 UTC m=+25.707224667"
Aug 04 17:52:24.347793 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:24.347762    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:52:24.347793 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:24.347783    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:24.348095 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:24.347852    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:24.348095 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:24.347973    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5x9t4" podUID="fba423bd-0c4b-4b5c-851d-f604236048c3"
Aug 04 17:52:24.348095 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:24.348081    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jgm5s" podUID="5ee91876-14a5-4691-9f54-0d3d11cf70e3"
Aug 04 17:52:24.348256 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:24.348189    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sstpm" podUID="6b581e21-8a6f-4ed7-8751-87941a6cbac3"
Aug 04 17:52:25.021790 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:25.021754    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qrkjb"
Aug 04 17:52:25.022541 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:25.022521    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qrkjb"
Aug 04 17:52:25.499756 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:25.499738    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 17:52:25.500193 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:25.500165    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g" event={"ID":"d5c319a9-83f7-4237-be47-bc28a033babe","Type":"ContainerStarted","Data":"2fb6cd55bb6d405fd36a7611d6bca20c803d23757b7ac9b78323abca78580038"}
Aug 04 17:52:25.500434 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:25.500403    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:25.500528 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:25.500473    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:25.500528 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:25.500520    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:25.500687 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:25.500670    2560 scope.go:117] "RemoveContainer" containerID="930daf0a3accf72b3ac0b40b3348673b040dc9847e4317ecb3ea2db6755a6098"
Aug 04 17:52:25.502262 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:25.502241    2560 generic.go:356] "Generic (PLEG): container finished" podID="8941bb3a-0305-4d69-8a95-bee3e685acbb" containerID="24ff446910342d78701cc358b1d365487bc442cbb36971c4a1f5bd826745deaf" exitCode=0
Aug 04 17:52:25.502353 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:25.502290    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fjnbn" event={"ID":"8941bb3a-0305-4d69-8a95-bee3e685acbb","Type":"ContainerDied","Data":"24ff446910342d78701cc358b1d365487bc442cbb36971c4a1f5bd826745deaf"}
Aug 04 17:52:25.502885 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:25.502636    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qrkjb"
Aug 04 17:52:25.503077 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:25.503026    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qrkjb"
Aug 04 17:52:25.517600 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:25.517578    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:25.517778 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:25.517754    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:26.347134 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:26.347097    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:26.347853 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:26.347097    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:52:26.347853 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:26.347208    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jgm5s" podUID="5ee91876-14a5-4691-9f54-0d3d11cf70e3"
Aug 04 17:52:26.347853 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:26.347302    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5x9t4" podUID="fba423bd-0c4b-4b5c-851d-f604236048c3"
Aug 04 17:52:26.347853 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:26.347111    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:26.347853 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:26.347400    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sstpm" podUID="6b581e21-8a6f-4ed7-8751-87941a6cbac3"
Aug 04 17:52:26.508854 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:26.508828    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 17:52:26.509262 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:26.509240    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g" event={"ID":"d5c319a9-83f7-4237-be47-bc28a033babe","Type":"ContainerStarted","Data":"1c04100f7c749bf2e0ac2f05df84a0973f70f2e1878a4a1136c62981b9da052d"}
Aug 04 17:52:26.561355 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:26.561307    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g" podStartSLOduration=9.178426393 podStartE2EDuration="28.561293028s" totalImagesPullingTime="19.382866635s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 17:51:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:52:26.556089827 +0000 UTC m=+28.739250003" watchObservedRunningTime="2026-08-04 17:52:26.561293028 +0000 UTC m=+28.744453204"
Aug 04 17:52:27.347760 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:27.347691    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jgm5s"]
Aug 04 17:52:27.348223 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:27.347815    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:27.348223 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:27.347921    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jgm5s" podUID="5ee91876-14a5-4691-9f54-0d3d11cf70e3"
Aug 04 17:52:27.350095 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:27.350076    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sstpm"]
Aug 04 17:52:27.350194 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:27.350170    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:27.350281 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:27.350262    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sstpm" podUID="6b581e21-8a6f-4ed7-8751-87941a6cbac3"
Aug 04 17:52:27.353601 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:27.353580    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5x9t4"]
Aug 04 17:52:27.353677 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:27.353653    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:52:27.353729 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:27.353711    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5x9t4" podUID="fba423bd-0c4b-4b5c-851d-f604236048c3"
Aug 04 17:52:27.513215 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:27.513186    2560 generic.go:356] "Generic (PLEG): container finished" podID="8941bb3a-0305-4d69-8a95-bee3e685acbb" containerID="0ebb13f4d0875357ec9f229606ac816cc2eb0d51fd1485f1bf7003aa31381d63" exitCode=0
Aug 04 17:52:27.513332 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:27.513256    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fjnbn" event={"ID":"8941bb3a-0305-4d69-8a95-bee3e685acbb","Type":"ContainerDied","Data":"0ebb13f4d0875357ec9f229606ac816cc2eb0d51fd1485f1bf7003aa31381d63"}
Aug 04 17:52:29.347837 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:29.347692    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:29.348206 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:29.347693    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:52:29.348206 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:29.347908    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sstpm" podUID="6b581e21-8a6f-4ed7-8751-87941a6cbac3"
Aug 04 17:52:29.348206 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:29.348022    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5x9t4" podUID="fba423bd-0c4b-4b5c-851d-f604236048c3"
Aug 04 17:52:29.348206 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:29.347692    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:29.348206 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:29.348118    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jgm5s" podUID="5ee91876-14a5-4691-9f54-0d3d11cf70e3"
Aug 04 17:52:29.520681 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:29.520619    2560 generic.go:356] "Generic (PLEG): container finished" podID="8941bb3a-0305-4d69-8a95-bee3e685acbb" containerID="4147fd85a77aeba4459970dc48f467b9638b4bf3bc2fd13a834c8014c06ec244" exitCode=0
Aug 04 17:52:29.520681 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:29.520660    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fjnbn" event={"ID":"8941bb3a-0305-4d69-8a95-bee3e685acbb","Type":"ContainerDied","Data":"4147fd85a77aeba4459970dc48f467b9638b4bf3bc2fd13a834c8014c06ec244"}
Aug 04 17:52:31.347720 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:31.347688    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:31.348396 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:31.347737    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:52:31.348396 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:31.347702    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:31.348396 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:31.347825    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sstpm" podUID="6b581e21-8a6f-4ed7-8751-87941a6cbac3"
Aug 04 17:52:31.348396 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:31.347930    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jgm5s" podUID="5ee91876-14a5-4691-9f54-0d3d11cf70e3"
Aug 04 17:52:31.348396 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:31.348047    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5x9t4" podUID="fba423bd-0c4b-4b5c-851d-f604236048c3"
Aug 04 17:52:32.021823 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:32.021788    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5ee91876-14a5-4691-9f54-0d3d11cf70e3-original-pull-secret\") pod \"global-pull-secret-syncer-jgm5s\" (UID: \"5ee91876-14a5-4691-9f54-0d3d11cf70e3\") " pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:32.022027 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:32.021925    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 17:52:32.022027 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:32.022008    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ee91876-14a5-4691-9f54-0d3d11cf70e3-original-pull-secret podName:5ee91876-14a5-4691-9f54-0d3d11cf70e3 nodeName:}" failed. No retries permitted until 2026-08-04 17:53:04.02196982 +0000 UTC m=+66.205129993 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5ee91876-14a5-4691-9f54-0d3d11cf70e3-original-pull-secret") pod "global-pull-secret-syncer-jgm5s" (UID: "5ee91876-14a5-4691-9f54-0d3d11cf70e3") : object "kube-system"/"original-pull-secret" not registered
Aug 04 17:52:32.122620 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:32.122584    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmwgb\" (UniqueName: \"kubernetes.io/projected/fba423bd-0c4b-4b5c-851d-f604236048c3-kube-api-access-vmwgb\") pod \"network-check-target-5x9t4\" (UID: \"fba423bd-0c4b-4b5c-851d-f604236048c3\") " pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:52:32.122795 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:32.122773    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 17:52:32.122862 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:32.122801    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 17:52:32.122862 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:32.122815    2560 projected.go:196] Error preparing data for projected volume kube-api-access-vmwgb for pod openshift-network-diagnostics/network-check-target-5x9t4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 17:52:32.123043 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:32.122875    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fba423bd-0c4b-4b5c-851d-f604236048c3-kube-api-access-vmwgb podName:fba423bd-0c4b-4b5c-851d-f604236048c3 nodeName:}" failed. No retries permitted until 2026-08-04 17:53:04.12285734 +0000 UTC m=+66.306017497 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vmwgb" (UniqueName: "kubernetes.io/projected/fba423bd-0c4b-4b5c-851d-f604236048c3-kube-api-access-vmwgb") pod "network-check-target-5x9t4" (UID: "fba423bd-0c4b-4b5c-851d-f604236048c3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 17:52:32.223800 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:32.223762    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b581e21-8a6f-4ed7-8751-87941a6cbac3-metrics-certs\") pod \"network-metrics-daemon-sstpm\" (UID: \"6b581e21-8a6f-4ed7-8751-87941a6cbac3\") " pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:32.223944 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:32.223922    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 17:52:32.224026 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:32.223983    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b581e21-8a6f-4ed7-8751-87941a6cbac3-metrics-certs podName:6b581e21-8a6f-4ed7-8751-87941a6cbac3 nodeName:}" failed. No retries permitted until 2026-08-04 17:53:04.223964046 +0000 UTC m=+66.407124201 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6b581e21-8a6f-4ed7-8751-87941a6cbac3-metrics-certs") pod "network-metrics-daemon-sstpm" (UID: "6b581e21-8a6f-4ed7-8751-87941a6cbac3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 17:52:33.082176 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.082097    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-172.ec2.internal" event="NodeReady"
Aug 04 17:52:33.082646 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.082244    2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Aug 04 17:52:33.123327 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.123287    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-jkhcl"]
Aug 04 17:52:33.127642 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.127617    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-kxp92"]
Aug 04 17:52:33.128449 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.128423    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-jkhcl"
Aug 04 17:52:33.131083 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.131062    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5955597dc4-jbsv8"]
Aug 04 17:52:33.131238 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.131215    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-kxp92"
Aug 04 17:52:33.134574 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.134227    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-69bc56d479-dx5kz"]
Aug 04 17:52:33.134574 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.134344    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:52:33.134574 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.134387    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Aug 04 17:52:33.134574 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.134455    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Aug 04 17:52:33.135459 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.135325    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Aug 04 17:52:33.135459 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.135341    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Aug 04 17:52:33.135459 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.135361    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-bwh6v\""
Aug 04 17:52:33.135459 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.135373    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-mvlcb\""
Aug 04 17:52:33.135459 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.135366    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Aug 04 17:52:33.136711 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.136694    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Aug 04 17:52:33.137557 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.137536    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-77696685b-ks7s5"]
Aug 04 17:52:33.137736 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.137659    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:33.140809 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.140756    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Aug 04 17:52:33.140809 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.140789    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Aug 04 17:52:33.141067 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.140834    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kwcpz\""
Aug 04 17:52:33.141067 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.140849    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Aug 04 17:52:33.141206 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.141080    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-jkhcl"]
Aug 04 17:52:33.141206 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.141104    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Aug 04 17:52:33.141206 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.141158    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-2frj8\""
Aug 04 17:52:33.141206 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.141161    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-ks7s5"
Aug 04 17:52:33.141704 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.141634    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Aug 04 17:52:33.141887 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.141870    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Aug 04 17:52:33.141965 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.141909    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Aug 04 17:52:33.142362 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.142011    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Aug 04 17:52:33.142362 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.142023    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Aug 04 17:52:33.147908 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.147889    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Aug 04 17:52:33.153846 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.148301    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Aug 04 17:52:33.153846 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.148576    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Aug 04 17:52:33.153846 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.148702    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Aug 04 17:52:33.153846 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.152533    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-n64q5"]
Aug 04 17:52:33.157075 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.155855    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-47zwf"]
Aug 04 17:52:33.157075 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.156559    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-k4rk2\""
Aug 04 17:52:33.157743 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.157680    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-n64q5"
Aug 04 17:52:33.160328 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.160310    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Aug 04 17:52:33.160405 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.160380    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-65fgd\""
Aug 04 17:52:33.161449 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.160665    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Aug 04 17:52:33.161449 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.161241    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Aug 04 17:52:33.162316 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.162297    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Aug 04 17:52:33.163004 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.162971    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-gtwkf"]
Aug 04 17:52:33.163479 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.163434    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Aug 04 17:52:33.163807 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.163785    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Aug 04 17:52:33.164236 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.164216    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-47zwf"
Aug 04 17:52:33.168740 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.166256    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Aug 04 17:52:33.168740 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.166396    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-kxp92"]
Aug 04 17:52:33.168740 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.166426    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-stb8t"]
Aug 04 17:52:33.168740 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.166782    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Aug 04 17:52:33.168740 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.167150    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Aug 04 17:52:33.168740 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.167343    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Aug 04 17:52:33.168740 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.167531    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-nqt6b\""
Aug 04 17:52:33.168740 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.167863    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf"
Aug 04 17:52:33.169876 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.169829    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-ks7s5"]
Aug 04 17:52:33.170048 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.170031    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-wt47b"]
Aug 04 17:52:33.170173 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.170156    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-stb8t"
Aug 04 17:52:33.172281 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.171660    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Aug 04 17:52:33.172281 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.171756    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Aug 04 17:52:33.172782 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.172764    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Aug 04 17:52:33.173035 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.173017    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-jj74k\""
Aug 04 17:52:33.174667 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.173193    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Aug 04 17:52:33.174667 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.173968    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Aug 04 17:52:33.174667 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.174246    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Aug 04 17:52:33.174667 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.174481    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-pblgh\""
Aug 04 17:52:33.177218 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.176050    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69bc56d479-dx5kz"]
Aug 04 17:52:33.177218 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.176105    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-t4zw5"]
Aug 04 17:52:33.177218 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.176483    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-wt47b"
Aug 04 17:52:33.180279 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.179416    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5s8jf"]
Aug 04 17:52:33.180279 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.179550    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-t4zw5"
Aug 04 17:52:33.181424 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.181401    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Aug 04 17:52:33.181644 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.181625    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 04 17:52:33.181906 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.181886    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-rtrlx\""
Aug 04 17:52:33.182146 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.182128    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 04 17:52:33.183463 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.182962    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-47zwf"]
Aug 04 17:52:33.183463 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.183050    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-n64q5"]
Aug 04 17:52:33.183463 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.183071    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-stb8t"]
Aug 04 17:52:33.183463 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.183091    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-gtwkf"]
Aug 04 17:52:33.183463 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.183111    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5s8jf"]
Aug 04 17:52:33.183463 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.183134    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-d58jd"]
Aug 04 17:52:33.183463 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.183433    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Aug 04 17:52:33.183777 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.183607    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Aug 04 17:52:33.184028 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.183847    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-w5dfh\""
Aug 04 17:52:33.184484 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.184233    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5s8jf"
Aug 04 17:52:33.184484 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.184250    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Aug 04 17:52:33.186260 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.186241    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5955597dc4-jbsv8"]
Aug 04 17:52:33.186354 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.186267    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-t4zw5"]
Aug 04 17:52:33.186354 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.186334    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-d58jd"
Aug 04 17:52:33.186883 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.186864    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Aug 04 17:52:33.187066 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.186945    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Aug 04 17:52:33.187407 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.187391    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Aug 04 17:52:33.187926 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.187909    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-z8v6x\""
Aug 04 17:52:33.188488 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.188467    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-wt47b"]
Aug 04 17:52:33.191527 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.191507    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-x4v5p\""
Aug 04 17:52:33.191919 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.191903    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Aug 04 17:52:33.192298 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.192282    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Aug 04 17:52:33.197158 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.193576    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-d58jd"]
Aug 04 17:52:33.231960 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.231932    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/165249a4-0e3d-4f20-9719-c91a6d7ce78f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-kxp92\" (UID: \"165249a4-0e3d-4f20-9719-c91a6d7ce78f\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-kxp92"
Aug 04 17:52:33.232070 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.231973    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b35d3198-f4e3-44a5-a0b7-08c987b5805f-stats-auth\") pod \"router-default-5955597dc4-jbsv8\" (UID: \"b35d3198-f4e3-44a5-a0b7-08c987b5805f\") " pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:52:33.232070 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.232017    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b35d3198-f4e3-44a5-a0b7-08c987b5805f-metrics-certs\") pod \"router-default-5955597dc4-jbsv8\" (UID: \"b35d3198-f4e3-44a5-a0b7-08c987b5805f\") " pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:52:33.232070 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.232065    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/887c097d-64f3-4f3f-99ac-7981ab483724-image-registry-private-configuration\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:33.232170 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.232086    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/887c097d-64f3-4f3f-99ac-7981ab483724-registry-certificates\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:33.232170 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.232108    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/887c097d-64f3-4f3f-99ac-7981ab483724-trusted-ca\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:33.232170 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.232150    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-registry-tls\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:33.232269 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.232211    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/165249a4-0e3d-4f20-9719-c91a6d7ce78f-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-kxp92\" (UID: \"165249a4-0e3d-4f20-9719-c91a6d7ce78f\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-kxp92"
Aug 04 17:52:33.232269 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.232240    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-bound-sa-token\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:33.232269 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.232262    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljf9z\" (UniqueName: \"kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-kube-api-access-ljf9z\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:33.232387 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.232309    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pww7\" (UniqueName: \"kubernetes.io/projected/90aff035-abe4-4fcf-bcd6-76f25a1061dc-kube-api-access-9pww7\") pod \"volume-data-source-validator-5b8776669-jkhcl\" (UID: \"90aff035-abe4-4fcf-bcd6-76f25a1061dc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-jkhcl"
Aug 04 17:52:33.232387 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.232328    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b35d3198-f4e3-44a5-a0b7-08c987b5805f-service-ca-bundle\") pod \"router-default-5955597dc4-jbsv8\" (UID: \"b35d3198-f4e3-44a5-a0b7-08c987b5805f\") " pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:52:33.232387 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.232346    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/887c097d-64f3-4f3f-99ac-7981ab483724-ca-trust-extracted\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:33.232387 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.232367    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p746h\" (UniqueName: \"kubernetes.io/projected/165249a4-0e3d-4f20-9719-c91a6d7ce78f-kube-api-access-p746h\") pod \"cluster-monitoring-operator-6f55c4f58d-kxp92\" (UID: \"165249a4-0e3d-4f20-9719-c91a6d7ce78f\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-kxp92"
Aug 04 17:52:33.232542 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.232400    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b35d3198-f4e3-44a5-a0b7-08c987b5805f-default-certificate\") pod \"router-default-5955597dc4-jbsv8\" (UID: \"b35d3198-f4e3-44a5-a0b7-08c987b5805f\") " pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:52:33.232542 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.232424    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/887c097d-64f3-4f3f-99ac-7981ab483724-installation-pull-secrets\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:33.232542 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.232456    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsf4w\" (UniqueName: \"kubernetes.io/projected/b35d3198-f4e3-44a5-a0b7-08c987b5805f-kube-api-access-gsf4w\") pod \"router-default-5955597dc4-jbsv8\" (UID: \"b35d3198-f4e3-44a5-a0b7-08c987b5805f\") " pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:52:33.332847 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.332770    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4tqd\" (UniqueName: \"kubernetes.io/projected/76442945-00cd-4bbd-8dc1-a65a9573b031-kube-api-access-b4tqd\") pod \"ingress-canary-5s8jf\" (UID: \"76442945-00cd-4bbd-8dc1-a65a9573b031\") " pod="openshift-ingress-canary/ingress-canary-5s8jf"
Aug 04 17:52:33.332847 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.332814    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3f8891ae-cf02-49a7-9144-ef455be4cdee-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-47zwf\" (UID: \"3f8891ae-cf02-49a7-9144-ef455be4cdee\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-47zwf"
Aug 04 17:52:33.332847 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.332839    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f49077e4-25d7-4cf7-8072-87c5a5e13829-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-t4zw5\" (UID: \"f49077e4-25d7-4cf7-8072-87c5a5e13829\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-t4zw5"
Aug 04 17:52:33.333137 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.332884    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f3c4ae8d-6d40-466c-a97c-daab363b3502-config\") pod \"service-ca-operator-784f97bcf6-n64q5\" (UID: \"f3c4ae8d-6d40-466c-a97c-daab363b3502\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-n64q5"
Aug 04 17:52:33.333137 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.332903    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f583466f-47c3-4403-a048-4c48550ac4fc-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-stb8t\" (UID: \"f583466f-47c3-4403-a048-4c48550ac4fc\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-stb8t"
Aug 04 17:52:33.333137 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.332926    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b35d3198-f4e3-44a5-a0b7-08c987b5805f-metrics-certs\") pod \"router-default-5955597dc4-jbsv8\" (UID: \"b35d3198-f4e3-44a5-a0b7-08c987b5805f\") " pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:52:33.333137 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.332949    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b29597f-59c4-4f28-a164-7d2e97248d83-trusted-ca-bundle\") pod \"insights-operator-77696685b-ks7s5\" (UID: \"6b29597f-59c4-4f28-a164-7d2e97248d83\") " pod="openshift-insights/insights-operator-77696685b-ks7s5"
Aug 04 17:52:33.333137 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:33.333067    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 04 17:52:33.333137 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:33.333121    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b35d3198-f4e3-44a5-a0b7-08c987b5805f-metrics-certs podName:b35d3198-f4e3-44a5-a0b7-08c987b5805f nodeName:}" failed. No retries permitted until 2026-08-04 17:52:33.833105276 +0000 UTC m=+36.016265447 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b35d3198-f4e3-44a5-a0b7-08c987b5805f-metrics-certs") pod "router-default-5955597dc4-jbsv8" (UID: "b35d3198-f4e3-44a5-a0b7-08c987b5805f") : secret "router-metrics-certs-default" not found
Aug 04 17:52:33.333591 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.333559    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/887c097d-64f3-4f3f-99ac-7981ab483724-image-registry-private-configuration\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:33.333666 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.333618    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/887c097d-64f3-4f3f-99ac-7981ab483724-trusted-ca\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:33.333666 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.333649    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf44n\" (UniqueName: \"kubernetes.io/projected/8eaf7533-2fc5-4da2-bec2-282dce644049-kube-api-access-jf44n\") pod \"console-operator-5ccd6b445f-gtwkf\" (UID: \"8eaf7533-2fc5-4da2-bec2-282dce644049\") " pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf"
Aug 04 17:52:33.333768 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.333702    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-bound-sa-token\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:33.333768 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.333730    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pww7\" (UniqueName: \"kubernetes.io/projected/90aff035-abe4-4fcf-bcd6-76f25a1061dc-kube-api-access-9pww7\") pod \"volume-data-source-validator-5b8776669-jkhcl\" (UID: \"90aff035-abe4-4fcf-bcd6-76f25a1061dc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-jkhcl"
Aug 04 17:52:33.333768 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.333752    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b35d3198-f4e3-44a5-a0b7-08c987b5805f-service-ca-bundle\") pod \"router-default-5955597dc4-jbsv8\" (UID: \"b35d3198-f4e3-44a5-a0b7-08c987b5805f\") " pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:52:33.333891 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.333778    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0a01b635-81f5-4c48-9906-dbf914e08592-config-volume\") pod \"dns-default-d58jd\" (UID: \"0a01b635-81f5-4c48-9906-dbf914e08592\") " pod="openshift-dns/dns-default-d58jd"
Aug 04 17:52:33.333891 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.333802    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p746h\" (UniqueName: \"kubernetes.io/projected/165249a4-0e3d-4f20-9719-c91a6d7ce78f-kube-api-access-p746h\") pod \"cluster-monitoring-operator-6f55c4f58d-kxp92\" (UID: \"165249a4-0e3d-4f20-9719-c91a6d7ce78f\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-kxp92"
Aug 04 17:52:33.333891 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.333828    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/887c097d-64f3-4f3f-99ac-7981ab483724-installation-pull-secrets\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:33.333891 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.333849    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ldlx\" (UniqueName: \"kubernetes.io/projected/7b436a96-7884-43ef-befb-4cff72a9bc6a-kube-api-access-5ldlx\") pod \"network-check-source-65bdbbff45-wt47b\" (UID: \"7b436a96-7884-43ef-befb-4cff72a9bc6a\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-wt47b"
Aug 04 17:52:33.333891 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.333873    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f583466f-47c3-4403-a048-4c48550ac4fc-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-stb8t\" (UID: \"f583466f-47c3-4403-a048-4c48550ac4fc\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-stb8t"
Aug 04 17:52:33.334137 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.333904    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsf4w\" (UniqueName: \"kubernetes.io/projected/b35d3198-f4e3-44a5-a0b7-08c987b5805f-kube-api-access-gsf4w\") pod \"router-default-5955597dc4-jbsv8\" (UID: \"b35d3198-f4e3-44a5-a0b7-08c987b5805f\") " pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:52:33.334137 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.333928    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b29597f-59c4-4f28-a164-7d2e97248d83-tmp\") pod \"insights-operator-77696685b-ks7s5\" (UID: \"6b29597f-59c4-4f28-a164-7d2e97248d83\") " pod="openshift-insights/insights-operator-77696685b-ks7s5"
Aug 04 17:52:33.334137 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.333948    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6b29597f-59c4-4f28-a164-7d2e97248d83-serving-cert\") pod \"insights-operator-77696685b-ks7s5\" (UID: \"6b29597f-59c4-4f28-a164-7d2e97248d83\") " pod="openshift-insights/insights-operator-77696685b-ks7s5"
Aug 04 17:52:33.334137 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.334003    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/165249a4-0e3d-4f20-9719-c91a6d7ce78f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-kxp92\" (UID: \"165249a4-0e3d-4f20-9719-c91a6d7ce78f\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-kxp92"
Aug 04 17:52:33.334137 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.334025    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b35d3198-f4e3-44a5-a0b7-08c987b5805f-stats-auth\") pod \"router-default-5955597dc4-jbsv8\" (UID: \"b35d3198-f4e3-44a5-a0b7-08c987b5805f\") " pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:52:33.334137 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.334051    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/887c097d-64f3-4f3f-99ac-7981ab483724-registry-certificates\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:33.334137 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.334071    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6b29597f-59c4-4f28-a164-7d2e97248d83-snapshots\") pod \"insights-operator-77696685b-ks7s5\" (UID: \"6b29597f-59c4-4f28-a164-7d2e97248d83\") " pod="openshift-insights/insights-operator-77696685b-ks7s5"
Aug 04 17:52:33.334137 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.334101    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmc2b\" (UniqueName: \"kubernetes.io/projected/f49077e4-25d7-4cf7-8072-87c5a5e13829-kube-api-access-nmc2b\") pod \"cluster-samples-operator-559cbf5745-t4zw5\" (UID: \"f49077e4-25d7-4cf7-8072-87c5a5e13829\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-t4zw5"
Aug 04 17:52:33.334137 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.334125    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sj6s\" (UniqueName: \"kubernetes.io/projected/0a01b635-81f5-4c48-9906-dbf914e08592-kube-api-access-8sj6s\") pod \"dns-default-d58jd\" (UID: \"0a01b635-81f5-4c48-9906-dbf914e08592\") " pod="openshift-dns/dns-default-d58jd"
Aug 04 17:52:33.334558 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:33.334169    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b35d3198-f4e3-44a5-a0b7-08c987b5805f-service-ca-bundle podName:b35d3198-f4e3-44a5-a0b7-08c987b5805f nodeName:}" failed. No retries permitted until 2026-08-04 17:52:33.834151711 +0000 UTC m=+36.017311883 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b35d3198-f4e3-44a5-a0b7-08c987b5805f-service-ca-bundle") pod "router-default-5955597dc4-jbsv8" (UID: "b35d3198-f4e3-44a5-a0b7-08c987b5805f") : configmap references non-existent config key: service-ca.crt
Aug 04 17:52:33.334558 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.334172    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctjdb\" (UniqueName: \"kubernetes.io/projected/f3c4ae8d-6d40-466c-a97c-daab363b3502-kube-api-access-ctjdb\") pod \"service-ca-operator-784f97bcf6-n64q5\" (UID: \"f3c4ae8d-6d40-466c-a97c-daab363b3502\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-n64q5"
Aug 04 17:52:33.334558 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:33.334530    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 04 17:52:33.334742 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.334557    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-registry-tls\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:33.334742 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:33.334590    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/165249a4-0e3d-4f20-9719-c91a6d7ce78f-cluster-monitoring-operator-tls podName:165249a4-0e3d-4f20-9719-c91a6d7ce78f nodeName:}" failed. No retries permitted until 2026-08-04 17:52:33.834574117 +0000 UTC m=+36.017734284 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/165249a4-0e3d-4f20-9719-c91a6d7ce78f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-kxp92" (UID: "165249a4-0e3d-4f20-9719-c91a6d7ce78f") : secret "cluster-monitoring-operator-tls" not found
Aug 04 17:52:33.334742 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.334687    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/887c097d-64f3-4f3f-99ac-7981ab483724-trusted-ca\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:33.334742 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.334705    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76442945-00cd-4bbd-8dc1-a65a9573b031-cert\") pod \"ingress-canary-5s8jf\" (UID: \"76442945-00cd-4bbd-8dc1-a65a9573b031\") " pod="openshift-ingress-canary/ingress-canary-5s8jf"
Aug 04 17:52:33.334958 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:33.334792    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 17:52:33.334958 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:33.334807    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69bc56d479-dx5kz: secret "image-registry-tls" not found
Aug 04 17:52:33.334958 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:33.334882    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-registry-tls podName:887c097d-64f3-4f3f-99ac-7981ab483724 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:33.834866991 +0000 UTC m=+36.018027145 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-registry-tls") pod "image-registry-69bc56d479-dx5kz" (UID: "887c097d-64f3-4f3f-99ac-7981ab483724") : secret "image-registry-tls" not found
Aug 04 17:52:33.334958 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.334933    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/165249a4-0e3d-4f20-9719-c91a6d7ce78f-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-kxp92\" (UID: \"165249a4-0e3d-4f20-9719-c91a6d7ce78f\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-kxp92"
Aug 04 17:52:33.335173 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.335020    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9brr\" (UniqueName: \"kubernetes.io/projected/6b29597f-59c4-4f28-a164-7d2e97248d83-kube-api-access-t9brr\") pod \"insights-operator-77696685b-ks7s5\" (UID: \"6b29597f-59c4-4f28-a164-7d2e97248d83\") " pod="openshift-insights/insights-operator-77696685b-ks7s5"
Aug 04 17:52:33.335173 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.335051    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a01b635-81f5-4c48-9906-dbf914e08592-tmp-dir\") pod \"dns-default-d58jd\" (UID: \"0a01b635-81f5-4c48-9906-dbf914e08592\") " pod="openshift-dns/dns-default-d58jd"
Aug 04 17:52:33.335173 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.335078    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljf9z\" (UniqueName: \"kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-kube-api-access-ljf9z\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:33.335173 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.335116    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8eaf7533-2fc5-4da2-bec2-282dce644049-serving-cert\") pod \"console-operator-5ccd6b445f-gtwkf\" (UID: \"8eaf7533-2fc5-4da2-bec2-282dce644049\") " pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf"
Aug 04 17:52:33.335173 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.335141    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8eaf7533-2fc5-4da2-bec2-282dce644049-trusted-ca\") pod \"console-operator-5ccd6b445f-gtwkf\" (UID: \"8eaf7533-2fc5-4da2-bec2-282dce644049\") " pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf"
Aug 04 17:52:33.335173 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.335168    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f3c4ae8d-6d40-466c-a97c-daab363b3502-serving-cert\") pod \"service-ca-operator-784f97bcf6-n64q5\" (UID: \"f3c4ae8d-6d40-466c-a97c-daab363b3502\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-n64q5"
Aug 04 17:52:33.335469 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.335371    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/887c097d-64f3-4f3f-99ac-7981ab483724-ca-trust-extracted\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:33.335469 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.335410    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b29597f-59c4-4f28-a164-7d2e97248d83-service-ca-bundle\") pod \"insights-operator-77696685b-ks7s5\" (UID: \"6b29597f-59c4-4f28-a164-7d2e97248d83\") " pod="openshift-insights/insights-operator-77696685b-ks7s5"
Aug 04 17:52:33.335469 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.335404    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/887c097d-64f3-4f3f-99ac-7981ab483724-registry-certificates\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:33.335469 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.335444    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a01b635-81f5-4c48-9906-dbf914e08592-metrics-tls\") pod \"dns-default-d58jd\" (UID: \"0a01b635-81f5-4c48-9906-dbf914e08592\") " pod="openshift-dns/dns-default-d58jd"
Aug 04 17:52:33.335666 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.335496    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8eaf7533-2fc5-4da2-bec2-282dce644049-config\") pod \"console-operator-5ccd6b445f-gtwkf\" (UID: \"8eaf7533-2fc5-4da2-bec2-282dce644049\") " pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf"
Aug 04 17:52:33.335666 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.335560    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qchbb\" (UniqueName: \"kubernetes.io/projected/3f8891ae-cf02-49a7-9144-ef455be4cdee-kube-api-access-qchbb\") pod \"kube-storage-version-migrator-operator-7878f4b88c-47zwf\" (UID: \"3f8891ae-cf02-49a7-9144-ef455be4cdee\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-47zwf"
Aug 04 17:52:33.335666 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.335612    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b35d3198-f4e3-44a5-a0b7-08c987b5805f-default-certificate\") pod \"router-default-5955597dc4-jbsv8\" (UID: \"b35d3198-f4e3-44a5-a0b7-08c987b5805f\") " pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:52:33.335666 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.335647    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3f8891ae-cf02-49a7-9144-ef455be4cdee-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-47zwf\" (UID: \"3f8891ae-cf02-49a7-9144-ef455be4cdee\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-47zwf"
Aug 04 17:52:33.335853 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.335837    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/887c097d-64f3-4f3f-99ac-7981ab483724-ca-trust-extracted\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:33.335910 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.335838    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/165249a4-0e3d-4f20-9719-c91a6d7ce78f-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-kxp92\" (UID: \"165249a4-0e3d-4f20-9719-c91a6d7ce78f\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-kxp92"
Aug 04 17:52:33.338346 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.338321    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/887c097d-64f3-4f3f-99ac-7981ab483724-installation-pull-secrets\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:33.338450 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.338355    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b35d3198-f4e3-44a5-a0b7-08c987b5805f-stats-auth\") pod \"router-default-5955597dc4-jbsv8\" (UID: \"b35d3198-f4e3-44a5-a0b7-08c987b5805f\") " pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:52:33.338450 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.338328    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b35d3198-f4e3-44a5-a0b7-08c987b5805f-default-certificate\") pod \"router-default-5955597dc4-jbsv8\" (UID: \"b35d3198-f4e3-44a5-a0b7-08c987b5805f\") " pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:52:33.338450 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.338324    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/887c097d-64f3-4f3f-99ac-7981ab483724-image-registry-private-configuration\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:33.343236 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.343209    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pww7\" (UniqueName: \"kubernetes.io/projected/90aff035-abe4-4fcf-bcd6-76f25a1061dc-kube-api-access-9pww7\") pod \"volume-data-source-validator-5b8776669-jkhcl\" (UID: \"90aff035-abe4-4fcf-bcd6-76f25a1061dc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-jkhcl"
Aug 04 17:52:33.343344 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.343243    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-bound-sa-token\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:33.345652 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.345629    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p746h\" (UniqueName: \"kubernetes.io/projected/165249a4-0e3d-4f20-9719-c91a6d7ce78f-kube-api-access-p746h\") pod \"cluster-monitoring-operator-6f55c4f58d-kxp92\" (UID: \"165249a4-0e3d-4f20-9719-c91a6d7ce78f\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-kxp92"
Aug 04 17:52:33.345788 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.345683    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsf4w\" (UniqueName: \"kubernetes.io/projected/b35d3198-f4e3-44a5-a0b7-08c987b5805f-kube-api-access-gsf4w\") pod \"router-default-5955597dc4-jbsv8\" (UID: \"b35d3198-f4e3-44a5-a0b7-08c987b5805f\") " pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:52:33.346243 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.346223    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljf9z\" (UniqueName: \"kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-kube-api-access-ljf9z\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:33.347243 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.347201    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:52:33.347345 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.347328    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:52:33.347407 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.347355    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:52:33.349777 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.349758    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 04 17:52:33.350272 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.350253    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-f7jt5\""
Aug 04 17:52:33.350522 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.350504    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 04 17:52:33.350611 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.350537    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-smv6k\""
Aug 04 17:52:33.357298 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.356361    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:33.436838 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.436804    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8eaf7533-2fc5-4da2-bec2-282dce644049-config\") pod \"console-operator-5ccd6b445f-gtwkf\" (UID: \"8eaf7533-2fc5-4da2-bec2-282dce644049\") " pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf"
Aug 04 17:52:33.436982 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.436858    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qchbb\" (UniqueName: \"kubernetes.io/projected/3f8891ae-cf02-49a7-9144-ef455be4cdee-kube-api-access-qchbb\") pod \"kube-storage-version-migrator-operator-7878f4b88c-47zwf\" (UID: \"3f8891ae-cf02-49a7-9144-ef455be4cdee\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-47zwf"
Aug 04 17:52:33.436982 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.436885    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3f8891ae-cf02-49a7-9144-ef455be4cdee-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-47zwf\" (UID: \"3f8891ae-cf02-49a7-9144-ef455be4cdee\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-47zwf"
Aug 04 17:52:33.436982 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.436900    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4tqd\" (UniqueName: \"kubernetes.io/projected/76442945-00cd-4bbd-8dc1-a65a9573b031-kube-api-access-b4tqd\") pod \"ingress-canary-5s8jf\" (UID: \"76442945-00cd-4bbd-8dc1-a65a9573b031\") " pod="openshift-ingress-canary/ingress-canary-5s8jf"
Aug 04 17:52:33.436982 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.436925    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3f8891ae-cf02-49a7-9144-ef455be4cdee-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-47zwf\" (UID: \"3f8891ae-cf02-49a7-9144-ef455be4cdee\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-47zwf"
Aug 04 17:52:33.436982 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.436951    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f49077e4-25d7-4cf7-8072-87c5a5e13829-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-t4zw5\" (UID: \"f49077e4-25d7-4cf7-8072-87c5a5e13829\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-t4zw5"
Aug 04 17:52:33.436982 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.436975    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f3c4ae8d-6d40-466c-a97c-daab363b3502-config\") pod \"service-ca-operator-784f97bcf6-n64q5\" (UID: \"f3c4ae8d-6d40-466c-a97c-daab363b3502\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-n64q5"
Aug 04 17:52:33.437298 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.437018    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f583466f-47c3-4403-a048-4c48550ac4fc-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-stb8t\" (UID: \"f583466f-47c3-4403-a048-4c48550ac4fc\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-stb8t"
Aug 04 17:52:33.437298 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.437054    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b29597f-59c4-4f28-a164-7d2e97248d83-trusted-ca-bundle\") pod \"insights-operator-77696685b-ks7s5\" (UID: \"6b29597f-59c4-4f28-a164-7d2e97248d83\") " pod="openshift-insights/insights-operator-77696685b-ks7s5"
Aug 04 17:52:33.437298 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.437084    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf44n\" (UniqueName: \"kubernetes.io/projected/8eaf7533-2fc5-4da2-bec2-282dce644049-kube-api-access-jf44n\") pod \"console-operator-5ccd6b445f-gtwkf\" (UID: \"8eaf7533-2fc5-4da2-bec2-282dce644049\") " pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf"
Aug 04 17:52:33.437298 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.437153    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0a01b635-81f5-4c48-9906-dbf914e08592-config-volume\") pod \"dns-default-d58jd\" (UID: \"0a01b635-81f5-4c48-9906-dbf914e08592\") " pod="openshift-dns/dns-default-d58jd"
Aug 04 17:52:33.437298 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.437181    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ldlx\" (UniqueName: \"kubernetes.io/projected/7b436a96-7884-43ef-befb-4cff72a9bc6a-kube-api-access-5ldlx\") pod \"network-check-source-65bdbbff45-wt47b\" (UID: \"7b436a96-7884-43ef-befb-4cff72a9bc6a\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-wt47b"
Aug 04 17:52:33.437298 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.437216    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f583466f-47c3-4403-a048-4c48550ac4fc-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-stb8t\" (UID: \"f583466f-47c3-4403-a048-4c48550ac4fc\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-stb8t"
Aug 04 17:52:33.437298 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.437255    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b29597f-59c4-4f28-a164-7d2e97248d83-tmp\") pod \"insights-operator-77696685b-ks7s5\" (UID: \"6b29597f-59c4-4f28-a164-7d2e97248d83\") " pod="openshift-insights/insights-operator-77696685b-ks7s5"
Aug 04 17:52:33.437298 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.437280    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6b29597f-59c4-4f28-a164-7d2e97248d83-serving-cert\") pod \"insights-operator-77696685b-ks7s5\" (UID: \"6b29597f-59c4-4f28-a164-7d2e97248d83\") " pod="openshift-insights/insights-operator-77696685b-ks7s5"
Aug 04 17:52:33.437604 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.437332    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6b29597f-59c4-4f28-a164-7d2e97248d83-snapshots\") pod \"insights-operator-77696685b-ks7s5\" (UID: \"6b29597f-59c4-4f28-a164-7d2e97248d83\") " pod="openshift-insights/insights-operator-77696685b-ks7s5"
Aug 04 17:52:33.437604 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.437363    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmc2b\" (UniqueName: \"kubernetes.io/projected/f49077e4-25d7-4cf7-8072-87c5a5e13829-kube-api-access-nmc2b\") pod \"cluster-samples-operator-559cbf5745-t4zw5\" (UID: \"f49077e4-25d7-4cf7-8072-87c5a5e13829\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-t4zw5"
Aug 04 17:52:33.437604 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.437389    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sj6s\" (UniqueName: \"kubernetes.io/projected/0a01b635-81f5-4c48-9906-dbf914e08592-kube-api-access-8sj6s\") pod \"dns-default-d58jd\" (UID: \"0a01b635-81f5-4c48-9906-dbf914e08592\") " pod="openshift-dns/dns-default-d58jd"
Aug 04 17:52:33.437604 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.437415    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctjdb\" (UniqueName: \"kubernetes.io/projected/f3c4ae8d-6d40-466c-a97c-daab363b3502-kube-api-access-ctjdb\") pod \"service-ca-operator-784f97bcf6-n64q5\" (UID: \"f3c4ae8d-6d40-466c-a97c-daab363b3502\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-n64q5"
Aug 04 17:52:33.437604 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.437455    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76442945-00cd-4bbd-8dc1-a65a9573b031-cert\") pod \"ingress-canary-5s8jf\" (UID: \"76442945-00cd-4bbd-8dc1-a65a9573b031\") " pod="openshift-ingress-canary/ingress-canary-5s8jf"
Aug 04 17:52:33.437604 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.437487    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9brr\" (UniqueName: \"kubernetes.io/projected/6b29597f-59c4-4f28-a164-7d2e97248d83-kube-api-access-t9brr\") pod \"insights-operator-77696685b-ks7s5\" (UID: \"6b29597f-59c4-4f28-a164-7d2e97248d83\") " pod="openshift-insights/insights-operator-77696685b-ks7s5"
Aug 04 17:52:33.437604 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.437511    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a01b635-81f5-4c48-9906-dbf914e08592-tmp-dir\") pod \"dns-default-d58jd\" (UID: \"0a01b635-81f5-4c48-9906-dbf914e08592\") " pod="openshift-dns/dns-default-d58jd"
Aug 04 17:52:33.437604 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.437517    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3f8891ae-cf02-49a7-9144-ef455be4cdee-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-47zwf\" (UID: \"3f8891ae-cf02-49a7-9144-ef455be4cdee\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-47zwf"
Aug 04 17:52:33.437604 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.437527    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8eaf7533-2fc5-4da2-bec2-282dce644049-config\") pod \"console-operator-5ccd6b445f-gtwkf\" (UID: \"8eaf7533-2fc5-4da2-bec2-282dce644049\") " pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf"
Aug 04 17:52:33.437604 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.437535    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8eaf7533-2fc5-4da2-bec2-282dce644049-serving-cert\") pod \"console-operator-5ccd6b445f-gtwkf\" (UID: \"8eaf7533-2fc5-4da2-bec2-282dce644049\") " pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf"
Aug 04 17:52:33.437604 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.437559    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8eaf7533-2fc5-4da2-bec2-282dce644049-trusted-ca\") pod \"console-operator-5ccd6b445f-gtwkf\" (UID: \"8eaf7533-2fc5-4da2-bec2-282dce644049\") " pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf"
Aug 04 17:52:33.438069 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.437721    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f3c4ae8d-6d40-466c-a97c-daab363b3502-config\") pod \"service-ca-operator-784f97bcf6-n64q5\" (UID: \"f3c4ae8d-6d40-466c-a97c-daab363b3502\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-n64q5"
Aug 04 17:52:33.438069 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:33.437806    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 04 17:52:33.438069 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:33.437859    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f583466f-47c3-4403-a048-4c48550ac4fc-networking-console-plugin-cert podName:f583466f-47c3-4403-a048-4c48550ac4fc nodeName:}" failed. No retries permitted until 2026-08-04 17:52:33.937842154 +0000 UTC m=+36.121002329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f583466f-47c3-4403-a048-4c48550ac4fc-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-stb8t" (UID: "f583466f-47c3-4403-a048-4c48550ac4fc") : secret "networking-console-plugin-cert" not found
Aug 04 17:52:33.438069 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.437888    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b29597f-59c4-4f28-a164-7d2e97248d83-tmp\") pod \"insights-operator-77696685b-ks7s5\" (UID: \"6b29597f-59c4-4f28-a164-7d2e97248d83\") " pod="openshift-insights/insights-operator-77696685b-ks7s5"
Aug 04 17:52:33.438069 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:33.438061    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Aug 04 17:52:33.438312 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:33.438121    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f49077e4-25d7-4cf7-8072-87c5a5e13829-samples-operator-tls podName:f49077e4-25d7-4cf7-8072-87c5a5e13829 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:33.938101234 +0000 UTC m=+36.121261402 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f49077e4-25d7-4cf7-8072-87c5a5e13829-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-t4zw5" (UID: "f49077e4-25d7-4cf7-8072-87c5a5e13829") : secret "samples-operator-tls" not found
Aug 04 17:52:33.438312 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.438162    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f3c4ae8d-6d40-466c-a97c-daab363b3502-serving-cert\") pod \"service-ca-operator-784f97bcf6-n64q5\" (UID: \"f3c4ae8d-6d40-466c-a97c-daab363b3502\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-n64q5"
Aug 04 17:52:33.438312 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.438270    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a01b635-81f5-4c48-9906-dbf914e08592-tmp-dir\") pod \"dns-default-d58jd\" (UID: \"0a01b635-81f5-4c48-9906-dbf914e08592\") " pod="openshift-dns/dns-default-d58jd"
Aug 04 17:52:33.438563 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.438542    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b29597f-59c4-4f28-a164-7d2e97248d83-service-ca-bundle\") pod \"insights-operator-77696685b-ks7s5\" (UID: \"6b29597f-59c4-4f28-a164-7d2e97248d83\") " pod="openshift-insights/insights-operator-77696685b-ks7s5"
Aug 04 17:52:33.438644 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.438592    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a01b635-81f5-4c48-9906-dbf914e08592-metrics-tls\") pod \"dns-default-d58jd\" (UID: \"0a01b635-81f5-4c48-9906-dbf914e08592\") " pod="openshift-dns/dns-default-d58jd"
Aug 04 17:52:33.438711 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:33.438696    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 17:52:33.438767 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:33.438747    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a01b635-81f5-4c48-9906-dbf914e08592-metrics-tls podName:0a01b635-81f5-4c48-9906-dbf914e08592 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:33.938729731 +0000 UTC m=+36.121889901 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a01b635-81f5-4c48-9906-dbf914e08592-metrics-tls") pod "dns-default-d58jd" (UID: "0a01b635-81f5-4c48-9906-dbf914e08592") : secret "dns-default-metrics-tls" not found
Aug 04 17:52:33.438841 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.438818    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b29597f-59c4-4f28-a164-7d2e97248d83-trusted-ca-bundle\") pod \"insights-operator-77696685b-ks7s5\" (UID: \"6b29597f-59c4-4f28-a164-7d2e97248d83\") " pod="openshift-insights/insights-operator-77696685b-ks7s5"
Aug 04 17:52:33.438911 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.438882    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8eaf7533-2fc5-4da2-bec2-282dce644049-trusted-ca\") pod \"console-operator-5ccd6b445f-gtwkf\" (UID: \"8eaf7533-2fc5-4da2-bec2-282dce644049\") " pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf"
Aug 04 17:52:33.438964 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.438886    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0a01b635-81f5-4c48-9906-dbf914e08592-config-volume\") pod \"dns-default-d58jd\" (UID: \"0a01b635-81f5-4c48-9906-dbf914e08592\") " pod="openshift-dns/dns-default-d58jd"
Aug 04 17:52:33.438964 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.438952    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f583466f-47c3-4403-a048-4c48550ac4fc-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-stb8t\" (UID: \"f583466f-47c3-4403-a048-4c48550ac4fc\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-stb8t"
Aug 04 17:52:33.439218 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.439199    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b29597f-59c4-4f28-a164-7d2e97248d83-service-ca-bundle\") pod \"insights-operator-77696685b-ks7s5\" (UID: \"6b29597f-59c4-4f28-a164-7d2e97248d83\") " pod="openshift-insights/insights-operator-77696685b-ks7s5"
Aug 04 17:52:33.439354 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.439334    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6b29597f-59c4-4f28-a164-7d2e97248d83-snapshots\") pod \"insights-operator-77696685b-ks7s5\" (UID: \"6b29597f-59c4-4f28-a164-7d2e97248d83\") " pod="openshift-insights/insights-operator-77696685b-ks7s5"
Aug 04 17:52:33.440652 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.440628    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-jkhcl"
Aug 04 17:52:33.440734 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.440646    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8eaf7533-2fc5-4da2-bec2-282dce644049-serving-cert\") pod \"console-operator-5ccd6b445f-gtwkf\" (UID: \"8eaf7533-2fc5-4da2-bec2-282dce644049\") " pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf"
Aug 04 17:52:33.440897 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.440872    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6b29597f-59c4-4f28-a164-7d2e97248d83-serving-cert\") pod \"insights-operator-77696685b-ks7s5\" (UID: \"6b29597f-59c4-4f28-a164-7d2e97248d83\") " pod="openshift-insights/insights-operator-77696685b-ks7s5"
Aug 04 17:52:33.441729 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.441699    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3f8891ae-cf02-49a7-9144-ef455be4cdee-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-47zwf\" (UID: \"3f8891ae-cf02-49a7-9144-ef455be4cdee\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-47zwf"
Aug 04 17:52:33.441863 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.441841    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f3c4ae8d-6d40-466c-a97c-daab363b3502-serving-cert\") pod \"service-ca-operator-784f97bcf6-n64q5\" (UID: \"f3c4ae8d-6d40-466c-a97c-daab363b3502\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-n64q5"
Aug 04 17:52:33.441924 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.441907    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76442945-00cd-4bbd-8dc1-a65a9573b031-cert\") pod \"ingress-canary-5s8jf\" (UID: \"76442945-00cd-4bbd-8dc1-a65a9573b031\") " pod="openshift-ingress-canary/ingress-canary-5s8jf"
Aug 04 17:52:33.450415 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.450390    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9brr\" (UniqueName: \"kubernetes.io/projected/6b29597f-59c4-4f28-a164-7d2e97248d83-kube-api-access-t9brr\") pod \"insights-operator-77696685b-ks7s5\" (UID: \"6b29597f-59c4-4f28-a164-7d2e97248d83\") " pod="openshift-insights/insights-operator-77696685b-ks7s5"
Aug 04 17:52:33.450515 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.450426    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctjdb\" (UniqueName: \"kubernetes.io/projected/f3c4ae8d-6d40-466c-a97c-daab363b3502-kube-api-access-ctjdb\") pod \"service-ca-operator-784f97bcf6-n64q5\" (UID: \"f3c4ae8d-6d40-466c-a97c-daab363b3502\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-n64q5"
Aug 04 17:52:33.450755 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.450738    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qchbb\" (UniqueName: \"kubernetes.io/projected/3f8891ae-cf02-49a7-9144-ef455be4cdee-kube-api-access-qchbb\") pod \"kube-storage-version-migrator-operator-7878f4b88c-47zwf\" (UID: \"3f8891ae-cf02-49a7-9144-ef455be4cdee\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-47zwf"
Aug 04 17:52:33.451833 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.451787    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4tqd\" (UniqueName: \"kubernetes.io/projected/76442945-00cd-4bbd-8dc1-a65a9573b031-kube-api-access-b4tqd\") pod \"ingress-canary-5s8jf\" (UID: \"76442945-00cd-4bbd-8dc1-a65a9573b031\") " pod="openshift-ingress-canary/ingress-canary-5s8jf"
Aug 04 17:52:33.452963 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.452939    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sj6s\" (UniqueName: \"kubernetes.io/projected/0a01b635-81f5-4c48-9906-dbf914e08592-kube-api-access-8sj6s\") pod \"dns-default-d58jd\" (UID: \"0a01b635-81f5-4c48-9906-dbf914e08592\") " pod="openshift-dns/dns-default-d58jd"
Aug 04 17:52:33.453498 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.453480    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ldlx\" (UniqueName: \"kubernetes.io/projected/7b436a96-7884-43ef-befb-4cff72a9bc6a-kube-api-access-5ldlx\") pod \"network-check-source-65bdbbff45-wt47b\" (UID: \"7b436a96-7884-43ef-befb-4cff72a9bc6a\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-wt47b"
Aug 04 17:52:33.453882 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.453863    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmc2b\" (UniqueName: \"kubernetes.io/projected/f49077e4-25d7-4cf7-8072-87c5a5e13829-kube-api-access-nmc2b\") pod \"cluster-samples-operator-559cbf5745-t4zw5\" (UID: \"f49077e4-25d7-4cf7-8072-87c5a5e13829\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-t4zw5"
Aug 04 17:52:33.455841 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.455817    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf44n\" (UniqueName: \"kubernetes.io/projected/8eaf7533-2fc5-4da2-bec2-282dce644049-kube-api-access-jf44n\") pod \"console-operator-5ccd6b445f-gtwkf\" (UID: \"8eaf7533-2fc5-4da2-bec2-282dce644049\") " pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf"
Aug 04 17:52:33.462351 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.462268    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:33.488246 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.488225    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-ks7s5"
Aug 04 17:52:33.501292 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.501267    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-n64q5"
Aug 04 17:52:33.508355 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.508326    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-47zwf"
Aug 04 17:52:33.515050 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.515028    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf"
Aug 04 17:52:33.529010 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.528979    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-wt47b"
Aug 04 17:52:33.543967 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.543941    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5s8jf"
Aug 04 17:52:33.842006 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.841956    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/165249a4-0e3d-4f20-9719-c91a6d7ce78f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-kxp92\" (UID: \"165249a4-0e3d-4f20-9719-c91a6d7ce78f\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-kxp92"
Aug 04 17:52:33.842183 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.842039    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-registry-tls\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:33.842183 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.842119    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b35d3198-f4e3-44a5-a0b7-08c987b5805f-metrics-certs\") pod \"router-default-5955597dc4-jbsv8\" (UID: \"b35d3198-f4e3-44a5-a0b7-08c987b5805f\") " pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:52:33.842183 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:33.842123    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 04 17:52:33.842183 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.842168    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b35d3198-f4e3-44a5-a0b7-08c987b5805f-service-ca-bundle\") pod \"router-default-5955597dc4-jbsv8\" (UID: \"b35d3198-f4e3-44a5-a0b7-08c987b5805f\") " pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:52:33.842316 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:33.842187    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 17:52:33.842316 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:33.842207    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69bc56d479-dx5kz: secret "image-registry-tls" not found
Aug 04 17:52:33.842316 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:33.842198    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/165249a4-0e3d-4f20-9719-c91a6d7ce78f-cluster-monitoring-operator-tls podName:165249a4-0e3d-4f20-9719-c91a6d7ce78f nodeName:}" failed. No retries permitted until 2026-08-04 17:52:34.842178178 +0000 UTC m=+37.025338356 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/165249a4-0e3d-4f20-9719-c91a6d7ce78f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-kxp92" (UID: "165249a4-0e3d-4f20-9719-c91a6d7ce78f") : secret "cluster-monitoring-operator-tls" not found
Aug 04 17:52:33.842316 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:33.842260    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 04 17:52:33.842316 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:33.842262    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b35d3198-f4e3-44a5-a0b7-08c987b5805f-service-ca-bundle podName:b35d3198-f4e3-44a5-a0b7-08c987b5805f nodeName:}" failed. No retries permitted until 2026-08-04 17:52:34.842246367 +0000 UTC m=+37.025406526 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b35d3198-f4e3-44a5-a0b7-08c987b5805f-service-ca-bundle") pod "router-default-5955597dc4-jbsv8" (UID: "b35d3198-f4e3-44a5-a0b7-08c987b5805f") : configmap references non-existent config key: service-ca.crt
Aug 04 17:52:33.842316 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:33.842294    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-registry-tls podName:887c097d-64f3-4f3f-99ac-7981ab483724 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:34.842284426 +0000 UTC m=+37.025444580 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-registry-tls") pod "image-registry-69bc56d479-dx5kz" (UID: "887c097d-64f3-4f3f-99ac-7981ab483724") : secret "image-registry-tls" not found
Aug 04 17:52:33.842316 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:33.842310    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b35d3198-f4e3-44a5-a0b7-08c987b5805f-metrics-certs podName:b35d3198-f4e3-44a5-a0b7-08c987b5805f nodeName:}" failed. No retries permitted until 2026-08-04 17:52:34.842303888 +0000 UTC m=+37.025464043 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b35d3198-f4e3-44a5-a0b7-08c987b5805f-metrics-certs") pod "router-default-5955597dc4-jbsv8" (UID: "b35d3198-f4e3-44a5-a0b7-08c987b5805f") : secret "router-metrics-certs-default" not found
Aug 04 17:52:33.943180 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.943146    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f49077e4-25d7-4cf7-8072-87c5a5e13829-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-t4zw5\" (UID: \"f49077e4-25d7-4cf7-8072-87c5a5e13829\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-t4zw5"
Aug 04 17:52:33.943499 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.943303    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f583466f-47c3-4403-a048-4c48550ac4fc-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-stb8t\" (UID: \"f583466f-47c3-4403-a048-4c48550ac4fc\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-stb8t"
Aug 04 17:52:33.943499 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:33.943435    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 04 17:52:33.943499 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:33.943435    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Aug 04 17:52:33.943687 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:33.943505    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f49077e4-25d7-4cf7-8072-87c5a5e13829-samples-operator-tls podName:f49077e4-25d7-4cf7-8072-87c5a5e13829 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:34.943485864 +0000 UTC m=+37.126646032 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f49077e4-25d7-4cf7-8072-87c5a5e13829-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-t4zw5" (UID: "f49077e4-25d7-4cf7-8072-87c5a5e13829") : secret "samples-operator-tls" not found
Aug 04 17:52:33.943687 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:33.943581    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f583466f-47c3-4403-a048-4c48550ac4fc-networking-console-plugin-cert podName:f583466f-47c3-4403-a048-4c48550ac4fc nodeName:}" failed. No retries permitted until 2026-08-04 17:52:34.943554354 +0000 UTC m=+37.126714522 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f583466f-47c3-4403-a048-4c48550ac4fc-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-stb8t" (UID: "f583466f-47c3-4403-a048-4c48550ac4fc") : secret "networking-console-plugin-cert" not found
Aug 04 17:52:33.943687 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:33.943672    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a01b635-81f5-4c48-9906-dbf914e08592-metrics-tls\") pod \"dns-default-d58jd\" (UID: \"0a01b635-81f5-4c48-9906-dbf914e08592\") " pod="openshift-dns/dns-default-d58jd"
Aug 04 17:52:33.943838 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:33.943821    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 17:52:33.943890 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:33.943878    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a01b635-81f5-4c48-9906-dbf914e08592-metrics-tls podName:0a01b635-81f5-4c48-9906-dbf914e08592 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:34.943864763 +0000 UTC m=+37.127024920 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a01b635-81f5-4c48-9906-dbf914e08592-metrics-tls") pod "dns-default-d58jd" (UID: "0a01b635-81f5-4c48-9906-dbf914e08592") : secret "dns-default-metrics-tls" not found
Aug 04 17:52:34.201309 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:34.201112    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-n64q5"]
Aug 04 17:52:34.204456 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:52:34.204430    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3c4ae8d_6d40_466c_a97c_daab363b3502.slice/crio-05a6b3c0fb33451d75f116828efcb0bd89d2db3a95e77843bbbb4f9d53c9273d WatchSource:0}: Error finding container 05a6b3c0fb33451d75f116828efcb0bd89d2db3a95e77843bbbb4f9d53c9273d: Status 404 returned error can't find the container with id 05a6b3c0fb33451d75f116828efcb0bd89d2db3a95e77843bbbb4f9d53c9273d
Aug 04 17:52:34.212761 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:34.212739    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5s8jf"]
Aug 04 17:52:34.215774 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:34.215754    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-ks7s5"]
Aug 04 17:52:34.216022 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:52:34.215981    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76442945_00cd_4bbd_8dc1_a65a9573b031.slice/crio-26ce089ec65d0a54f1120eddf2d76425603d17a40698c01b42ef63c46c4aee27 WatchSource:0}: Error finding container 26ce089ec65d0a54f1120eddf2d76425603d17a40698c01b42ef63c46c4aee27: Status 404 returned error can't find the container with id 26ce089ec65d0a54f1120eddf2d76425603d17a40698c01b42ef63c46c4aee27
Aug 04 17:52:34.217525 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:34.217478    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-jkhcl"]
Aug 04 17:52:34.218534 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:34.218510    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-wt47b"]
Aug 04 17:52:34.225453 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:34.225395    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-47zwf"]
Aug 04 17:52:34.227104 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:52:34.227085    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b436a96_7884_43ef_befb_4cff72a9bc6a.slice/crio-db5df6a1f91af9502dc88d9e326fa4dc09c7ad70a57aafc164cf071c2374b53d WatchSource:0}: Error finding container db5df6a1f91af9502dc88d9e326fa4dc09c7ad70a57aafc164cf071c2374b53d: Status 404 returned error can't find the container with id db5df6a1f91af9502dc88d9e326fa4dc09c7ad70a57aafc164cf071c2374b53d
Aug 04 17:52:34.227324 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:52:34.227303    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b29597f_59c4_4f28_a164_7d2e97248d83.slice/crio-8bfd40b93ccd9787515c3fc3abfa83bfabb0f7b8e2ee9433cd728f3a23ca04eb WatchSource:0}: Error finding container 8bfd40b93ccd9787515c3fc3abfa83bfabb0f7b8e2ee9433cd728f3a23ca04eb: Status 404 returned error can't find the container with id 8bfd40b93ccd9787515c3fc3abfa83bfabb0f7b8e2ee9433cd728f3a23ca04eb
Aug 04 17:52:34.228157 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:52:34.228139    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90aff035_abe4_4fcf_bcd6_76f25a1061dc.slice/crio-f744cc852c1123fa2c276ac85f6f1a6dba4f6d1448a97d45e326bfb9ec8cef04 WatchSource:0}: Error finding container f744cc852c1123fa2c276ac85f6f1a6dba4f6d1448a97d45e326bfb9ec8cef04: Status 404 returned error can't find the container with id f744cc852c1123fa2c276ac85f6f1a6dba4f6d1448a97d45e326bfb9ec8cef04
Aug 04 17:52:34.228947 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:52:34.228925    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f8891ae_cf02_49a7_9144_ef455be4cdee.slice/crio-f8cf65c5232cb4a6fcc177bfee634bf9604200eb07ede3dc41b49047c1fd10a0 WatchSource:0}: Error finding container f8cf65c5232cb4a6fcc177bfee634bf9604200eb07ede3dc41b49047c1fd10a0: Status 404 returned error can't find the container with id f8cf65c5232cb4a6fcc177bfee634bf9604200eb07ede3dc41b49047c1fd10a0
Aug 04 17:52:34.245259 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:34.245202    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-gtwkf"]
Aug 04 17:52:34.247745 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:52:34.247724    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eaf7533_2fc5_4da2_bec2_282dce644049.slice/crio-3a9932d5defd05fe71bb65eaf69f5bcc5600e471c9413bcd8619edf161f12e84 WatchSource:0}: Error finding container 3a9932d5defd05fe71bb65eaf69f5bcc5600e471c9413bcd8619edf161f12e84: Status 404 returned error can't find the container with id 3a9932d5defd05fe71bb65eaf69f5bcc5600e471c9413bcd8619edf161f12e84
Aug 04 17:52:34.535612 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:34.535587    2560 generic.go:356] "Generic (PLEG): container finished" podID="8941bb3a-0305-4d69-8a95-bee3e685acbb" containerID="1d0cd3344598c5ff87ae2cb9a00a13c85f062f87991ce7b99aa25249704710e1" exitCode=0
Aug 04 17:52:34.535715 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:34.535658    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fjnbn" event={"ID":"8941bb3a-0305-4d69-8a95-bee3e685acbb","Type":"ContainerDied","Data":"1d0cd3344598c5ff87ae2cb9a00a13c85f062f87991ce7b99aa25249704710e1"}
Aug 04 17:52:34.536927 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:34.536896    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5s8jf" event={"ID":"76442945-00cd-4bbd-8dc1-a65a9573b031","Type":"ContainerStarted","Data":"26ce089ec65d0a54f1120eddf2d76425603d17a40698c01b42ef63c46c4aee27"}
Aug 04 17:52:34.538021 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:34.537965    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-jkhcl" event={"ID":"90aff035-abe4-4fcf-bcd6-76f25a1061dc","Type":"ContainerStarted","Data":"f744cc852c1123fa2c276ac85f6f1a6dba4f6d1448a97d45e326bfb9ec8cef04"}
Aug 04 17:52:34.539170 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:34.539144    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-47zwf" event={"ID":"3f8891ae-cf02-49a7-9144-ef455be4cdee","Type":"ContainerStarted","Data":"f8cf65c5232cb4a6fcc177bfee634bf9604200eb07ede3dc41b49047c1fd10a0"}
Aug 04 17:52:34.540272 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:34.540237    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-ks7s5" event={"ID":"6b29597f-59c4-4f28-a164-7d2e97248d83","Type":"ContainerStarted","Data":"8bfd40b93ccd9787515c3fc3abfa83bfabb0f7b8e2ee9433cd728f3a23ca04eb"}
Aug 04 17:52:34.541360 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:34.541337    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-n64q5" event={"ID":"f3c4ae8d-6d40-466c-a97c-daab363b3502","Type":"ContainerStarted","Data":"05a6b3c0fb33451d75f116828efcb0bd89d2db3a95e77843bbbb4f9d53c9273d"}
Aug 04 17:52:34.546610 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:34.546588    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-wt47b" event={"ID":"7b436a96-7884-43ef-befb-4cff72a9bc6a","Type":"ContainerStarted","Data":"db5df6a1f91af9502dc88d9e326fa4dc09c7ad70a57aafc164cf071c2374b53d"}
Aug 04 17:52:34.547721 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:34.547700    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf" event={"ID":"8eaf7533-2fc5-4da2-bec2-282dce644049","Type":"ContainerStarted","Data":"3a9932d5defd05fe71bb65eaf69f5bcc5600e471c9413bcd8619edf161f12e84"}
Aug 04 17:52:34.865767 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:34.865676    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b35d3198-f4e3-44a5-a0b7-08c987b5805f-metrics-certs\") pod \"router-default-5955597dc4-jbsv8\" (UID: \"b35d3198-f4e3-44a5-a0b7-08c987b5805f\") " pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:52:34.865767 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:34.865763    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b35d3198-f4e3-44a5-a0b7-08c987b5805f-service-ca-bundle\") pod \"router-default-5955597dc4-jbsv8\" (UID: \"b35d3198-f4e3-44a5-a0b7-08c987b5805f\") " pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:52:34.865980 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:34.865825    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/165249a4-0e3d-4f20-9719-c91a6d7ce78f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-kxp92\" (UID: \"165249a4-0e3d-4f20-9719-c91a6d7ce78f\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-kxp92"
Aug 04 17:52:34.865980 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:34.865866    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-registry-tls\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:34.866169 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:34.866044    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 17:52:34.866169 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:34.866061    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69bc56d479-dx5kz: secret "image-registry-tls" not found
Aug 04 17:52:34.866169 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:34.866126    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-registry-tls podName:887c097d-64f3-4f3f-99ac-7981ab483724 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:36.866106482 +0000 UTC m=+39.049266640 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-registry-tls") pod "image-registry-69bc56d479-dx5kz" (UID: "887c097d-64f3-4f3f-99ac-7981ab483724") : secret "image-registry-tls" not found
Aug 04 17:52:34.866318 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:34.866205    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 04 17:52:34.866318 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:34.866242    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b35d3198-f4e3-44a5-a0b7-08c987b5805f-metrics-certs podName:b35d3198-f4e3-44a5-a0b7-08c987b5805f nodeName:}" failed. No retries permitted until 2026-08-04 17:52:36.866229121 +0000 UTC m=+39.049389294 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b35d3198-f4e3-44a5-a0b7-08c987b5805f-metrics-certs") pod "router-default-5955597dc4-jbsv8" (UID: "b35d3198-f4e3-44a5-a0b7-08c987b5805f") : secret "router-metrics-certs-default" not found
Aug 04 17:52:34.866318 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:34.866312    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b35d3198-f4e3-44a5-a0b7-08c987b5805f-service-ca-bundle podName:b35d3198-f4e3-44a5-a0b7-08c987b5805f nodeName:}" failed. No retries permitted until 2026-08-04 17:52:36.866302149 +0000 UTC m=+39.049462305 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b35d3198-f4e3-44a5-a0b7-08c987b5805f-service-ca-bundle") pod "router-default-5955597dc4-jbsv8" (UID: "b35d3198-f4e3-44a5-a0b7-08c987b5805f") : configmap references non-existent config key: service-ca.crt
Aug 04 17:52:34.866473 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:34.866371    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 04 17:52:34.866473 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:34.866402    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/165249a4-0e3d-4f20-9719-c91a6d7ce78f-cluster-monitoring-operator-tls podName:165249a4-0e3d-4f20-9719-c91a6d7ce78f nodeName:}" failed. No retries permitted until 2026-08-04 17:52:36.866392123 +0000 UTC m=+39.049552291 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/165249a4-0e3d-4f20-9719-c91a6d7ce78f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-kxp92" (UID: "165249a4-0e3d-4f20-9719-c91a6d7ce78f") : secret "cluster-monitoring-operator-tls" not found
Aug 04 17:52:34.966865 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:34.966828    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f583466f-47c3-4403-a048-4c48550ac4fc-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-stb8t\" (UID: \"f583466f-47c3-4403-a048-4c48550ac4fc\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-stb8t"
Aug 04 17:52:34.967072 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:34.966955    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a01b635-81f5-4c48-9906-dbf914e08592-metrics-tls\") pod \"dns-default-d58jd\" (UID: \"0a01b635-81f5-4c48-9906-dbf914e08592\") " pod="openshift-dns/dns-default-d58jd"
Aug 04 17:52:34.967072 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:34.967014    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f49077e4-25d7-4cf7-8072-87c5a5e13829-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-t4zw5\" (UID: \"f49077e4-25d7-4cf7-8072-87c5a5e13829\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-t4zw5"
Aug 04 17:52:34.967196 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:34.967150    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Aug 04 17:52:34.967249 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:34.967208    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f49077e4-25d7-4cf7-8072-87c5a5e13829-samples-operator-tls podName:f49077e4-25d7-4cf7-8072-87c5a5e13829 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:36.967191053 +0000 UTC m=+39.150351230 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f49077e4-25d7-4cf7-8072-87c5a5e13829-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-t4zw5" (UID: "f49077e4-25d7-4cf7-8072-87c5a5e13829") : secret "samples-operator-tls" not found
Aug 04 17:52:34.967597 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:34.967578    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 04 17:52:34.967649 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:34.967640    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f583466f-47c3-4403-a048-4c48550ac4fc-networking-console-plugin-cert podName:f583466f-47c3-4403-a048-4c48550ac4fc nodeName:}" failed. No retries permitted until 2026-08-04 17:52:36.967623776 +0000 UTC m=+39.150783948 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f583466f-47c3-4403-a048-4c48550ac4fc-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-stb8t" (UID: "f583466f-47c3-4403-a048-4c48550ac4fc") : secret "networking-console-plugin-cert" not found
Aug 04 17:52:34.967711 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:34.967699    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 17:52:34.967757 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:34.967737    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a01b635-81f5-4c48-9906-dbf914e08592-metrics-tls podName:0a01b635-81f5-4c48-9906-dbf914e08592 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:36.9677261 +0000 UTC m=+39.150886269 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a01b635-81f5-4c48-9906-dbf914e08592-metrics-tls") pod "dns-default-d58jd" (UID: "0a01b635-81f5-4c48-9906-dbf914e08592") : secret "dns-default-metrics-tls" not found
Aug 04 17:52:35.564427 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:35.564388    2560 generic.go:356] "Generic (PLEG): container finished" podID="8941bb3a-0305-4d69-8a95-bee3e685acbb" containerID="5babe268345a0ac7c396ec7efca6b08a50ab6142ea4f533ef3602d06696547c8" exitCode=0
Aug 04 17:52:35.564949 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:35.564475    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fjnbn" event={"ID":"8941bb3a-0305-4d69-8a95-bee3e685acbb","Type":"ContainerDied","Data":"5babe268345a0ac7c396ec7efca6b08a50ab6142ea4f533ef3602d06696547c8"}
Aug 04 17:52:36.572472 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:36.572207    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fjnbn" event={"ID":"8941bb3a-0305-4d69-8a95-bee3e685acbb","Type":"ContainerStarted","Data":"1e50bf1609491348d5246b6975019bce18f22162bd90e6de256cd33edfe3972e"}
Aug 04 17:52:36.603470 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:36.603381    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fjnbn" podStartSLOduration=7.503414337 podStartE2EDuration="38.603364552s" totalImagesPullingTime="31.099950215s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 17:51:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:52:36.601902313 +0000 UTC m=+38.785062490" watchObservedRunningTime="2026-08-04 17:52:36.603364552 +0000 UTC m=+38.786524729"
Aug 04 17:52:36.744960 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:36.744924    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:36.840162 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:36.840129    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:36.888612 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:36.888523    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b35d3198-f4e3-44a5-a0b7-08c987b5805f-metrics-certs\") pod \"router-default-5955597dc4-jbsv8\" (UID: \"b35d3198-f4e3-44a5-a0b7-08c987b5805f\") " pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:52:36.888612 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:36.888605    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b35d3198-f4e3-44a5-a0b7-08c987b5805f-service-ca-bundle\") pod \"router-default-5955597dc4-jbsv8\" (UID: \"b35d3198-f4e3-44a5-a0b7-08c987b5805f\") " pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:52:36.888786 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:36.888676    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/165249a4-0e3d-4f20-9719-c91a6d7ce78f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-kxp92\" (UID: \"165249a4-0e3d-4f20-9719-c91a6d7ce78f\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-kxp92"
Aug 04 17:52:36.888786 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:36.888720    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-registry-tls\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:36.888886 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:36.888863    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 17:52:36.888886 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:36.888883    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69bc56d479-dx5kz: secret "image-registry-tls" not found
Aug 04 17:52:36.889663 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:36.888937    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-registry-tls podName:887c097d-64f3-4f3f-99ac-7981ab483724 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:40.888918587 +0000 UTC m=+43.072078754 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-registry-tls") pod "image-registry-69bc56d479-dx5kz" (UID: "887c097d-64f3-4f3f-99ac-7981ab483724") : secret "image-registry-tls" not found
Aug 04 17:52:36.889663 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:36.889332    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 04 17:52:36.889663 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:36.889393    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b35d3198-f4e3-44a5-a0b7-08c987b5805f-metrics-certs podName:b35d3198-f4e3-44a5-a0b7-08c987b5805f nodeName:}" failed. No retries permitted until 2026-08-04 17:52:40.889376003 +0000 UTC m=+43.072536176 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b35d3198-f4e3-44a5-a0b7-08c987b5805f-metrics-certs") pod "router-default-5955597dc4-jbsv8" (UID: "b35d3198-f4e3-44a5-a0b7-08c987b5805f") : secret "router-metrics-certs-default" not found
Aug 04 17:52:36.889663 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:36.889460    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b35d3198-f4e3-44a5-a0b7-08c987b5805f-service-ca-bundle podName:b35d3198-f4e3-44a5-a0b7-08c987b5805f nodeName:}" failed. No retries permitted until 2026-08-04 17:52:40.889450475 +0000 UTC m=+43.072610648 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b35d3198-f4e3-44a5-a0b7-08c987b5805f-service-ca-bundle") pod "router-default-5955597dc4-jbsv8" (UID: "b35d3198-f4e3-44a5-a0b7-08c987b5805f") : configmap references non-existent config key: service-ca.crt
Aug 04 17:52:36.889663 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:36.889533    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 04 17:52:36.889663 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:36.889564    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/165249a4-0e3d-4f20-9719-c91a6d7ce78f-cluster-monitoring-operator-tls podName:165249a4-0e3d-4f20-9719-c91a6d7ce78f nodeName:}" failed. No retries permitted until 2026-08-04 17:52:40.889556738 +0000 UTC m=+43.072716906 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/165249a4-0e3d-4f20-9719-c91a6d7ce78f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-kxp92" (UID: "165249a4-0e3d-4f20-9719-c91a6d7ce78f") : secret "cluster-monitoring-operator-tls" not found
Aug 04 17:52:36.908200 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:36.908175    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:36.989860 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:36.989829    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f583466f-47c3-4403-a048-4c48550ac4fc-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-stb8t\" (UID: \"f583466f-47c3-4403-a048-4c48550ac4fc\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-stb8t"
Aug 04 17:52:36.990063 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:36.989939    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a01b635-81f5-4c48-9906-dbf914e08592-metrics-tls\") pod \"dns-default-d58jd\" (UID: \"0a01b635-81f5-4c48-9906-dbf914e08592\") " pod="openshift-dns/dns-default-d58jd"
Aug 04 17:52:36.990063 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:36.989982    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f49077e4-25d7-4cf7-8072-87c5a5e13829-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-t4zw5\" (UID: \"f49077e4-25d7-4cf7-8072-87c5a5e13829\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-t4zw5"
Aug 04 17:52:36.990151 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:36.990136    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Aug 04 17:52:36.990198 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:36.990189    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f49077e4-25d7-4cf7-8072-87c5a5e13829-samples-operator-tls podName:f49077e4-25d7-4cf7-8072-87c5a5e13829 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:40.99017271 +0000 UTC m=+43.173332879 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f49077e4-25d7-4cf7-8072-87c5a5e13829-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-t4zw5" (UID: "f49077e4-25d7-4cf7-8072-87c5a5e13829") : secret "samples-operator-tls" not found
Aug 04 17:52:36.990501 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:36.990474    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 04 17:52:36.990574 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:36.990526    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f583466f-47c3-4403-a048-4c48550ac4fc-networking-console-plugin-cert podName:f583466f-47c3-4403-a048-4c48550ac4fc nodeName:}" failed. No retries permitted until 2026-08-04 17:52:40.990512071 +0000 UTC m=+43.173672232 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f583466f-47c3-4403-a048-4c48550ac4fc-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-stb8t" (UID: "f583466f-47c3-4403-a048-4c48550ac4fc") : secret "networking-console-plugin-cert" not found
Aug 04 17:52:36.990639 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:36.990574    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 17:52:36.990639 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:36.990595    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a01b635-81f5-4c48-9906-dbf914e08592-metrics-tls podName:0a01b635-81f5-4c48-9906-dbf914e08592 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:40.990588796 +0000 UTC m=+43.173748950 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a01b635-81f5-4c48-9906-dbf914e08592-metrics-tls") pod "dns-default-d58jd" (UID: "0a01b635-81f5-4c48-9906-dbf914e08592") : secret "dns-default-metrics-tls" not found
Aug 04 17:52:37.780197 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:37.780160    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:37.881902 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:37.881870    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:38.834770 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:38.834734    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:38.934381 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:38.934294    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:40.923540 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:40.923504    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b35d3198-f4e3-44a5-a0b7-08c987b5805f-metrics-certs\") pod \"router-default-5955597dc4-jbsv8\" (UID: \"b35d3198-f4e3-44a5-a0b7-08c987b5805f\") " pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:52:40.924046 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:40.923564    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b35d3198-f4e3-44a5-a0b7-08c987b5805f-service-ca-bundle\") pod \"router-default-5955597dc4-jbsv8\" (UID: \"b35d3198-f4e3-44a5-a0b7-08c987b5805f\") " pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:52:40.924046 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:40.923707    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b35d3198-f4e3-44a5-a0b7-08c987b5805f-service-ca-bundle podName:b35d3198-f4e3-44a5-a0b7-08c987b5805f nodeName:}" failed. No retries permitted until 2026-08-04 17:52:48.923684417 +0000 UTC m=+51.106844571 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b35d3198-f4e3-44a5-a0b7-08c987b5805f-service-ca-bundle") pod "router-default-5955597dc4-jbsv8" (UID: "b35d3198-f4e3-44a5-a0b7-08c987b5805f") : configmap references non-existent config key: service-ca.crt
Aug 04 17:52:40.924046 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:40.923703    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 04 17:52:40.924046 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:40.923764    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/165249a4-0e3d-4f20-9719-c91a6d7ce78f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-kxp92\" (UID: \"165249a4-0e3d-4f20-9719-c91a6d7ce78f\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-kxp92"
Aug 04 17:52:40.924046 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:40.923794    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b35d3198-f4e3-44a5-a0b7-08c987b5805f-metrics-certs podName:b35d3198-f4e3-44a5-a0b7-08c987b5805f nodeName:}" failed. No retries permitted until 2026-08-04 17:52:48.923774608 +0000 UTC m=+51.106934779 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b35d3198-f4e3-44a5-a0b7-08c987b5805f-metrics-certs") pod "router-default-5955597dc4-jbsv8" (UID: "b35d3198-f4e3-44a5-a0b7-08c987b5805f") : secret "router-metrics-certs-default" not found
Aug 04 17:52:40.924046 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:40.923839    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 04 17:52:40.924046 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:40.923865    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-registry-tls\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:40.924046 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:40.923886    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/165249a4-0e3d-4f20-9719-c91a6d7ce78f-cluster-monitoring-operator-tls podName:165249a4-0e3d-4f20-9719-c91a6d7ce78f nodeName:}" failed. No retries permitted until 2026-08-04 17:52:48.923874776 +0000 UTC m=+51.107034931 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/165249a4-0e3d-4f20-9719-c91a6d7ce78f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-kxp92" (UID: "165249a4-0e3d-4f20-9719-c91a6d7ce78f") : secret "cluster-monitoring-operator-tls" not found
Aug 04 17:52:40.924046 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:40.923937    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 17:52:40.924046 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:40.923949    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69bc56d479-dx5kz: secret "image-registry-tls" not found
Aug 04 17:52:40.924046 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:40.924006    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-registry-tls podName:887c097d-64f3-4f3f-99ac-7981ab483724 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:48.923977093 +0000 UTC m=+51.107137276 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-registry-tls") pod "image-registry-69bc56d479-dx5kz" (UID: "887c097d-64f3-4f3f-99ac-7981ab483724") : secret "image-registry-tls" not found
Aug 04 17:52:40.987787 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:40.987749    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:41.025048 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:41.025012    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f49077e4-25d7-4cf7-8072-87c5a5e13829-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-t4zw5\" (UID: \"f49077e4-25d7-4cf7-8072-87c5a5e13829\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-t4zw5"
Aug 04 17:52:41.025194 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:41.025140    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f583466f-47c3-4403-a048-4c48550ac4fc-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-stb8t\" (UID: \"f583466f-47c3-4403-a048-4c48550ac4fc\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-stb8t"
Aug 04 17:52:41.025194 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:41.025172    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Aug 04 17:52:41.025310 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:41.025238    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f49077e4-25d7-4cf7-8072-87c5a5e13829-samples-operator-tls podName:f49077e4-25d7-4cf7-8072-87c5a5e13829 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:49.025217746 +0000 UTC m=+51.208377922 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f49077e4-25d7-4cf7-8072-87c5a5e13829-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-t4zw5" (UID: "f49077e4-25d7-4cf7-8072-87c5a5e13829") : secret "samples-operator-tls" not found
Aug 04 17:52:41.025310 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:41.025274    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a01b635-81f5-4c48-9906-dbf914e08592-metrics-tls\") pod \"dns-default-d58jd\" (UID: \"0a01b635-81f5-4c48-9906-dbf914e08592\") " pod="openshift-dns/dns-default-d58jd"
Aug 04 17:52:41.025310 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:41.025289    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 04 17:52:41.025464 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:41.025352    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f583466f-47c3-4403-a048-4c48550ac4fc-networking-console-plugin-cert podName:f583466f-47c3-4403-a048-4c48550ac4fc nodeName:}" failed. No retries permitted until 2026-08-04 17:52:49.025339439 +0000 UTC m=+51.208499593 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f583466f-47c3-4403-a048-4c48550ac4fc-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-stb8t" (UID: "f583466f-47c3-4403-a048-4c48550ac4fc") : secret "networking-console-plugin-cert" not found
Aug 04 17:52:41.025464 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:41.025410    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 17:52:41.025464 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:41.025447    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a01b635-81f5-4c48-9906-dbf914e08592-metrics-tls podName:0a01b635-81f5-4c48-9906-dbf914e08592 nodeName:}" failed. No retries permitted until 2026-08-04 17:52:49.025435268 +0000 UTC m=+51.208595423 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a01b635-81f5-4c48-9906-dbf914e08592-metrics-tls") pod "dns-default-d58jd" (UID: "0a01b635-81f5-4c48-9906-dbf914e08592") : secret "dns-default-metrics-tls" not found
Aug 04 17:52:41.089423 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:41.089346    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:41.188896 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:41.188818    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:41.293224 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:41.293191    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:42.019665 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:42.019632    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:42.121398 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:42.121323    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:42.591183 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:42.591021    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-47zwf" event={"ID":"3f8891ae-cf02-49a7-9144-ef455be4cdee","Type":"ContainerStarted","Data":"6430cb26b5d8f22c4d722b6c0af5b7517d5dfab636107a31eb82904e90f73824"}
Aug 04 17:52:42.592599 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:42.592571    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-ks7s5" event={"ID":"6b29597f-59c4-4f28-a164-7d2e97248d83","Type":"ContainerStarted","Data":"a1dd384f132b841755ab487547db8cc2ba7071828ef891034f47cd92d8d34b41"}
Aug 04 17:52:42.594386 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:42.594363    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-n64q5" event={"ID":"f3c4ae8d-6d40-466c-a97c-daab363b3502","Type":"ContainerStarted","Data":"9ff9fa30b85b2c26ec6275d72344ddc13bc2cbb91fb51645f2cde43805d7b978"}
Aug 04 17:52:42.595925 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:42.595901    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-wt47b" event={"ID":"7b436a96-7884-43ef-befb-4cff72a9bc6a","Type":"ContainerStarted","Data":"016969f7ad694148155869e6f9f45c66b45efd12ed994c33a98ff5fa26a99c2e"}
Aug 04 17:52:42.597658 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:42.597637    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/0.log"
Aug 04 17:52:42.597740 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:42.597678    2560 generic.go:356] "Generic (PLEG): container finished" podID="8eaf7533-2fc5-4da2-bec2-282dce644049" containerID="37cdbe795b4e8c5d28871512dcebb577b7d97e2f663f1febee463b807ad88793" exitCode=255
Aug 04 17:52:42.597807 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:42.597741    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf" event={"ID":"8eaf7533-2fc5-4da2-bec2-282dce644049","Type":"ContainerDied","Data":"37cdbe795b4e8c5d28871512dcebb577b7d97e2f663f1febee463b807ad88793"}
Aug 04 17:52:42.597961 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:42.597946    2560 scope.go:117] "RemoveContainer" containerID="37cdbe795b4e8c5d28871512dcebb577b7d97e2f663f1febee463b807ad88793"
Aug 04 17:52:42.600838 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:42.600809    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5s8jf" event={"ID":"76442945-00cd-4bbd-8dc1-a65a9573b031","Type":"ContainerStarted","Data":"8701c9c7c9b3a1c65e885550ef51c142e22d3313f1b4700ab4112c8aeb6ffeb7"}
Aug 04 17:52:42.602333 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:42.602310    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-jkhcl" event={"ID":"90aff035-abe4-4fcf-bcd6-76f25a1061dc","Type":"ContainerStarted","Data":"05da75f842b6e698589bbf3b01c87bb35698e68cfeb4a8053c6598167951f6f9"}
Aug 04 17:52:42.616021 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:42.615419    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-47zwf" podStartSLOduration=33.983250609 podStartE2EDuration="41.615404287s" totalImagesPullingTime="7.632153678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 17:52:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:52:42.614325181 +0000 UTC m=+44.797485360" watchObservedRunningTime="2026-08-04 17:52:42.615404287 +0000 UTC m=+44.798564455"
Aug 04 17:52:42.635768 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:42.635723    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5s8jf" podStartSLOduration=1.990969419 podStartE2EDuration="9.635706159s" totalImagesPullingTime="7.64473674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 17:52:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:52:42.633801938 +0000 UTC m=+44.816962119" watchObservedRunningTime="2026-08-04 17:52:42.635706159 +0000 UTC m=+44.818866336"
Aug 04 17:52:42.682655 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:42.682604    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-n64q5" podStartSLOduration=34.026550526 podStartE2EDuration="41.682586188s" totalImagesPullingTime="7.656035662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 17:52:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:52:42.68127882 +0000 UTC m=+44.864439021" watchObservedRunningTime="2026-08-04 17:52:42.682586188 +0000 UTC m=+44.865746365"
Aug 04 17:52:42.683450 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:42.683413    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-77696685b-ks7s5" podStartSLOduration=34.050398553 podStartE2EDuration="41.683403445s" totalImagesPullingTime="7.633004892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 17:52:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:52:42.660602802 +0000 UTC m=+44.843762980" watchObservedRunningTime="2026-08-04 17:52:42.683403445 +0000 UTC m=+44.866563622"
Aug 04 17:52:42.727039 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:42.726977    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-wt47b" podStartSLOduration=34.077475437 podStartE2EDuration="41.726959584s" totalImagesPullingTime="7.649484147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 17:52:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:52:42.725491097 +0000 UTC m=+44.908651275" watchObservedRunningTime="2026-08-04 17:52:42.726959584 +0000 UTC m=+44.910119761"
Aug 04 17:52:42.745526 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:42.745482    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-jkhcl" podStartSLOduration=35.892762734 podStartE2EDuration="41.745464842s" totalImagesPullingTime="5.852702108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 17:52:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:52:42.744673028 +0000 UTC m=+44.927833205" watchObservedRunningTime="2026-08-04 17:52:42.745464842 +0000 UTC m=+44.928625021"
Aug 04 17:52:42.855010 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:42.854925    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:42.953290 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:42.953263    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:43.323187 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.323147    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-c94lk"]
Aug 04 17:52:43.325770 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.325637    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:43.345688 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.345632    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-c94lk"
Aug 04 17:52:43.347384 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.347365    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-c94lk"]
Aug 04 17:52:43.348084 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.348047    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kwp6w\""
Aug 04 17:52:43.348252 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.348234    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Aug 04 17:52:43.348326 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.348245    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Aug 04 17:52:43.399976 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.399948    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:43.448430 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.448402    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/568af22c-f1cf-44ba-abe1-508565a26ebd-crio-socket\") pod \"insights-runtime-extractor-c94lk\" (UID: \"568af22c-f1cf-44ba-abe1-508565a26ebd\") " pod="openshift-insights/insights-runtime-extractor-c94lk"
Aug 04 17:52:43.448524 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.448447    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66frv\" (UniqueName: \"kubernetes.io/projected/568af22c-f1cf-44ba-abe1-508565a26ebd-kube-api-access-66frv\") pod \"insights-runtime-extractor-c94lk\" (UID: \"568af22c-f1cf-44ba-abe1-508565a26ebd\") " pod="openshift-insights/insights-runtime-extractor-c94lk"
Aug 04 17:52:43.448524 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.448467    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/568af22c-f1cf-44ba-abe1-508565a26ebd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c94lk\" (UID: \"568af22c-f1cf-44ba-abe1-508565a26ebd\") " pod="openshift-insights/insights-runtime-extractor-c94lk"
Aug 04 17:52:43.448605 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.448558    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/568af22c-f1cf-44ba-abe1-508565a26ebd-data-volume\") pod \"insights-runtime-extractor-c94lk\" (UID: \"568af22c-f1cf-44ba-abe1-508565a26ebd\") " pod="openshift-insights/insights-runtime-extractor-c94lk"
Aug 04 17:52:43.448710 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.448692    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/568af22c-f1cf-44ba-abe1-508565a26ebd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c94lk\" (UID: \"568af22c-f1cf-44ba-abe1-508565a26ebd\") " pod="openshift-insights/insights-runtime-extractor-c94lk"
Aug 04 17:52:43.515518 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.515496    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf"
Aug 04 17:52:43.515604 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.515538    2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf"
Aug 04 17:52:43.550029 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.550007    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/568af22c-f1cf-44ba-abe1-508565a26ebd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c94lk\" (UID: \"568af22c-f1cf-44ba-abe1-508565a26ebd\") " pod="openshift-insights/insights-runtime-extractor-c94lk"
Aug 04 17:52:43.550114 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.550050    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/568af22c-f1cf-44ba-abe1-508565a26ebd-crio-socket\") pod \"insights-runtime-extractor-c94lk\" (UID: \"568af22c-f1cf-44ba-abe1-508565a26ebd\") " pod="openshift-insights/insights-runtime-extractor-c94lk"
Aug 04 17:52:43.550114 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.550096    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66frv\" (UniqueName: \"kubernetes.io/projected/568af22c-f1cf-44ba-abe1-508565a26ebd-kube-api-access-66frv\") pod \"insights-runtime-extractor-c94lk\" (UID: \"568af22c-f1cf-44ba-abe1-508565a26ebd\") " pod="openshift-insights/insights-runtime-extractor-c94lk"
Aug 04 17:52:43.550191 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.550127    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/568af22c-f1cf-44ba-abe1-508565a26ebd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c94lk\" (UID: \"568af22c-f1cf-44ba-abe1-508565a26ebd\") " pod="openshift-insights/insights-runtime-extractor-c94lk"
Aug 04 17:52:43.550191 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:43.550142    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 17:52:43.550191 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.550179    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/568af22c-f1cf-44ba-abe1-508565a26ebd-data-volume\") pod \"insights-runtime-extractor-c94lk\" (UID: \"568af22c-f1cf-44ba-abe1-508565a26ebd\") " pod="openshift-insights/insights-runtime-extractor-c94lk"
Aug 04 17:52:43.550330 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:43.550199    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/568af22c-f1cf-44ba-abe1-508565a26ebd-kube-rbac-proxy-cm podName:568af22c-f1cf-44ba-abe1-508565a26ebd nodeName:}" failed. No retries permitted until 2026-08-04 17:52:44.050180717 +0000 UTC m=+46.233340875 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/568af22c-f1cf-44ba-abe1-508565a26ebd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-c94lk" (UID: "568af22c-f1cf-44ba-abe1-508565a26ebd") : configmap "kube-rbac-proxy" not found
Aug 04 17:52:43.550330 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:43.550280    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 17:52:43.550330 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.550293    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/568af22c-f1cf-44ba-abe1-508565a26ebd-crio-socket\") pod \"insights-runtime-extractor-c94lk\" (UID: \"568af22c-f1cf-44ba-abe1-508565a26ebd\") " pod="openshift-insights/insights-runtime-extractor-c94lk"
Aug 04 17:52:43.550447 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:43.550339    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/568af22c-f1cf-44ba-abe1-508565a26ebd-insights-runtime-extractor-tls podName:568af22c-f1cf-44ba-abe1-508565a26ebd nodeName:}" failed. No retries permitted until 2026-08-04 17:52:44.050320556 +0000 UTC m=+46.233480713 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/568af22c-f1cf-44ba-abe1-508565a26ebd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c94lk" (UID: "568af22c-f1cf-44ba-abe1-508565a26ebd") : secret "insights-runtime-extractor-tls" not found
Aug 04 17:52:43.550485 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.550465    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/568af22c-f1cf-44ba-abe1-508565a26ebd-data-volume\") pod \"insights-runtime-extractor-c94lk\" (UID: \"568af22c-f1cf-44ba-abe1-508565a26ebd\") " pod="openshift-insights/insights-runtime-extractor-c94lk"
Aug 04 17:52:43.564796 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.564715    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:43.565605 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.565589    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66frv\" (UniqueName: \"kubernetes.io/projected/568af22c-f1cf-44ba-abe1-508565a26ebd-kube-api-access-66frv\") pod \"insights-runtime-extractor-c94lk\" (UID: \"568af22c-f1cf-44ba-abe1-508565a26ebd\") " pod="openshift-insights/insights-runtime-extractor-c94lk"
Aug 04 17:52:43.607375 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.607308    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/1.log"
Aug 04 17:52:43.607716 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.607701    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/0.log"
Aug 04 17:52:43.607803 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.607740    2560 generic.go:356] "Generic (PLEG): container finished" podID="8eaf7533-2fc5-4da2-bec2-282dce644049" containerID="e5e7fc8ccc0e0a3594f202e61497d28a32f41be222379cbbb469eaea2213864b" exitCode=255
Aug 04 17:52:43.607856 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.607829    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf" event={"ID":"8eaf7533-2fc5-4da2-bec2-282dce644049","Type":"ContainerDied","Data":"e5e7fc8ccc0e0a3594f202e61497d28a32f41be222379cbbb469eaea2213864b"}
Aug 04 17:52:43.619699 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.619679    2560 scope.go:117] "RemoveContainer" containerID="37cdbe795b4e8c5d28871512dcebb577b7d97e2f663f1febee463b807ad88793"
Aug 04 17:52:43.619796 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.619783    2560 scope.go:117] "RemoveContainer" containerID="e5e7fc8ccc0e0a3594f202e61497d28a32f41be222379cbbb469eaea2213864b"
Aug 04 17:52:43.619958 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:43.619939    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-gtwkf_openshift-console-operator(8eaf7533-2fc5-4da2-bec2-282dce644049)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf" podUID="8eaf7533-2fc5-4da2-bec2-282dce644049"
Aug 04 17:52:43.632777 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.632757    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:43.966229 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.966198    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:43.980841 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:43.980812    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-wtfkt"]
Aug 04 17:52:44.013109 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:44.013084    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-wtfkt"]
Aug 04 17:52:44.013236 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:44.013199    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-wtfkt"
Aug 04 17:52:44.015774 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:44.015753    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Aug 04 17:52:44.015912 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:44.015846    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-ghl2m\""
Aug 04 17:52:44.016087 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:44.016072    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Aug 04 17:52:44.037671 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:44.037640    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:44.054642 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:44.054618    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/568af22c-f1cf-44ba-abe1-508565a26ebd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c94lk\" (UID: \"568af22c-f1cf-44ba-abe1-508565a26ebd\") " pod="openshift-insights/insights-runtime-extractor-c94lk"
Aug 04 17:52:44.054744 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:44.054674    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l88z8\" (UniqueName: \"kubernetes.io/projected/d3f38436-e32b-4010-81ee-bf8d1f811a17-kube-api-access-l88z8\") pod \"migrator-68bc745d87-wtfkt\" (UID: \"d3f38436-e32b-4010-81ee-bf8d1f811a17\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-wtfkt"
Aug 04 17:52:44.054744 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:44.054705    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/568af22c-f1cf-44ba-abe1-508565a26ebd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c94lk\" (UID: \"568af22c-f1cf-44ba-abe1-508565a26ebd\") " pod="openshift-insights/insights-runtime-extractor-c94lk"
Aug 04 17:52:44.054744 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:44.054730    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 17:52:44.054851 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:44.054781    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/568af22c-f1cf-44ba-abe1-508565a26ebd-kube-rbac-proxy-cm podName:568af22c-f1cf-44ba-abe1-508565a26ebd nodeName:}" failed. No retries permitted until 2026-08-04 17:52:45.054766621 +0000 UTC m=+47.237926799 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/568af22c-f1cf-44ba-abe1-508565a26ebd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-c94lk" (UID: "568af22c-f1cf-44ba-abe1-508565a26ebd") : configmap "kube-rbac-proxy" not found
Aug 04 17:52:44.054851 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:44.054838    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 17:52:44.054938 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:44.054901    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/568af22c-f1cf-44ba-abe1-508565a26ebd-insights-runtime-extractor-tls podName:568af22c-f1cf-44ba-abe1-508565a26ebd nodeName:}" failed. No retries permitted until 2026-08-04 17:52:45.054878067 +0000 UTC m=+47.238038223 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/568af22c-f1cf-44ba-abe1-508565a26ebd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c94lk" (UID: "568af22c-f1cf-44ba-abe1-508565a26ebd") : secret "insights-runtime-extractor-tls" not found
Aug 04 17:52:44.104627 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:44.104590    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:44.156041 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:44.156017    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l88z8\" (UniqueName: \"kubernetes.io/projected/d3f38436-e32b-4010-81ee-bf8d1f811a17-kube-api-access-l88z8\") pod \"migrator-68bc745d87-wtfkt\" (UID: \"d3f38436-e32b-4010-81ee-bf8d1f811a17\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-wtfkt"
Aug 04 17:52:44.166411 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:44.166385    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l88z8\" (UniqueName: \"kubernetes.io/projected/d3f38436-e32b-4010-81ee-bf8d1f811a17-kube-api-access-l88z8\") pod \"migrator-68bc745d87-wtfkt\" (UID: \"d3f38436-e32b-4010-81ee-bf8d1f811a17\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-wtfkt"
Aug 04 17:52:44.203438 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:44.203410    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:44.322253 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:44.322200    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-wtfkt"
Aug 04 17:52:44.449059 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:44.449036    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lzkvj_0b696fd5-c22f-480a-a9bd-a2f8a19784ae/dns-node-resolver/0.log"
Aug 04 17:52:44.484663 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:44.484644    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-wtfkt"]
Aug 04 17:52:44.486580 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:52:44.486558    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3f38436_e32b_4010_81ee_bf8d1f811a17.slice/crio-1db77f6ef3f27c2758516b53d3561ea765608be82d28aa52e503edd4719f93bf WatchSource:0}: Error finding container 1db77f6ef3f27c2758516b53d3561ea765608be82d28aa52e503edd4719f93bf: Status 404 returned error can't find the container with id 1db77f6ef3f27c2758516b53d3561ea765608be82d28aa52e503edd4719f93bf
Aug 04 17:52:44.616924 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:44.616862    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/1.log"
Aug 04 17:52:44.617246 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:44.617227    2560 scope.go:117] "RemoveContainer" containerID="e5e7fc8ccc0e0a3594f202e61497d28a32f41be222379cbbb469eaea2213864b"
Aug 04 17:52:44.617437 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:44.617411    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-gtwkf_openshift-console-operator(8eaf7533-2fc5-4da2-bec2-282dce644049)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf" podUID="8eaf7533-2fc5-4da2-bec2-282dce644049"
Aug 04 17:52:44.618213 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:44.618192    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-wtfkt" event={"ID":"d3f38436-e32b-4010-81ee-bf8d1f811a17","Type":"ContainerStarted","Data":"1db77f6ef3f27c2758516b53d3561ea765608be82d28aa52e503edd4719f93bf"}
Aug 04 17:52:45.062366 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:45.062335    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/568af22c-f1cf-44ba-abe1-508565a26ebd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c94lk\" (UID: \"568af22c-f1cf-44ba-abe1-508565a26ebd\") " pod="openshift-insights/insights-runtime-extractor-c94lk"
Aug 04 17:52:45.062585 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:45.062426    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/568af22c-f1cf-44ba-abe1-508565a26ebd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c94lk\" (UID: \"568af22c-f1cf-44ba-abe1-508565a26ebd\") " pod="openshift-insights/insights-runtime-extractor-c94lk"
Aug 04 17:52:45.062585 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:45.062459    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 17:52:45.062585 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:45.062523    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/568af22c-f1cf-44ba-abe1-508565a26ebd-kube-rbac-proxy-cm podName:568af22c-f1cf-44ba-abe1-508565a26ebd nodeName:}" failed. No retries permitted until 2026-08-04 17:52:47.06250434 +0000 UTC m=+49.245664496 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/568af22c-f1cf-44ba-abe1-508565a26ebd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-c94lk" (UID: "568af22c-f1cf-44ba-abe1-508565a26ebd") : configmap "kube-rbac-proxy" not found
Aug 04 17:52:45.062760 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:45.062598    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 17:52:45.062760 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:45.062670    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/568af22c-f1cf-44ba-abe1-508565a26ebd-insights-runtime-extractor-tls podName:568af22c-f1cf-44ba-abe1-508565a26ebd nodeName:}" failed. No retries permitted until 2026-08-04 17:52:47.062655295 +0000 UTC m=+49.245815658 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/568af22c-f1cf-44ba-abe1-508565a26ebd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c94lk" (UID: "568af22c-f1cf-44ba-abe1-508565a26ebd") : secret "insights-runtime-extractor-tls" not found
Aug 04 17:52:45.648855 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:45.648831    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mdtdv_a4441318-a230-45e7-8ab5-bf4049533e35/node-ca/0.log"
Aug 04 17:52:45.760914 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:45.760881    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-88czv"]
Aug 04 17:52:45.773523 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:45.773501    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-88czv"]
Aug 04 17:52:45.773664 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:45.773584    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-88czv"
Aug 04 17:52:45.775957 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:45.775923    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-xwckb\""
Aug 04 17:52:45.775957 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:45.775952    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Aug 04 17:52:45.776743 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:45.776698    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Aug 04 17:52:45.776743 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:45.776712    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Aug 04 17:52:45.776895 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:45.776802    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Aug 04 17:52:45.802598 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:45.802554    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:45.863300 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:45.863271    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:45.870285 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:45.870255    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr4p9\" (UniqueName: \"kubernetes.io/projected/d14272f1-8d07-433b-aad9-44f315935e8e-kube-api-access-rr4p9\") pod \"service-ca-98c88fbb5-88czv\" (UID: \"d14272f1-8d07-433b-aad9-44f315935e8e\") " pod="openshift-service-ca/service-ca-98c88fbb5-88czv"
Aug 04 17:52:45.870393 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:45.870317    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d14272f1-8d07-433b-aad9-44f315935e8e-signing-key\") pod \"service-ca-98c88fbb5-88czv\" (UID: \"d14272f1-8d07-433b-aad9-44f315935e8e\") " pod="openshift-service-ca/service-ca-98c88fbb5-88czv"
Aug 04 17:52:45.870393 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:45.870348    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d14272f1-8d07-433b-aad9-44f315935e8e-signing-cabundle\") pod \"service-ca-98c88fbb5-88czv\" (UID: \"d14272f1-8d07-433b-aad9-44f315935e8e\") " pod="openshift-service-ca/service-ca-98c88fbb5-88czv"
Aug 04 17:52:45.971680 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:45.971653    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d14272f1-8d07-433b-aad9-44f315935e8e-signing-key\") pod \"service-ca-98c88fbb5-88czv\" (UID: \"d14272f1-8d07-433b-aad9-44f315935e8e\") " pod="openshift-service-ca/service-ca-98c88fbb5-88czv"
Aug 04 17:52:45.971795 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:45.971703    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d14272f1-8d07-433b-aad9-44f315935e8e-signing-cabundle\") pod \"service-ca-98c88fbb5-88czv\" (UID: \"d14272f1-8d07-433b-aad9-44f315935e8e\") " pod="openshift-service-ca/service-ca-98c88fbb5-88czv"
Aug 04 17:52:45.971957 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:45.971937    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rr4p9\" (UniqueName: \"kubernetes.io/projected/d14272f1-8d07-433b-aad9-44f315935e8e-kube-api-access-rr4p9\") pod \"service-ca-98c88fbb5-88czv\" (UID: \"d14272f1-8d07-433b-aad9-44f315935e8e\") " pod="openshift-service-ca/service-ca-98c88fbb5-88czv"
Aug 04 17:52:45.972300 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:45.972282    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d14272f1-8d07-433b-aad9-44f315935e8e-signing-cabundle\") pod \"service-ca-98c88fbb5-88czv\" (UID: \"d14272f1-8d07-433b-aad9-44f315935e8e\") " pod="openshift-service-ca/service-ca-98c88fbb5-88czv"
Aug 04 17:52:45.974075 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:45.974059    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d14272f1-8d07-433b-aad9-44f315935e8e-signing-key\") pod \"service-ca-98c88fbb5-88czv\" (UID: \"d14272f1-8d07-433b-aad9-44f315935e8e\") " pod="openshift-service-ca/service-ca-98c88fbb5-88czv"
Aug 04 17:52:45.981692 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:45.981670    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr4p9\" (UniqueName: \"kubernetes.io/projected/d14272f1-8d07-433b-aad9-44f315935e8e-kube-api-access-rr4p9\") pod \"service-ca-98c88fbb5-88czv\" (UID: \"d14272f1-8d07-433b-aad9-44f315935e8e\") " pod="openshift-service-ca/service-ca-98c88fbb5-88czv"
Aug 04 17:52:46.062698 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:46.062684    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5s8jf_76442945-00cd-4bbd-8dc1-a65a9573b031/serve-healthcheck-canary/0.log"
Aug 04 17:52:46.083659 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:46.083639    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-88czv"
Aug 04 17:52:46.248747 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:46.248726    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-88czv"]
Aug 04 17:52:46.250775 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:52:46.250748    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd14272f1_8d07_433b_aad9_44f315935e8e.slice/crio-cdc4ec852663710d02ba7fdef16533a5b257a1e813bb1b964da57a24a15fff4b WatchSource:0}: Error finding container cdc4ec852663710d02ba7fdef16533a5b257a1e813bb1b964da57a24a15fff4b: Status 404 returned error can't find the container with id cdc4ec852663710d02ba7fdef16533a5b257a1e813bb1b964da57a24a15fff4b
Aug 04 17:52:46.627645 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:46.627550    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-wtfkt" event={"ID":"d3f38436-e32b-4010-81ee-bf8d1f811a17","Type":"ContainerStarted","Data":"d21e9760801a713ed4473e7d18a9f38420a348e5e3debd9ff8560edf78f07778"}
Aug 04 17:52:46.627645 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:46.627614    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-wtfkt" event={"ID":"d3f38436-e32b-4010-81ee-bf8d1f811a17","Type":"ContainerStarted","Data":"1c3f244871a6dc71aa62665350701dae7cc43947a6c5352e7b82df0e9c60b467"}
Aug 04 17:52:46.629053 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:46.629025    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-88czv" event={"ID":"d14272f1-8d07-433b-aad9-44f315935e8e","Type":"ContainerStarted","Data":"2ff3813fcebdeda5ab6bc7104f02beb739761529bbe32364e52f0a8eb576d2b6"}
Aug 04 17:52:46.629143 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:46.629069    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-88czv" event={"ID":"d14272f1-8d07-433b-aad9-44f315935e8e","Type":"ContainerStarted","Data":"cdc4ec852663710d02ba7fdef16533a5b257a1e813bb1b964da57a24a15fff4b"}
Aug 04 17:52:46.665319 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:46.665270    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-wtfkt" podStartSLOduration=2.123375703 podStartE2EDuration="3.665255694s" totalImagesPullingTime="1.541879991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 17:52:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:52:46.664452499 +0000 UTC m=+48.847612678" watchObservedRunningTime="2026-08-04 17:52:46.665255694 +0000 UTC m=+48.848415893"
Aug 04 17:52:46.683979 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:46.683944    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-98c88fbb5-88czv" podStartSLOduration=1.6839296259999998 podStartE2EDuration="1.683929626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 17:52:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:52:46.682867847 +0000 UTC m=+48.866028022" watchObservedRunningTime="2026-08-04 17:52:46.683929626 +0000 UTC m=+48.867089837"
Aug 04 17:52:46.841246 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:46.841193    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:47.082531 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:47.082505    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/568af22c-f1cf-44ba-abe1-508565a26ebd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c94lk\" (UID: \"568af22c-f1cf-44ba-abe1-508565a26ebd\") " pod="openshift-insights/insights-runtime-extractor-c94lk"
Aug 04 17:52:47.082674 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:47.082644    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/568af22c-f1cf-44ba-abe1-508565a26ebd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c94lk\" (UID: \"568af22c-f1cf-44ba-abe1-508565a26ebd\") " pod="openshift-insights/insights-runtime-extractor-c94lk"
Aug 04 17:52:47.082714 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:47.082672    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 17:52:47.082745 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:47.082720    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 17:52:47.082777 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:47.082743    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/568af22c-f1cf-44ba-abe1-508565a26ebd-insights-runtime-extractor-tls podName:568af22c-f1cf-44ba-abe1-508565a26ebd nodeName:}" failed. No retries permitted until 2026-08-04 17:52:51.082725391 +0000 UTC m=+53.265885568 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/568af22c-f1cf-44ba-abe1-508565a26ebd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c94lk" (UID: "568af22c-f1cf-44ba-abe1-508565a26ebd") : secret "insights-runtime-extractor-tls" not found
Aug 04 17:52:47.082777 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:47.082762    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/568af22c-f1cf-44ba-abe1-508565a26ebd-kube-rbac-proxy-cm podName:568af22c-f1cf-44ba-abe1-508565a26ebd nodeName:}" failed. No retries permitted until 2026-08-04 17:52:51.082752305 +0000 UTC m=+53.265912461 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/568af22c-f1cf-44ba-abe1-508565a26ebd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-c94lk" (UID: "568af22c-f1cf-44ba-abe1-508565a26ebd") : configmap "kube-rbac-proxy" not found
Aug 04 17:52:47.305042 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:47.305013    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:47.498983 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:47.498950    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:47.593319 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:47.593288    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:47.703476 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:47.699781    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:47.792535 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:47.792475    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:47.884740 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:47.884712    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:47.979872 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:47.979839    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log" limit=1024
Aug 04 17:52:49.001552 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:49.001520    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b35d3198-f4e3-44a5-a0b7-08c987b5805f-service-ca-bundle\") pod \"router-default-5955597dc4-jbsv8\" (UID: \"b35d3198-f4e3-44a5-a0b7-08c987b5805f\") " pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:52:49.001941 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:49.001577    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/165249a4-0e3d-4f20-9719-c91a6d7ce78f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-kxp92\" (UID: \"165249a4-0e3d-4f20-9719-c91a6d7ce78f\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-kxp92"
Aug 04 17:52:49.001941 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:49.001653    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-registry-tls\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:52:49.001941 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:49.001671    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 04 17:52:49.001941 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:49.001717    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/165249a4-0e3d-4f20-9719-c91a6d7ce78f-cluster-monitoring-operator-tls podName:165249a4-0e3d-4f20-9719-c91a6d7ce78f nodeName:}" failed. No retries permitted until 2026-08-04 17:53:05.001703564 +0000 UTC m=+67.184863718 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/165249a4-0e3d-4f20-9719-c91a6d7ce78f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-kxp92" (UID: "165249a4-0e3d-4f20-9719-c91a6d7ce78f") : secret "cluster-monitoring-operator-tls" not found
Aug 04 17:52:49.001941 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:49.001774    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 17:52:49.001941 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:49.001788    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69bc56d479-dx5kz: secret "image-registry-tls" not found
Aug 04 17:52:49.001941 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:49.001800    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b35d3198-f4e3-44a5-a0b7-08c987b5805f-service-ca-bundle podName:b35d3198-f4e3-44a5-a0b7-08c987b5805f nodeName:}" failed. No retries permitted until 2026-08-04 17:53:05.00178669 +0000 UTC m=+67.184946847 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b35d3198-f4e3-44a5-a0b7-08c987b5805f-service-ca-bundle") pod "router-default-5955597dc4-jbsv8" (UID: "b35d3198-f4e3-44a5-a0b7-08c987b5805f") : configmap references non-existent config key: service-ca.crt
Aug 04 17:52:49.001941 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:49.001774    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b35d3198-f4e3-44a5-a0b7-08c987b5805f-metrics-certs\") pod \"router-default-5955597dc4-jbsv8\" (UID: \"b35d3198-f4e3-44a5-a0b7-08c987b5805f\") " pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:52:49.001941 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:49.001827    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 04 17:52:49.001941 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:49.001834    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-registry-tls podName:887c097d-64f3-4f3f-99ac-7981ab483724 nodeName:}" failed. No retries permitted until 2026-08-04 17:53:05.001819054 +0000 UTC m=+67.184979210 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-registry-tls") pod "image-registry-69bc56d479-dx5kz" (UID: "887c097d-64f3-4f3f-99ac-7981ab483724") : secret "image-registry-tls" not found
Aug 04 17:52:49.001941 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:49.001852    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b35d3198-f4e3-44a5-a0b7-08c987b5805f-metrics-certs podName:b35d3198-f4e3-44a5-a0b7-08c987b5805f nodeName:}" failed. No retries permitted until 2026-08-04 17:53:05.001845017 +0000 UTC m=+67.185005200 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b35d3198-f4e3-44a5-a0b7-08c987b5805f-metrics-certs") pod "router-default-5955597dc4-jbsv8" (UID: "b35d3198-f4e3-44a5-a0b7-08c987b5805f") : secret "router-metrics-certs-default" not found
Aug 04 17:52:49.102205 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:49.102160    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a01b635-81f5-4c48-9906-dbf914e08592-metrics-tls\") pod \"dns-default-d58jd\" (UID: \"0a01b635-81f5-4c48-9906-dbf914e08592\") " pod="openshift-dns/dns-default-d58jd"
Aug 04 17:52:49.102205 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:49.102214    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f49077e4-25d7-4cf7-8072-87c5a5e13829-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-t4zw5\" (UID: \"f49077e4-25d7-4cf7-8072-87c5a5e13829\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-t4zw5"
Aug 04 17:52:49.102450 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:49.102305    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 17:52:49.102450 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:49.102388    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a01b635-81f5-4c48-9906-dbf914e08592-metrics-tls podName:0a01b635-81f5-4c48-9906-dbf914e08592 nodeName:}" failed. No retries permitted until 2026-08-04 17:53:05.102365549 +0000 UTC m=+67.285525710 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a01b635-81f5-4c48-9906-dbf914e08592-metrics-tls") pod "dns-default-d58jd" (UID: "0a01b635-81f5-4c48-9906-dbf914e08592") : secret "dns-default-metrics-tls" not found
Aug 04 17:52:49.102556 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:49.102453    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f583466f-47c3-4403-a048-4c48550ac4fc-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-stb8t\" (UID: \"f583466f-47c3-4403-a048-4c48550ac4fc\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-stb8t"
Aug 04 17:52:49.102614 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:49.102574    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 04 17:52:49.102657 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:49.102621    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f583466f-47c3-4403-a048-4c48550ac4fc-networking-console-plugin-cert podName:f583466f-47c3-4403-a048-4c48550ac4fc nodeName:}" failed. No retries permitted until 2026-08-04 17:53:05.102605422 +0000 UTC m=+67.285765603 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f583466f-47c3-4403-a048-4c48550ac4fc-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-stb8t" (UID: "f583466f-47c3-4403-a048-4c48550ac4fc") : secret "networking-console-plugin-cert" not found
Aug 04 17:52:49.104668 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:49.104639    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f49077e4-25d7-4cf7-8072-87c5a5e13829-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-t4zw5\" (UID: \"f49077e4-25d7-4cf7-8072-87c5a5e13829\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-t4zw5"
Aug 04 17:52:49.138582 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:49.138555    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-t4zw5"
Aug 04 17:52:49.295549 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:49.295483    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-t4zw5"]
Aug 04 17:52:49.641106 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:49.641032    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-t4zw5" event={"ID":"f49077e4-25d7-4cf7-8072-87c5a5e13829","Type":"ContainerStarted","Data":"30dba23c8c8b8979fa5d86c55ee1943126bd9ec13dd724ac1c0dd7b5a46de3de"}
Aug 04 17:52:51.119583 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:51.119550    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/568af22c-f1cf-44ba-abe1-508565a26ebd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c94lk\" (UID: \"568af22c-f1cf-44ba-abe1-508565a26ebd\") " pod="openshift-insights/insights-runtime-extractor-c94lk"
Aug 04 17:52:51.120252 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:51.119700    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/568af22c-f1cf-44ba-abe1-508565a26ebd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c94lk\" (UID: \"568af22c-f1cf-44ba-abe1-508565a26ebd\") " pod="openshift-insights/insights-runtime-extractor-c94lk"
Aug 04 17:52:51.120252 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:51.119707    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 17:52:51.120252 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:51.119803    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/568af22c-f1cf-44ba-abe1-508565a26ebd-insights-runtime-extractor-tls podName:568af22c-f1cf-44ba-abe1-508565a26ebd nodeName:}" failed. No retries permitted until 2026-08-04 17:52:59.119784422 +0000 UTC m=+61.302944596 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/568af22c-f1cf-44ba-abe1-508565a26ebd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c94lk" (UID: "568af22c-f1cf-44ba-abe1-508565a26ebd") : secret "insights-runtime-extractor-tls" not found
Aug 04 17:52:51.120430 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:51.120250    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/568af22c-f1cf-44ba-abe1-508565a26ebd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c94lk\" (UID: \"568af22c-f1cf-44ba-abe1-508565a26ebd\") " pod="openshift-insights/insights-runtime-extractor-c94lk"
Aug 04 17:52:52.653544 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:52.653500    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-t4zw5" event={"ID":"f49077e4-25d7-4cf7-8072-87c5a5e13829","Type":"ContainerStarted","Data":"a686a96677bc1de5bf5580426fa3aff94209631ae54f0d8bb05db9823fadc4f7"}
Aug 04 17:52:52.653926 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:52.653556    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-t4zw5" event={"ID":"f49077e4-25d7-4cf7-8072-87c5a5e13829","Type":"ContainerStarted","Data":"f8c267e9bef290c11ffa67139b2f53924435d7ebd7f80fa19fa4b52f023cb793"}
Aug 04 17:52:52.676343 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:52.676300    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-t4zw5" podStartSLOduration=49.463715661 podStartE2EDuration="51.676286591s" totalImagesPullingTime="2.21257093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 17:52:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:52:52.675449324 +0000 UTC m=+54.858609501" watchObservedRunningTime="2026-08-04 17:52:52.676286591 +0000 UTC m=+54.859446766"
Aug 04 17:52:53.515356 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:53.515319    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf"
Aug 04 17:52:53.515543 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:53.515384    2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf"
Aug 04 17:52:53.515774 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:53.515755    2560 scope.go:117] "RemoveContainer" containerID="e5e7fc8ccc0e0a3594f202e61497d28a32f41be222379cbbb469eaea2213864b"
Aug 04 17:52:53.658300 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:53.658277    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/2.log"
Aug 04 17:52:53.658746 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:53.658733    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/1.log"
Aug 04 17:52:53.658795 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:53.658764    2560 generic.go:356] "Generic (PLEG): container finished" podID="8eaf7533-2fc5-4da2-bec2-282dce644049" containerID="09117ca5021af3fba9d9c7d2f3b48c589a2965c222279b94345c1f030444d24b" exitCode=255
Aug 04 17:52:53.658861 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:53.658842    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf" event={"ID":"8eaf7533-2fc5-4da2-bec2-282dce644049","Type":"ContainerDied","Data":"09117ca5021af3fba9d9c7d2f3b48c589a2965c222279b94345c1f030444d24b"}
Aug 04 17:52:53.658918 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:53.658899    2560 scope.go:117] "RemoveContainer" containerID="e5e7fc8ccc0e0a3594f202e61497d28a32f41be222379cbbb469eaea2213864b"
Aug 04 17:52:53.659407 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:53.659247    2560 scope.go:117] "RemoveContainer" containerID="09117ca5021af3fba9d9c7d2f3b48c589a2965c222279b94345c1f030444d24b"
Aug 04 17:52:53.659520 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:52:53.659411    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-5ccd6b445f-gtwkf_openshift-console-operator(8eaf7533-2fc5-4da2-bec2-282dce644049)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf" podUID="8eaf7533-2fc5-4da2-bec2-282dce644049"
Aug 04 17:52:54.666342 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:54.666317    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/2.log"
Aug 04 17:52:57.527617 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:57.527590    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2wj5g"
Aug 04 17:52:59.187637 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:59.187600    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/568af22c-f1cf-44ba-abe1-508565a26ebd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c94lk\" (UID: \"568af22c-f1cf-44ba-abe1-508565a26ebd\") " pod="openshift-insights/insights-runtime-extractor-c94lk"
Aug 04 17:52:59.190101 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:59.190078    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/568af22c-f1cf-44ba-abe1-508565a26ebd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c94lk\" (UID: \"568af22c-f1cf-44ba-abe1-508565a26ebd\") " pod="openshift-insights/insights-runtime-extractor-c94lk"
Aug 04 17:52:59.259873 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:59.259844    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kwp6w\""
Aug 04 17:52:59.266184 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:59.266169    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-c94lk"
Aug 04 17:52:59.404041 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:59.404018    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-c94lk"]
Aug 04 17:52:59.406366 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:52:59.406334    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod568af22c_f1cf_44ba_abe1_508565a26ebd.slice/crio-e808ab614315ae5952f38ae8e07ed66ee9cddfe8590e78dcf92e88be693d7072 WatchSource:0}: Error finding container e808ab614315ae5952f38ae8e07ed66ee9cddfe8590e78dcf92e88be693d7072: Status 404 returned error can't find the container with id e808ab614315ae5952f38ae8e07ed66ee9cddfe8590e78dcf92e88be693d7072
Aug 04 17:52:59.686357 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:59.686312    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-c94lk" event={"ID":"568af22c-f1cf-44ba-abe1-508565a26ebd","Type":"ContainerStarted","Data":"63cb1cac2db605c88260f50baefaf5b167a6fe4bea91360e2f40c8f8e1420b6b"}
Aug 04 17:52:59.686525 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:52:59.686429    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-c94lk" event={"ID":"568af22c-f1cf-44ba-abe1-508565a26ebd","Type":"ContainerStarted","Data":"e808ab614315ae5952f38ae8e07ed66ee9cddfe8590e78dcf92e88be693d7072"}
Aug 04 17:53:00.691718 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:00.691684    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-c94lk" event={"ID":"568af22c-f1cf-44ba-abe1-508565a26ebd","Type":"ContainerStarted","Data":"a954c0e72c14d202a26eaa1f6f088ce83845a80a2068182951710670ca85894c"}
Aug 04 17:53:02.700361 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:02.700322    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-c94lk" event={"ID":"568af22c-f1cf-44ba-abe1-508565a26ebd","Type":"ContainerStarted","Data":"ece1377d0274e9b841a4ac6398f1bc291ac231e2c7196eed3c2422b0362c4144"}
Aug 04 17:53:02.734695 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:02.734645    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-c94lk" podStartSLOduration=17.306275539 podStartE2EDuration="19.73462987s" totalImagesPullingTime="2.428354331s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 17:52:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:53:02.734378249 +0000 UTC m=+64.917538442" watchObservedRunningTime="2026-08-04 17:53:02.73462987 +0000 UTC m=+64.917790090"
Aug 04 17:53:03.516047 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:03.516014    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf"
Aug 04 17:53:03.516220 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:03.516074    2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf"
Aug 04 17:53:03.516378 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:03.516364    2560 scope.go:117] "RemoveContainer" containerID="09117ca5021af3fba9d9c7d2f3b48c589a2965c222279b94345c1f030444d24b"
Aug 04 17:53:03.516541 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:53:03.516522    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-5ccd6b445f-gtwkf_openshift-console-operator(8eaf7533-2fc5-4da2-bec2-282dce644049)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf" podUID="8eaf7533-2fc5-4da2-bec2-282dce644049"
Aug 04 17:53:04.027855 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:04.027806    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5ee91876-14a5-4691-9f54-0d3d11cf70e3-original-pull-secret\") pod \"global-pull-secret-syncer-jgm5s\" (UID: \"5ee91876-14a5-4691-9f54-0d3d11cf70e3\") " pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:53:04.031156 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:04.031136    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 04 17:53:04.040651 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:04.040626    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5ee91876-14a5-4691-9f54-0d3d11cf70e3-original-pull-secret\") pod \"global-pull-secret-syncer-jgm5s\" (UID: \"5ee91876-14a5-4691-9f54-0d3d11cf70e3\") " pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:53:04.128908 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:04.128878    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmwgb\" (UniqueName: \"kubernetes.io/projected/fba423bd-0c4b-4b5c-851d-f604236048c3-kube-api-access-vmwgb\") pod \"network-check-target-5x9t4\" (UID: \"fba423bd-0c4b-4b5c-851d-f604236048c3\") " pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:53:04.131211 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:04.131194    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmwgb\" (UniqueName: \"kubernetes.io/projected/fba423bd-0c4b-4b5c-851d-f604236048c3-kube-api-access-vmwgb\") pod \"network-check-target-5x9t4\" (UID: \"fba423bd-0c4b-4b5c-851d-f604236048c3\") " pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:53:04.229591 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:04.229565    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b581e21-8a6f-4ed7-8751-87941a6cbac3-metrics-certs\") pod \"network-metrics-daemon-sstpm\" (UID: \"6b581e21-8a6f-4ed7-8751-87941a6cbac3\") " pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:53:04.232723 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:04.232708    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 04 17:53:04.242004 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:04.241962    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b581e21-8a6f-4ed7-8751-87941a6cbac3-metrics-certs\") pod \"network-metrics-daemon-sstpm\" (UID: \"6b581e21-8a6f-4ed7-8751-87941a6cbac3\") " pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:53:04.266182 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:04.266167    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jgm5s"
Aug 04 17:53:04.277900 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:04.277852    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-f7jt5\""
Aug 04 17:53:04.279967 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:04.279951    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-smv6k\""
Aug 04 17:53:04.283032 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:04.283015    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:53:04.288004 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:04.287964    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sstpm"
Aug 04 17:53:04.416468 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:04.416437    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jgm5s"]
Aug 04 17:53:04.421054 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:53:04.421032    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ee91876_14a5_4691_9f54_0d3d11cf70e3.slice/crio-a1fbe4475ecd0e43f317f453f6e5d871044a1e1d09e4ac57e899275222775274 WatchSource:0}: Error finding container a1fbe4475ecd0e43f317f453f6e5d871044a1e1d09e4ac57e899275222775274: Status 404 returned error can't find the container with id a1fbe4475ecd0e43f317f453f6e5d871044a1e1d09e4ac57e899275222775274
Aug 04 17:53:04.441980 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:04.441952    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5x9t4"]
Aug 04 17:53:04.443768 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:53:04.443743    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfba423bd_0c4b_4b5c_851d_f604236048c3.slice/crio-0eb6ab8187284c3dd53d56267b5ba067c4a9c4af63fd1d9a8f847ac6f0832a53 WatchSource:0}: Error finding container 0eb6ab8187284c3dd53d56267b5ba067c4a9c4af63fd1d9a8f847ac6f0832a53: Status 404 returned error can't find the container with id 0eb6ab8187284c3dd53d56267b5ba067c4a9c4af63fd1d9a8f847ac6f0832a53
Aug 04 17:53:04.460965 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:04.460937    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sstpm"]
Aug 04 17:53:04.462805 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:53:04.462783    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b581e21_8a6f_4ed7_8751_87941a6cbac3.slice/crio-7d27fb183667d1bc826ae44d692cfd59dad2db423cbeaddfb6277d8a7716ca75 WatchSource:0}: Error finding container 7d27fb183667d1bc826ae44d692cfd59dad2db423cbeaddfb6277d8a7716ca75: Status 404 returned error can't find the container with id 7d27fb183667d1bc826ae44d692cfd59dad2db423cbeaddfb6277d8a7716ca75
Aug 04 17:53:04.707750 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:04.707720    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5x9t4" event={"ID":"fba423bd-0c4b-4b5c-851d-f604236048c3","Type":"ContainerStarted","Data":"39b48e315dd9f84c91467d5e94bbc5432988476d318a4500fd1c004acd93811c"}
Aug 04 17:53:04.707867 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:04.707774    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5x9t4" event={"ID":"fba423bd-0c4b-4b5c-851d-f604236048c3","Type":"ContainerStarted","Data":"0eb6ab8187284c3dd53d56267b5ba067c4a9c4af63fd1d9a8f847ac6f0832a53"}
Aug 04 17:53:04.707867 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:04.707797    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:53:04.708788 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:04.708768    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jgm5s" event={"ID":"5ee91876-14a5-4691-9f54-0d3d11cf70e3","Type":"ContainerStarted","Data":"a1fbe4475ecd0e43f317f453f6e5d871044a1e1d09e4ac57e899275222775274"}
Aug 04 17:53:04.709803 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:04.709783    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sstpm" event={"ID":"6b581e21-8a6f-4ed7-8751-87941a6cbac3","Type":"ContainerStarted","Data":"7d27fb183667d1bc826ae44d692cfd59dad2db423cbeaddfb6277d8a7716ca75"}
Aug 04 17:53:04.744862 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:04.744825    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5x9t4" podStartSLOduration=66.744811783 podStartE2EDuration="1m6.744811783s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 17:51:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:53:04.743971716 +0000 UTC m=+66.927131891" watchObservedRunningTime="2026-08-04 17:53:04.744811783 +0000 UTC m=+66.927971951"
Aug 04 17:53:05.036151 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.036087    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-registry-tls\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:53:05.036151 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.036139    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b35d3198-f4e3-44a5-a0b7-08c987b5805f-metrics-certs\") pod \"router-default-5955597dc4-jbsv8\" (UID: \"b35d3198-f4e3-44a5-a0b7-08c987b5805f\") " pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:53:05.036518 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.036166    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b35d3198-f4e3-44a5-a0b7-08c987b5805f-service-ca-bundle\") pod \"router-default-5955597dc4-jbsv8\" (UID: \"b35d3198-f4e3-44a5-a0b7-08c987b5805f\") " pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:53:05.036518 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.036197    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/165249a4-0e3d-4f20-9719-c91a6d7ce78f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-kxp92\" (UID: \"165249a4-0e3d-4f20-9719-c91a6d7ce78f\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-kxp92"
Aug 04 17:53:05.036849 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.036826    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b35d3198-f4e3-44a5-a0b7-08c987b5805f-service-ca-bundle\") pod \"router-default-5955597dc4-jbsv8\" (UID: \"b35d3198-f4e3-44a5-a0b7-08c987b5805f\") " pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:53:05.038587 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.038566    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/165249a4-0e3d-4f20-9719-c91a6d7ce78f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-kxp92\" (UID: \"165249a4-0e3d-4f20-9719-c91a6d7ce78f\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-kxp92"
Aug 04 17:53:05.038668 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.038585    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b35d3198-f4e3-44a5-a0b7-08c987b5805f-metrics-certs\") pod \"router-default-5955597dc4-jbsv8\" (UID: \"b35d3198-f4e3-44a5-a0b7-08c987b5805f\") " pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:53:05.038668 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.038612    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-registry-tls\") pod \"image-registry-69bc56d479-dx5kz\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") " pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:53:05.137593 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.137565    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f583466f-47c3-4403-a048-4c48550ac4fc-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-stb8t\" (UID: \"f583466f-47c3-4403-a048-4c48550ac4fc\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-stb8t"
Aug 04 17:53:05.137733 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.137632    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a01b635-81f5-4c48-9906-dbf914e08592-metrics-tls\") pod \"dns-default-d58jd\" (UID: \"0a01b635-81f5-4c48-9906-dbf914e08592\") " pod="openshift-dns/dns-default-d58jd"
Aug 04 17:53:05.140293 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.140265    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f583466f-47c3-4403-a048-4c48550ac4fc-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-stb8t\" (UID: \"f583466f-47c3-4403-a048-4c48550ac4fc\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-stb8t"
Aug 04 17:53:05.143690 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.143666    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a01b635-81f5-4c48-9906-dbf914e08592-metrics-tls\") pod \"dns-default-d58jd\" (UID: \"0a01b635-81f5-4c48-9906-dbf914e08592\") " pod="openshift-dns/dns-default-d58jd"
Aug 04 17:53:05.253041 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.253011    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-mvlcb\""
Aug 04 17:53:05.261259 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.261225    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-kxp92"
Aug 04 17:53:05.267311 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.267287    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-2frj8\""
Aug 04 17:53:05.274879 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.274860    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:53:05.279751 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.279723    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kwcpz\""
Aug 04 17:53:05.287265 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.287149    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:53:05.325359 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.325136    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-pblgh\""
Aug 04 17:53:05.334343 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.334317    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-stb8t"
Aug 04 17:53:05.361021 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.359044    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-x4v5p\""
Aug 04 17:53:05.368016 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.361602    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-d58jd"
Aug 04 17:53:05.497799 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.497004    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-kxp92"]
Aug 04 17:53:05.526435 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.526410    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5955597dc4-jbsv8"]
Aug 04 17:53:05.575611 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.575569    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69bc56d479-dx5kz"]
Aug 04 17:53:05.579039 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:53:05.579014    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod887c097d_64f3_4f3f_99ac_7981ab483724.slice/crio-197a8df10beb90b1c81a4231f00b9b2aacf29961f1e5852319852867c3fd698c WatchSource:0}: Error finding container 197a8df10beb90b1c81a4231f00b9b2aacf29961f1e5852319852867c3fd698c: Status 404 returned error can't find the container with id 197a8df10beb90b1c81a4231f00b9b2aacf29961f1e5852319852867c3fd698c
Aug 04 17:53:05.598969 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.598261    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-stb8t"]
Aug 04 17:53:05.639724 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.639450    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-d58jd"]
Aug 04 17:53:05.640489 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:53:05.640458    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a01b635_81f5_4c48_9906_dbf914e08592.slice/crio-f9e19d5fd3aab406b44c00964ea20e0fa7bc4bca0a720391a686a5f1da403140 WatchSource:0}: Error finding container f9e19d5fd3aab406b44c00964ea20e0fa7bc4bca0a720391a686a5f1da403140: Status 404 returned error can't find the container with id f9e19d5fd3aab406b44c00964ea20e0fa7bc4bca0a720391a686a5f1da403140
Aug 04 17:53:05.721505 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.721465    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-stb8t" event={"ID":"f583466f-47c3-4403-a048-4c48550ac4fc","Type":"ContainerStarted","Data":"6cb6fe7c2dc8e38196fd44e31a47c352ac5bb57b6aba5a8df28283c60671e34f"}
Aug 04 17:53:05.724336 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.724013    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5955597dc4-jbsv8" event={"ID":"b35d3198-f4e3-44a5-a0b7-08c987b5805f","Type":"ContainerStarted","Data":"7ca92573dae664ba611bb5a13493a2c88953eaf2bd398c0560cdc9a563437f8c"}
Aug 04 17:53:05.724336 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.724064    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5955597dc4-jbsv8" event={"ID":"b35d3198-f4e3-44a5-a0b7-08c987b5805f","Type":"ContainerStarted","Data":"9f3ffd7332ff13b53f94264a9c25f92c97ea8a4daa5516bc5eff26693e3b1210"}
Aug 04 17:53:05.725678 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.725653    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-d58jd" event={"ID":"0a01b635-81f5-4c48-9906-dbf914e08592","Type":"ContainerStarted","Data":"f9e19d5fd3aab406b44c00964ea20e0fa7bc4bca0a720391a686a5f1da403140"}
Aug 04 17:53:05.728530 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.728507    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69bc56d479-dx5kz" event={"ID":"887c097d-64f3-4f3f-99ac-7981ab483724","Type":"ContainerStarted","Data":"24b72002a63d2c6b0a4398ef75c48a149bfb907e67cf52934d00b856895241c8"}
Aug 04 17:53:05.728623 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.728544    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69bc56d479-dx5kz" event={"ID":"887c097d-64f3-4f3f-99ac-7981ab483724","Type":"ContainerStarted","Data":"197a8df10beb90b1c81a4231f00b9b2aacf29961f1e5852319852867c3fd698c"}
Aug 04 17:53:05.728934 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.728919    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:53:05.730696 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.730631    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-kxp92" event={"ID":"165249a4-0e3d-4f20-9719-c91a6d7ce78f","Type":"ContainerStarted","Data":"6d0912c63a290e16998e508e9ed5ae892b92337d1c628784576b92157994ad9e"}
Aug 04 17:53:05.745565 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.745520    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5955597dc4-jbsv8" podStartSLOduration=64.745506092 podStartE2EDuration="1m4.745506092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 17:52:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:53:05.745299185 +0000 UTC m=+67.928459361" watchObservedRunningTime="2026-08-04 17:53:05.745506092 +0000 UTC m=+67.928666269"
Aug 04 17:53:05.774010 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:05.773511    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-69bc56d479-dx5kz" podStartSLOduration=67.773492513 podStartE2EDuration="1m7.773492513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 17:51:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:53:05.772241804 +0000 UTC m=+67.955401981" watchObservedRunningTime="2026-08-04 17:53:05.773492513 +0000 UTC m=+67.956652691"
Aug 04 17:53:06.276051 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:06.276014    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:53:06.278698 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:06.278677    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:53:06.739914 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:06.739878    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sstpm" event={"ID":"6b581e21-8a6f-4ed7-8751-87941a6cbac3","Type":"ContainerStarted","Data":"f87c6433945480343a55b16ea1d6e7e89f951e9060665b4001113dc2e8080442"}
Aug 04 17:53:06.740143 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:06.739937    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sstpm" event={"ID":"6b581e21-8a6f-4ed7-8751-87941a6cbac3","Type":"ContainerStarted","Data":"f3cc0081e6f0a28abb62ce8c812279c894cc8aeadd0243515728c561d7e1b8f5"}
Aug 04 17:53:06.740143 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:06.740020    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:53:06.740827 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:06.740789    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5955597dc4-jbsv8"
Aug 04 17:53:06.762716 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:06.762465    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-sstpm" podStartSLOduration=66.915148989 podStartE2EDuration="1m8.7624473s" totalImagesPullingTime="1.847298311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 17:51:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:53:06.759763652 +0000 UTC m=+68.942923828" watchObservedRunningTime="2026-08-04 17:53:06.7624473 +0000 UTC m=+68.945607477"
Aug 04 17:53:07.703381 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.702677    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b96f974f-gk8cd"]
Aug 04 17:53:07.710254 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.705982    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8f454c54d-szxr2"]
Aug 04 17:53:07.710254 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.706102    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b96f974f-gk8cd"
Aug 04 17:53:07.710254 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.708668    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Aug 04 17:53:07.710511 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.710365    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Aug 04 17:53:07.710570 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.710516    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Aug 04 17:53:07.711019 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.710657    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Aug 04 17:53:07.713656 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.713624    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf"]
Aug 04 17:53:07.713782 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.713750    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8f454c54d-szxr2"
Aug 04 17:53:07.723710 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.721081    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-jg465\""
Aug 04 17:53:07.724552 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.724151    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Aug 04 17:53:07.731899 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.730434    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf"
Aug 04 17:53:07.734038 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.732535    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8f454c54d-szxr2"]
Aug 04 17:53:07.736446 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.736426    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Aug 04 17:53:07.736636 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.736619    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Aug 04 17:53:07.736762 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.736746    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Aug 04 17:53:07.737020 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.736970    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Aug 04 17:53:07.737339 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.737310    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b96f974f-gk8cd"]
Aug 04 17:53:07.740669 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.740650    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf"]
Aug 04 17:53:07.863639 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.863609    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0f6758a7-eac1-43db-a1ae-0ab1f5ca1794-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8f454c54d-szxr2\" (UID: \"0f6758a7-eac1-43db-a1ae-0ab1f5ca1794\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8f454c54d-szxr2"
Aug 04 17:53:07.863788 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.863644    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5668e200-9383-4699-a08a-d7a95bf12489-hub\") pod \"cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf\" (UID: \"5668e200-9383-4699-a08a-d7a95bf12489\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf"
Aug 04 17:53:07.863788 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.863700    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-644gg\" (UniqueName: \"kubernetes.io/projected/197611ae-97d3-4385-afdc-a65c1b708c2e-kube-api-access-644gg\") pod \"klusterlet-addon-workmgr-55b96f974f-gk8cd\" (UID: \"197611ae-97d3-4385-afdc-a65c1b708c2e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b96f974f-gk8cd"
Aug 04 17:53:07.863788 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.863775    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5668e200-9383-4699-a08a-d7a95bf12489-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf\" (UID: \"5668e200-9383-4699-a08a-d7a95bf12489\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf"
Aug 04 17:53:07.863917 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.863897    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/197611ae-97d3-4385-afdc-a65c1b708c2e-klusterlet-config\") pod \"klusterlet-addon-workmgr-55b96f974f-gk8cd\" (UID: \"197611ae-97d3-4385-afdc-a65c1b708c2e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b96f974f-gk8cd"
Aug 04 17:53:07.863971 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.863932    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5668e200-9383-4699-a08a-d7a95bf12489-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf\" (UID: \"5668e200-9383-4699-a08a-d7a95bf12489\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf"
Aug 04 17:53:07.864048 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.863973    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvbwg\" (UniqueName: \"kubernetes.io/projected/5668e200-9383-4699-a08a-d7a95bf12489-kube-api-access-tvbwg\") pod \"cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf\" (UID: \"5668e200-9383-4699-a08a-d7a95bf12489\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf"
Aug 04 17:53:07.864109 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.864061    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d65zg\" (UniqueName: \"kubernetes.io/projected/0f6758a7-eac1-43db-a1ae-0ab1f5ca1794-kube-api-access-d65zg\") pod \"managed-serviceaccount-addon-agent-8f454c54d-szxr2\" (UID: \"0f6758a7-eac1-43db-a1ae-0ab1f5ca1794\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8f454c54d-szxr2"
Aug 04 17:53:07.864171 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.864137    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5668e200-9383-4699-a08a-d7a95bf12489-ca\") pod \"cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf\" (UID: \"5668e200-9383-4699-a08a-d7a95bf12489\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf"
Aug 04 17:53:07.864210 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.864167    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5668e200-9383-4699-a08a-d7a95bf12489-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf\" (UID: \"5668e200-9383-4699-a08a-d7a95bf12489\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf"
Aug 04 17:53:07.864522 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.864473    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/197611ae-97d3-4385-afdc-a65c1b708c2e-tmp\") pod \"klusterlet-addon-workmgr-55b96f974f-gk8cd\" (UID: \"197611ae-97d3-4385-afdc-a65c1b708c2e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b96f974f-gk8cd"
Aug 04 17:53:07.965238 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.965154    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0f6758a7-eac1-43db-a1ae-0ab1f5ca1794-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8f454c54d-szxr2\" (UID: \"0f6758a7-eac1-43db-a1ae-0ab1f5ca1794\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8f454c54d-szxr2"
Aug 04 17:53:07.965238 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.965197    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5668e200-9383-4699-a08a-d7a95bf12489-hub\") pod \"cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf\" (UID: \"5668e200-9383-4699-a08a-d7a95bf12489\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf"
Aug 04 17:53:07.965238 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.965223    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-644gg\" (UniqueName: \"kubernetes.io/projected/197611ae-97d3-4385-afdc-a65c1b708c2e-kube-api-access-644gg\") pod \"klusterlet-addon-workmgr-55b96f974f-gk8cd\" (UID: \"197611ae-97d3-4385-afdc-a65c1b708c2e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b96f974f-gk8cd"
Aug 04 17:53:07.965497 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.965248    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5668e200-9383-4699-a08a-d7a95bf12489-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf\" (UID: \"5668e200-9383-4699-a08a-d7a95bf12489\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf"
Aug 04 17:53:07.965497 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.965286    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/197611ae-97d3-4385-afdc-a65c1b708c2e-klusterlet-config\") pod \"klusterlet-addon-workmgr-55b96f974f-gk8cd\" (UID: \"197611ae-97d3-4385-afdc-a65c1b708c2e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b96f974f-gk8cd"
Aug 04 17:53:07.965497 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.965308    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5668e200-9383-4699-a08a-d7a95bf12489-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf\" (UID: \"5668e200-9383-4699-a08a-d7a95bf12489\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf"
Aug 04 17:53:07.965497 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.965331    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvbwg\" (UniqueName: \"kubernetes.io/projected/5668e200-9383-4699-a08a-d7a95bf12489-kube-api-access-tvbwg\") pod \"cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf\" (UID: \"5668e200-9383-4699-a08a-d7a95bf12489\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf"
Aug 04 17:53:07.965497 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.965358    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d65zg\" (UniqueName: \"kubernetes.io/projected/0f6758a7-eac1-43db-a1ae-0ab1f5ca1794-kube-api-access-d65zg\") pod \"managed-serviceaccount-addon-agent-8f454c54d-szxr2\" (UID: \"0f6758a7-eac1-43db-a1ae-0ab1f5ca1794\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8f454c54d-szxr2"
Aug 04 17:53:07.965497 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.965392    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5668e200-9383-4699-a08a-d7a95bf12489-ca\") pod \"cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf\" (UID: \"5668e200-9383-4699-a08a-d7a95bf12489\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf"
Aug 04 17:53:07.965497 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.965416    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5668e200-9383-4699-a08a-d7a95bf12489-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf\" (UID: \"5668e200-9383-4699-a08a-d7a95bf12489\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf"
Aug 04 17:53:07.965497 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.965483    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/197611ae-97d3-4385-afdc-a65c1b708c2e-tmp\") pod \"klusterlet-addon-workmgr-55b96f974f-gk8cd\" (UID: \"197611ae-97d3-4385-afdc-a65c1b708c2e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b96f974f-gk8cd"
Aug 04 17:53:07.965932 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.965911    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/197611ae-97d3-4385-afdc-a65c1b708c2e-tmp\") pod \"klusterlet-addon-workmgr-55b96f974f-gk8cd\" (UID: \"197611ae-97d3-4385-afdc-a65c1b708c2e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b96f974f-gk8cd"
Aug 04 17:53:07.966721 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.966695    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5668e200-9383-4699-a08a-d7a95bf12489-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf\" (UID: \"5668e200-9383-4699-a08a-d7a95bf12489\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf"
Aug 04 17:53:07.969548 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.969524    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5668e200-9383-4699-a08a-d7a95bf12489-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf\" (UID: \"5668e200-9383-4699-a08a-d7a95bf12489\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf"
Aug 04 17:53:07.969668 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.969642    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5668e200-9383-4699-a08a-d7a95bf12489-ca\") pod \"cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf\" (UID: \"5668e200-9383-4699-a08a-d7a95bf12489\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf"
Aug 04 17:53:07.970113 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.970081    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5668e200-9383-4699-a08a-d7a95bf12489-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf\" (UID: \"5668e200-9383-4699-a08a-d7a95bf12489\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf"
Aug 04 17:53:07.971372 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.971345    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5668e200-9383-4699-a08a-d7a95bf12489-hub\") pod \"cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf\" (UID: \"5668e200-9383-4699-a08a-d7a95bf12489\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf"
Aug 04 17:53:07.971372 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.971362    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/197611ae-97d3-4385-afdc-a65c1b708c2e-klusterlet-config\") pod \"klusterlet-addon-workmgr-55b96f974f-gk8cd\" (UID: \"197611ae-97d3-4385-afdc-a65c1b708c2e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b96f974f-gk8cd"
Aug 04 17:53:07.971971 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.971940    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0f6758a7-eac1-43db-a1ae-0ab1f5ca1794-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8f454c54d-szxr2\" (UID: \"0f6758a7-eac1-43db-a1ae-0ab1f5ca1794\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8f454c54d-szxr2"
Aug 04 17:53:07.975882 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.975859    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-644gg\" (UniqueName: \"kubernetes.io/projected/197611ae-97d3-4385-afdc-a65c1b708c2e-kube-api-access-644gg\") pod \"klusterlet-addon-workmgr-55b96f974f-gk8cd\" (UID: \"197611ae-97d3-4385-afdc-a65c1b708c2e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b96f974f-gk8cd"
Aug 04 17:53:07.977353 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.977328    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvbwg\" (UniqueName: \"kubernetes.io/projected/5668e200-9383-4699-a08a-d7a95bf12489-kube-api-access-tvbwg\") pod \"cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf\" (UID: \"5668e200-9383-4699-a08a-d7a95bf12489\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf"
Aug 04 17:53:07.979278 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:07.979231    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d65zg\" (UniqueName: \"kubernetes.io/projected/0f6758a7-eac1-43db-a1ae-0ab1f5ca1794-kube-api-access-d65zg\") pod \"managed-serviceaccount-addon-agent-8f454c54d-szxr2\" (UID: \"0f6758a7-eac1-43db-a1ae-0ab1f5ca1794\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8f454c54d-szxr2"
Aug 04 17:53:08.036048 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:08.036006    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b96f974f-gk8cd"
Aug 04 17:53:08.058661 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:08.058630    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8f454c54d-szxr2"
Aug 04 17:53:08.065852 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:08.065834    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf"
Aug 04 17:53:10.098689 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:10.098656    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8f454c54d-szxr2"]
Aug 04 17:53:10.131531 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:10.130348    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf"]
Aug 04 17:53:10.132840 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:53:10.132815    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5668e200_9383_4699_a08a_d7a95bf12489.slice/crio-a8485d5da93868b856085e06fc1451a6f442843778fb4b1130cd5d26cf71088a WatchSource:0}: Error finding container a8485d5da93868b856085e06fc1451a6f442843778fb4b1130cd5d26cf71088a: Status 404 returned error can't find the container with id a8485d5da93868b856085e06fc1451a6f442843778fb4b1130cd5d26cf71088a
Aug 04 17:53:10.329052 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:10.329017    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b96f974f-gk8cd"]
Aug 04 17:53:10.332346 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:53:10.332317    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod197611ae_97d3_4385_afdc_a65c1b708c2e.slice/crio-6958d59006bf97518e530dc58a68491c4c2f038b6e6e7703b4f612d8fc452d6f WatchSource:0}: Error finding container 6958d59006bf97518e530dc58a68491c4c2f038b6e6e7703b4f612d8fc452d6f: Status 404 returned error can't find the container with id 6958d59006bf97518e530dc58a68491c4c2f038b6e6e7703b4f612d8fc452d6f
Aug 04 17:53:10.758765 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:10.758730    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b96f974f-gk8cd" event={"ID":"197611ae-97d3-4385-afdc-a65c1b708c2e","Type":"ContainerStarted","Data":"6958d59006bf97518e530dc58a68491c4c2f038b6e6e7703b4f612d8fc452d6f"}
Aug 04 17:53:10.760300 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:10.760271    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-kxp92" event={"ID":"165249a4-0e3d-4f20-9719-c91a6d7ce78f","Type":"ContainerStarted","Data":"cbfdbf40f756d9e30fce64ce78ffe28e36bb6aae3974c5e5099e69cc4eb24220"}
Aug 04 17:53:10.761496 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:10.761469    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf" event={"ID":"5668e200-9383-4699-a08a-d7a95bf12489","Type":"ContainerStarted","Data":"a8485d5da93868b856085e06fc1451a6f442843778fb4b1130cd5d26cf71088a"}
Aug 04 17:53:10.762633 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:10.762609    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8f454c54d-szxr2" event={"ID":"0f6758a7-eac1-43db-a1ae-0ab1f5ca1794","Type":"ContainerStarted","Data":"34beba9bc340c34a9c6699498dd575af62bc26b7a4f9ec7fd8e71a34f2d6ce3a"}
Aug 04 17:53:10.764144 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:10.764122    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-stb8t" event={"ID":"f583466f-47c3-4403-a048-4c48550ac4fc","Type":"ContainerStarted","Data":"defb0321175c72831b1ef3ab4dd6f82c9accc32ffd62e1bd4917ffbaa6d862f8"}
Aug 04 17:53:10.765523 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:10.765499    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jgm5s" event={"ID":"5ee91876-14a5-4691-9f54-0d3d11cf70e3","Type":"ContainerStarted","Data":"c4732e4d6e1f12ace8b00818548fcb941226007f44588fa53bc842cb0cb2b179"}
Aug 04 17:53:10.767274 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:10.767256    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-d58jd" event={"ID":"0a01b635-81f5-4c48-9906-dbf914e08592","Type":"ContainerStarted","Data":"197936e0c77fa76342d1d98b2d0027ea2f08ebd79a882a48863363f96b5f3b11"}
Aug 04 17:53:10.767350 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:10.767290    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-d58jd" event={"ID":"0a01b635-81f5-4c48-9906-dbf914e08592","Type":"ContainerStarted","Data":"157aeafc198e66de2e0999ba8a16fc6040e420867fc1d780ea84d162f8f38572"}
Aug 04 17:53:10.767422 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:10.767405    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-d58jd"
Aug 04 17:53:10.783869 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:10.783830    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-kxp92" podStartSLOduration=65.391779186 podStartE2EDuration="1m9.7838177s" totalImagesPullingTime="4.392038514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 17:52:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:53:10.78196006 +0000 UTC m=+72.965120235" watchObservedRunningTime="2026-08-04 17:53:10.7838177 +0000 UTC m=+72.966977884"
Aug 04 17:53:10.806271 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:10.806228    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-d58jd" podStartSLOduration=33.541082029 podStartE2EDuration="37.806218484s" totalImagesPullingTime="4.265136455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 17:52:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:53:10.805216572 +0000 UTC m=+72.988376749" watchObservedRunningTime="2026-08-04 17:53:10.806218484 +0000 UTC m=+72.989378660"
Aug 04 17:53:10.833325 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:10.833236    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-jgm5s" podStartSLOduration=67.261607198 podStartE2EDuration="1m12.833216079s" totalImagesPullingTime="5.571608881s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 17:51:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:53:10.832914992 +0000 UTC m=+73.016075168" watchObservedRunningTime="2026-08-04 17:53:10.833216079 +0000 UTC m=+73.016376270"
Aug 04 17:53:10.860545 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:10.860499    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-f77f48c6d-stb8t" podStartSLOduration=37.570092237 podStartE2EDuration="41.860486414s" totalImagesPullingTime="4.290394177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 17:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:53:10.858455853 +0000 UTC m=+73.041616031" watchObservedRunningTime="2026-08-04 17:53:10.860486414 +0000 UTC m=+73.043646590"
Aug 04 17:53:16.347568 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:16.347531    2560 scope.go:117] "RemoveContainer" containerID="09117ca5021af3fba9d9c7d2f3b48c589a2965c222279b94345c1f030444d24b"
Aug 04 17:53:16.798465 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:16.798393    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/2.log"
Aug 04 17:53:16.798620 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:16.798506    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf" event={"ID":"8eaf7533-2fc5-4da2-bec2-282dce644049","Type":"ContainerStarted","Data":"6ce14735bfdb0a60ed025c5d1d6803a069c7d2877a15c7c50a49fe71055a2a50"}
Aug 04 17:53:16.798869 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:16.798837    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf"
Aug 04 17:53:16.800842 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:16.800817    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf" event={"ID":"5668e200-9383-4699-a08a-d7a95bf12489","Type":"ContainerStarted","Data":"630c7d4b782e818f10c6f559f65878bff344ba8d26a241c14f773d80f6a512bb"}
Aug 04 17:53:16.800957 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:16.800852    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf" event={"ID":"5668e200-9383-4699-a08a-d7a95bf12489","Type":"ContainerStarted","Data":"453c3572270d9586c1fc398b5d59c32f2dbd1872ab46ba231b007d7051c719aa"}
Aug 04 17:53:16.800957 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:16.800870    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf" event={"ID":"5668e200-9383-4699-a08a-d7a95bf12489","Type":"ContainerStarted","Data":"a52902cac7706bff955f2821d2b42018fa0d8b35bcd0b904123ab7688e9e2ea2"}
Aug 04 17:53:16.802333 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:16.802309    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8f454c54d-szxr2" event={"ID":"0f6758a7-eac1-43db-a1ae-0ab1f5ca1794","Type":"ContainerStarted","Data":"33f60007db2aa7bcf161337f4402d39dcf74e20c57ef0176071d046893d3c650"}
Aug 04 17:53:16.860630 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:16.860595    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dbb86bdf8-hvnbf" podStartSLOduration=4.081859974 podStartE2EDuration="9.86058176s" totalImagesPullingTime="5.778721786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 17:53:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:53:16.860280122 +0000 UTC m=+79.043440298" watchObservedRunningTime="2026-08-04 17:53:16.86058176 +0000 UTC m=+79.043741940"
Aug 04 17:53:16.861057 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:16.861034    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf" podStartSLOduration=68.247346807 podStartE2EDuration="1m15.861027839s" totalImagesPullingTime="7.613681032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 17:52:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:53:16.830025168 +0000 UTC m=+79.013185544" watchObservedRunningTime="2026-08-04 17:53:16.861027839 +0000 UTC m=+79.044188015"
Aug 04 17:53:16.886804 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:16.886772    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8f454c54d-szxr2" podStartSLOduration=4.095148643 podStartE2EDuration="9.886760234s" totalImagesPullingTime="5.791611591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 17:53:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:53:16.884610471 +0000 UTC m=+79.067770644" watchObservedRunningTime="2026-08-04 17:53:16.886760234 +0000 UTC m=+79.069920409"
Aug 04 17:53:17.183557 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:17.183518    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-5ccd6b445f-gtwkf"
Aug 04 17:53:20.776504 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:20.776399    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-d58jd"
Aug 04 17:53:22.224910 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.224005    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gb6pp"]
Aug 04 17:53:22.229530 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.228746    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:22.232355 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.232191    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-7kr7n\""
Aug 04 17:53:22.232857 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.232820    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Aug 04 17:53:22.233254 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.233225    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Aug 04 17:53:22.233553 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.233535    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Aug 04 17:53:22.234938 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.234924    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Aug 04 17:53:22.290637 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.290608    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-sys\") pod \"node-exporter-gb6pp\" (UID: \"32e2da66-4bc5-4f64-9ff7-0d202f34d48e\") " pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:22.290750 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.290654    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-root\") pod \"node-exporter-gb6pp\" (UID: \"32e2da66-4bc5-4f64-9ff7-0d202f34d48e\") " pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:22.290750 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.290685    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-node-exporter-wtmp\") pod \"node-exporter-gb6pp\" (UID: \"32e2da66-4bc5-4f64-9ff7-0d202f34d48e\") " pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:22.290750 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.290719    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-metrics-client-ca\") pod \"node-exporter-gb6pp\" (UID: \"32e2da66-4bc5-4f64-9ff7-0d202f34d48e\") " pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:22.290750 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.290746    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gb6pp\" (UID: \"32e2da66-4bc5-4f64-9ff7-0d202f34d48e\") " pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:22.290954 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.290834    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-node-exporter-textfile\") pod \"node-exporter-gb6pp\" (UID: \"32e2da66-4bc5-4f64-9ff7-0d202f34d48e\") " pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:22.290954 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.290885    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dqqr\" (UniqueName: \"kubernetes.io/projected/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-kube-api-access-4dqqr\") pod \"node-exporter-gb6pp\" (UID: \"32e2da66-4bc5-4f64-9ff7-0d202f34d48e\") " pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:22.290954 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.290916    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-node-exporter-tls\") pod \"node-exporter-gb6pp\" (UID: \"32e2da66-4bc5-4f64-9ff7-0d202f34d48e\") " pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:22.291144 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.290982    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-node-exporter-accelerators-collector-config\") pod \"node-exporter-gb6pp\" (UID: \"32e2da66-4bc5-4f64-9ff7-0d202f34d48e\") " pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:22.392921 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.392891    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-node-exporter-textfile\") pod \"node-exporter-gb6pp\" (UID: \"32e2da66-4bc5-4f64-9ff7-0d202f34d48e\") " pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:22.393082 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.392951    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dqqr\" (UniqueName: \"kubernetes.io/projected/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-kube-api-access-4dqqr\") pod \"node-exporter-gb6pp\" (UID: \"32e2da66-4bc5-4f64-9ff7-0d202f34d48e\") " pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:22.393258 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:53:22.393239    2560 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Aug 04 17:53:22.393336 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.393315    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-node-exporter-textfile\") pod \"node-exporter-gb6pp\" (UID: \"32e2da66-4bc5-4f64-9ff7-0d202f34d48e\") " pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:22.393336 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:53:22.393322    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-node-exporter-tls podName:32e2da66-4bc5-4f64-9ff7-0d202f34d48e nodeName:}" failed. No retries permitted until 2026-08-04 17:53:22.893300355 +0000 UTC m=+85.076460513 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-node-exporter-tls") pod "node-exporter-gb6pp" (UID: "32e2da66-4bc5-4f64-9ff7-0d202f34d48e") : secret "node-exporter-tls" not found
Aug 04 17:53:22.393429 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.392982    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-node-exporter-tls\") pod \"node-exporter-gb6pp\" (UID: \"32e2da66-4bc5-4f64-9ff7-0d202f34d48e\") " pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:22.393429 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.393396    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-node-exporter-accelerators-collector-config\") pod \"node-exporter-gb6pp\" (UID: \"32e2da66-4bc5-4f64-9ff7-0d202f34d48e\") " pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:22.393499 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.393426    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-sys\") pod \"node-exporter-gb6pp\" (UID: \"32e2da66-4bc5-4f64-9ff7-0d202f34d48e\") " pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:22.393499 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.393460    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-root\") pod \"node-exporter-gb6pp\" (UID: \"32e2da66-4bc5-4f64-9ff7-0d202f34d48e\") " pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:22.393563 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.393503    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-node-exporter-wtmp\") pod \"node-exporter-gb6pp\" (UID: \"32e2da66-4bc5-4f64-9ff7-0d202f34d48e\") " pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:22.393563 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.393543    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-metrics-client-ca\") pod \"node-exporter-gb6pp\" (UID: \"32e2da66-4bc5-4f64-9ff7-0d202f34d48e\") " pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:22.393626 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.393576    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gb6pp\" (UID: \"32e2da66-4bc5-4f64-9ff7-0d202f34d48e\") " pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:22.393942 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.393925    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-root\") pod \"node-exporter-gb6pp\" (UID: \"32e2da66-4bc5-4f64-9ff7-0d202f34d48e\") " pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:22.394031 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.393981    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-sys\") pod \"node-exporter-gb6pp\" (UID: \"32e2da66-4bc5-4f64-9ff7-0d202f34d48e\") " pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:22.394089 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.394025    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-node-exporter-accelerators-collector-config\") pod \"node-exporter-gb6pp\" (UID: \"32e2da66-4bc5-4f64-9ff7-0d202f34d48e\") " pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:22.394142 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.394123    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-node-exporter-wtmp\") pod \"node-exporter-gb6pp\" (UID: \"32e2da66-4bc5-4f64-9ff7-0d202f34d48e\") " pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:22.394446 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.394424    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-metrics-client-ca\") pod \"node-exporter-gb6pp\" (UID: \"32e2da66-4bc5-4f64-9ff7-0d202f34d48e\") " pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:22.396260 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.396236    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gb6pp\" (UID: \"32e2da66-4bc5-4f64-9ff7-0d202f34d48e\") " pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:22.407687 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.407666    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dqqr\" (UniqueName: \"kubernetes.io/projected/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-kube-api-access-4dqqr\") pod \"node-exporter-gb6pp\" (UID: \"32e2da66-4bc5-4f64-9ff7-0d202f34d48e\") " pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:22.899500 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.899470    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-node-exporter-tls\") pod \"node-exporter-gb6pp\" (UID: \"32e2da66-4bc5-4f64-9ff7-0d202f34d48e\") " pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:22.901915 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:22.901890    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/32e2da66-4bc5-4f64-9ff7-0d202f34d48e-node-exporter-tls\") pod \"node-exporter-gb6pp\" (UID: \"32e2da66-4bc5-4f64-9ff7-0d202f34d48e\") " pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:23.143277 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:23.143240    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gb6pp"
Aug 04 17:53:23.157317 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:53:23.157292    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32e2da66_4bc5_4f64_9ff7_0d202f34d48e.slice/crio-d4681b9621566c816607850951b5a009b64be686c5af530a266e4673a0aa9aea WatchSource:0}: Error finding container d4681b9621566c816607850951b5a009b64be686c5af530a266e4673a0aa9aea: Status 404 returned error can't find the container with id d4681b9621566c816607850951b5a009b64be686c5af530a266e4673a0aa9aea
Aug 04 17:53:23.838489 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:23.838447    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gb6pp" event={"ID":"32e2da66-4bc5-4f64-9ff7-0d202f34d48e","Type":"ContainerStarted","Data":"d4681b9621566c816607850951b5a009b64be686c5af530a266e4673a0aa9aea"}
Aug 04 17:53:25.293457 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:25.293421    2560 patch_prober.go:28] interesting pod/image-registry-69bc56d479-dx5kz container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Aug 04 17:53:25.293879 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:25.293481    2560 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-69bc56d479-dx5kz" podUID="887c097d-64f3-4f3f-99ac-7981ab483724" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 04 17:53:25.848515 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:25.848436    2560 generic.go:356] "Generic (PLEG): container finished" podID="32e2da66-4bc5-4f64-9ff7-0d202f34d48e" containerID="63648aeef2cb89dcef8928c9866586657c541166d501e416fc3c6cb75e31fe9c" exitCode=0
Aug 04 17:53:25.848669 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:25.848569    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gb6pp" event={"ID":"32e2da66-4bc5-4f64-9ff7-0d202f34d48e","Type":"ContainerDied","Data":"63648aeef2cb89dcef8928c9866586657c541166d501e416fc3c6cb75e31fe9c"}
Aug 04 17:53:25.850092 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:25.850065    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b96f974f-gk8cd" event={"ID":"197611ae-97d3-4385-afdc-a65c1b708c2e","Type":"ContainerStarted","Data":"77851beac57490c1972b2eac457618523ab36baa1dcd8546f0ea3e3248ad9ae8"}
Aug 04 17:53:25.850292 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:25.850277    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b96f974f-gk8cd"
Aug 04 17:53:25.852356 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:25.852335    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b96f974f-gk8cd"
Aug 04 17:53:25.890407 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:25.890364    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55b96f974f-gk8cd" podStartSLOduration=3.660821577 podStartE2EDuration="18.890351363s" totalImagesPullingTime="15.229529786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 17:53:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:53:25.889486134 +0000 UTC m=+88.072646309" watchObservedRunningTime="2026-08-04 17:53:25.890351363 +0000 UTC m=+88.073511579"
Aug 04 17:53:26.855700 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:26.855652    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gb6pp" event={"ID":"32e2da66-4bc5-4f64-9ff7-0d202f34d48e","Type":"ContainerStarted","Data":"288a8b5809c4b735b5841e9c869d8c5d496cb7ab521d5b3f4c8cfccf93d80141"}
Aug 04 17:53:26.856179 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:26.855719    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gb6pp" event={"ID":"32e2da66-4bc5-4f64-9ff7-0d202f34d48e","Type":"ContainerStarted","Data":"2fa64d64bc57fb2d233447677bf5fc6fe869840688180a137fb5cfe5e03537b4"}
Aug 04 17:53:26.889783 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:26.889739    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gb6pp" podStartSLOduration=2.420620499 podStartE2EDuration="4.889725373s" totalImagesPullingTime="2.392225934s" totalInitContainerRuntime="76.87894ms" isStatefulPod=true podCreationTimestamp="2026-08-04 17:53:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:53:26.888328241 +0000 UTC m=+89.071488417" watchObservedRunningTime="2026-08-04 17:53:26.889725373 +0000 UTC m=+89.072885549"
Aug 04 17:53:27.751098 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:27.751069    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:53:32.014082 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:32.014044    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69bc56d479-dx5kz"]
Aug 04 17:53:35.733102 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:35.733065    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5x9t4"
Aug 04 17:53:52.952433 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:52.952398    2560 generic.go:356] "Generic (PLEG): container finished" podID="6b29597f-59c4-4f28-a164-7d2e97248d83" containerID="a1dd384f132b841755ab487547db8cc2ba7071828ef891034f47cd92d8d34b41" exitCode=0
Aug 04 17:53:52.952835 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:52.952475    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-ks7s5" event={"ID":"6b29597f-59c4-4f28-a164-7d2e97248d83","Type":"ContainerDied","Data":"a1dd384f132b841755ab487547db8cc2ba7071828ef891034f47cd92d8d34b41"}
Aug 04 17:53:52.952877 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:52.952848    2560 scope.go:117] "RemoveContainer" containerID="a1dd384f132b841755ab487547db8cc2ba7071828ef891034f47cd92d8d34b41"
Aug 04 17:53:53.958588 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:53.958553    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-ks7s5" event={"ID":"6b29597f-59c4-4f28-a164-7d2e97248d83","Type":"ContainerStarted","Data":"d80d209807b9f0b81ae1f8c64d795ddb006a5fc35a27c3f0db29a39f4693d8ce"}
Aug 04 17:53:57.034878 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.034791    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-69bc56d479-dx5kz" podUID="887c097d-64f3-4f3f-99ac-7981ab483724" containerName="registry" containerID="cri-o://24b72002a63d2c6b0a4398ef75c48a149bfb907e67cf52934d00b856895241c8" gracePeriod=30
Aug 04 17:53:57.295748 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.295676    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:53:57.373837 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.373804    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/887c097d-64f3-4f3f-99ac-7981ab483724-registry-certificates\") pod \"887c097d-64f3-4f3f-99ac-7981ab483724\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") "
Aug 04 17:53:57.374056 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.373850    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/887c097d-64f3-4f3f-99ac-7981ab483724-installation-pull-secrets\") pod \"887c097d-64f3-4f3f-99ac-7981ab483724\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") "
Aug 04 17:53:57.374056 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.373885    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-registry-tls\") pod \"887c097d-64f3-4f3f-99ac-7981ab483724\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") "
Aug 04 17:53:57.374056 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.373920    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-bound-sa-token\") pod \"887c097d-64f3-4f3f-99ac-7981ab483724\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") "
Aug 04 17:53:57.374056 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.373978    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/887c097d-64f3-4f3f-99ac-7981ab483724-image-registry-private-configuration\") pod \"887c097d-64f3-4f3f-99ac-7981ab483724\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") "
Aug 04 17:53:57.374056 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.374027    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/887c097d-64f3-4f3f-99ac-7981ab483724-trusted-ca\") pod \"887c097d-64f3-4f3f-99ac-7981ab483724\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") "
Aug 04 17:53:57.374326 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.374072    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ljf9z\" (UniqueName: \"kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-kube-api-access-ljf9z\") pod \"887c097d-64f3-4f3f-99ac-7981ab483724\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") "
Aug 04 17:53:57.374326 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.374100    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/887c097d-64f3-4f3f-99ac-7981ab483724-ca-trust-extracted\") pod \"887c097d-64f3-4f3f-99ac-7981ab483724\" (UID: \"887c097d-64f3-4f3f-99ac-7981ab483724\") "
Aug 04 17:53:57.374440 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.374327    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/887c097d-64f3-4f3f-99ac-7981ab483724-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "887c097d-64f3-4f3f-99ac-7981ab483724" (UID: "887c097d-64f3-4f3f-99ac-7981ab483724"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 17:53:57.374622 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.374587    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/887c097d-64f3-4f3f-99ac-7981ab483724-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "887c097d-64f3-4f3f-99ac-7981ab483724" (UID: "887c097d-64f3-4f3f-99ac-7981ab483724"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 17:53:57.376816 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.376723    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/887c097d-64f3-4f3f-99ac-7981ab483724-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "887c097d-64f3-4f3f-99ac-7981ab483724" (UID: "887c097d-64f3-4f3f-99ac-7981ab483724"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 17:53:57.376816 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.376789    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "887c097d-64f3-4f3f-99ac-7981ab483724" (UID: "887c097d-64f3-4f3f-99ac-7981ab483724"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 17:53:57.376816 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.376800    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "887c097d-64f3-4f3f-99ac-7981ab483724" (UID: "887c097d-64f3-4f3f-99ac-7981ab483724"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 17:53:57.377067 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.377022    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/887c097d-64f3-4f3f-99ac-7981ab483724-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "887c097d-64f3-4f3f-99ac-7981ab483724" (UID: "887c097d-64f3-4f3f-99ac-7981ab483724"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 17:53:57.377256 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.377222    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-kube-api-access-ljf9z" (OuterVolumeSpecName: "kube-api-access-ljf9z") pod "887c097d-64f3-4f3f-99ac-7981ab483724" (UID: "887c097d-64f3-4f3f-99ac-7981ab483724"). InnerVolumeSpecName "kube-api-access-ljf9z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 17:53:57.382671 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.382649    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/887c097d-64f3-4f3f-99ac-7981ab483724-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "887c097d-64f3-4f3f-99ac-7981ab483724" (UID: "887c097d-64f3-4f3f-99ac-7981ab483724"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 17:53:57.475936 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.475892    2560 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/887c097d-64f3-4f3f-99ac-7981ab483724-registry-certificates\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 17:53:57.475936 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.475928    2560 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/887c097d-64f3-4f3f-99ac-7981ab483724-installation-pull-secrets\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 17:53:57.475936 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.475943    2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-registry-tls\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 17:53:57.476194 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.475959    2560 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-bound-sa-token\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 17:53:57.476194 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.475975    2560 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/887c097d-64f3-4f3f-99ac-7981ab483724-image-registry-private-configuration\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 17:53:57.476194 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.476023    2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/887c097d-64f3-4f3f-99ac-7981ab483724-trusted-ca\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 17:53:57.476194 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.476039    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljf9z\" (UniqueName: \"kubernetes.io/projected/887c097d-64f3-4f3f-99ac-7981ab483724-kube-api-access-ljf9z\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 17:53:57.476194 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.476054    2560 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/887c097d-64f3-4f3f-99ac-7981ab483724-ca-trust-extracted\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 17:53:57.974355 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.974317    2560 generic.go:356] "Generic (PLEG): container finished" podID="887c097d-64f3-4f3f-99ac-7981ab483724" containerID="24b72002a63d2c6b0a4398ef75c48a149bfb907e67cf52934d00b856895241c8" exitCode=0
Aug 04 17:53:57.974603 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.974368    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69bc56d479-dx5kz"
Aug 04 17:53:57.974603 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.974413    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69bc56d479-dx5kz" event={"ID":"887c097d-64f3-4f3f-99ac-7981ab483724","Type":"ContainerDied","Data":"24b72002a63d2c6b0a4398ef75c48a149bfb907e67cf52934d00b856895241c8"}
Aug 04 17:53:57.974603 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.974471    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69bc56d479-dx5kz" event={"ID":"887c097d-64f3-4f3f-99ac-7981ab483724","Type":"ContainerDied","Data":"197a8df10beb90b1c81a4231f00b9b2aacf29961f1e5852319852867c3fd698c"}
Aug 04 17:53:57.974603 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.974497    2560 scope.go:117] "RemoveContainer" containerID="24b72002a63d2c6b0a4398ef75c48a149bfb907e67cf52934d00b856895241c8"
Aug 04 17:53:57.983102 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.983078    2560 scope.go:117] "RemoveContainer" containerID="24b72002a63d2c6b0a4398ef75c48a149bfb907e67cf52934d00b856895241c8"
Aug 04 17:53:57.983384 ip-10-0-128-172 kubenswrapper[2560]: E0804 17:53:57.983360    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24b72002a63d2c6b0a4398ef75c48a149bfb907e67cf52934d00b856895241c8\": container with ID starting with 24b72002a63d2c6b0a4398ef75c48a149bfb907e67cf52934d00b856895241c8 not found: ID does not exist" containerID="24b72002a63d2c6b0a4398ef75c48a149bfb907e67cf52934d00b856895241c8"
Aug 04 17:53:57.983469 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:57.983391    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24b72002a63d2c6b0a4398ef75c48a149bfb907e67cf52934d00b856895241c8"} err="failed to get container status \"24b72002a63d2c6b0a4398ef75c48a149bfb907e67cf52934d00b856895241c8\": rpc error: code = NotFound desc = could not find container \"24b72002a63d2c6b0a4398ef75c48a149bfb907e67cf52934d00b856895241c8\": container with ID starting with 24b72002a63d2c6b0a4398ef75c48a149bfb907e67cf52934d00b856895241c8 not found: ID does not exist"
Aug 04 17:53:58.002146 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:58.002119    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69bc56d479-dx5kz"]
Aug 04 17:53:58.008801 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:58.008770    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-69bc56d479-dx5kz"]
Aug 04 17:53:58.352537 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:53:58.352503    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="887c097d-64f3-4f3f-99ac-7981ab483724" path="/var/lib/kubelet/pods/887c097d-64f3-4f3f-99ac-7981ab483724/volumes"
Aug 04 17:54:02.998313 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:54:02.998276    2560 generic.go:356] "Generic (PLEG): container finished" podID="f3c4ae8d-6d40-466c-a97c-daab363b3502" containerID="9ff9fa30b85b2c26ec6275d72344ddc13bc2cbb91fb51645f2cde43805d7b978" exitCode=0
Aug 04 17:54:02.998715 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:54:02.998350    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-n64q5" event={"ID":"f3c4ae8d-6d40-466c-a97c-daab363b3502","Type":"ContainerDied","Data":"9ff9fa30b85b2c26ec6275d72344ddc13bc2cbb91fb51645f2cde43805d7b978"}
Aug 04 17:54:02.998715 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:54:02.998677    2560 scope.go:117] "RemoveContainer" containerID="9ff9fa30b85b2c26ec6275d72344ddc13bc2cbb91fb51645f2cde43805d7b978"
Aug 04 17:54:04.005210 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:54:04.005164    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-n64q5" event={"ID":"f3c4ae8d-6d40-466c-a97c-daab363b3502","Type":"ContainerStarted","Data":"19178b8c9fd3850c93a664e58d6185e8bd92853b4f99c6f2348518d347360d64"}
Aug 04 17:54:19.072057 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:54:19.072021    2560 generic.go:356] "Generic (PLEG): container finished" podID="3f8891ae-cf02-49a7-9144-ef455be4cdee" containerID="6430cb26b5d8f22c4d722b6c0af5b7517d5dfab636107a31eb82904e90f73824" exitCode=0
Aug 04 17:54:19.072467 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:54:19.072098    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-47zwf" event={"ID":"3f8891ae-cf02-49a7-9144-ef455be4cdee","Type":"ContainerDied","Data":"6430cb26b5d8f22c4d722b6c0af5b7517d5dfab636107a31eb82904e90f73824"}
Aug 04 17:54:19.072467 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:54:19.072442    2560 scope.go:117] "RemoveContainer" containerID="6430cb26b5d8f22c4d722b6c0af5b7517d5dfab636107a31eb82904e90f73824"
Aug 04 17:54:20.077644 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:54:20.077610    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-47zwf" event={"ID":"3f8891ae-cf02-49a7-9144-ef455be4cdee","Type":"ContainerStarted","Data":"5c48de79aaaa126ae3c94428a814738f0aa2f96b239c34ccf1a69d56c6615a56"}
Aug 04 17:56:58.249869 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:56:58.249843    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/2.log"
Aug 04 17:56:58.255262 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:56:58.255238    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/2.log"
Aug 04 17:56:58.255401 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:56:58.255362    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 17:56:58.259609 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:56:58.259592    2560 kubelet.go:1648] "Image garbage collection succeeded"
Aug 04 17:56:58.261680 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:56:58.261661    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 17:58:01.623917 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:01.623844    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5fd6db6bcb-98q8z"]
Aug 04 17:58:01.624318 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:01.624192    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="887c097d-64f3-4f3f-99ac-7981ab483724" containerName="registry"
Aug 04 17:58:01.624318 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:01.624207    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="887c097d-64f3-4f3f-99ac-7981ab483724" containerName="registry"
Aug 04 17:58:01.624318 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:01.624261    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="887c097d-64f3-4f3f-99ac-7981ab483724" containerName="registry"
Aug 04 17:58:01.627272 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:01.627257    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5fd6db6bcb-98q8z"
Aug 04 17:58:01.634667 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:01.634645    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Aug 04 17:58:01.634832 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:01.634807    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Aug 04 17:58:01.635889 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:01.635871    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-w7vjj\""
Aug 04 17:58:01.636297 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:01.636282    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Aug 04 17:58:01.647067 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:01.647049    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5fd6db6bcb-98q8z"]
Aug 04 17:58:01.648658 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:01.648642    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Aug 04 17:58:01.784484 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:01.784459    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/521e0e75-e94e-4541-a553-6ef6b6330eb4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5fd6db6bcb-98q8z\" (UID: \"521e0e75-e94e-4541-a553-6ef6b6330eb4\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd6db6bcb-98q8z"
Aug 04 17:58:01.784599 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:01.784494    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/521e0e75-e94e-4541-a553-6ef6b6330eb4-manifests\") pod \"opendatahub-operator-controller-manager-5fd6db6bcb-98q8z\" (UID: \"521e0e75-e94e-4541-a553-6ef6b6330eb4\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd6db6bcb-98q8z"
Aug 04 17:58:01.784599 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:01.784532    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klh5j\" (UniqueName: \"kubernetes.io/projected/521e0e75-e94e-4541-a553-6ef6b6330eb4-kube-api-access-klh5j\") pod \"opendatahub-operator-controller-manager-5fd6db6bcb-98q8z\" (UID: \"521e0e75-e94e-4541-a553-6ef6b6330eb4\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd6db6bcb-98q8z"
Aug 04 17:58:01.784599 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:01.784552    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/521e0e75-e94e-4541-a553-6ef6b6330eb4-tmp\") pod \"opendatahub-operator-controller-manager-5fd6db6bcb-98q8z\" (UID: \"521e0e75-e94e-4541-a553-6ef6b6330eb4\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd6db6bcb-98q8z"
Aug 04 17:58:01.784599 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:01.784568    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/521e0e75-e94e-4541-a553-6ef6b6330eb4-webhook-cert\") pod \"opendatahub-operator-controller-manager-5fd6db6bcb-98q8z\" (UID: \"521e0e75-e94e-4541-a553-6ef6b6330eb4\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd6db6bcb-98q8z"
Aug 04 17:58:01.885826 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:01.885765    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klh5j\" (UniqueName: \"kubernetes.io/projected/521e0e75-e94e-4541-a553-6ef6b6330eb4-kube-api-access-klh5j\") pod \"opendatahub-operator-controller-manager-5fd6db6bcb-98q8z\" (UID: \"521e0e75-e94e-4541-a553-6ef6b6330eb4\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd6db6bcb-98q8z"
Aug 04 17:58:01.885826 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:01.885799    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/521e0e75-e94e-4541-a553-6ef6b6330eb4-tmp\") pod \"opendatahub-operator-controller-manager-5fd6db6bcb-98q8z\" (UID: \"521e0e75-e94e-4541-a553-6ef6b6330eb4\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd6db6bcb-98q8z"
Aug 04 17:58:01.885826 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:01.885819    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/521e0e75-e94e-4541-a553-6ef6b6330eb4-webhook-cert\") pod \"opendatahub-operator-controller-manager-5fd6db6bcb-98q8z\" (UID: \"521e0e75-e94e-4541-a553-6ef6b6330eb4\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd6db6bcb-98q8z"
Aug 04 17:58:01.886083 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:01.885843    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/521e0e75-e94e-4541-a553-6ef6b6330eb4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5fd6db6bcb-98q8z\" (UID: \"521e0e75-e94e-4541-a553-6ef6b6330eb4\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd6db6bcb-98q8z"
Aug 04 17:58:01.886083 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:01.885970    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/521e0e75-e94e-4541-a553-6ef6b6330eb4-manifests\") pod \"opendatahub-operator-controller-manager-5fd6db6bcb-98q8z\" (UID: \"521e0e75-e94e-4541-a553-6ef6b6330eb4\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd6db6bcb-98q8z"
Aug 04 17:58:01.886316 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:01.886290    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/521e0e75-e94e-4541-a553-6ef6b6330eb4-manifests\") pod \"opendatahub-operator-controller-manager-5fd6db6bcb-98q8z\" (UID: \"521e0e75-e94e-4541-a553-6ef6b6330eb4\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd6db6bcb-98q8z"
Aug 04 17:58:01.888203 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:01.888184    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/521e0e75-e94e-4541-a553-6ef6b6330eb4-tmp\") pod \"opendatahub-operator-controller-manager-5fd6db6bcb-98q8z\" (UID: \"521e0e75-e94e-4541-a553-6ef6b6330eb4\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd6db6bcb-98q8z"
Aug 04 17:58:01.888415 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:01.888398    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/521e0e75-e94e-4541-a553-6ef6b6330eb4-webhook-cert\") pod \"opendatahub-operator-controller-manager-5fd6db6bcb-98q8z\" (UID: \"521e0e75-e94e-4541-a553-6ef6b6330eb4\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd6db6bcb-98q8z"
Aug 04 17:58:01.888478 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:01.888431    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/521e0e75-e94e-4541-a553-6ef6b6330eb4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5fd6db6bcb-98q8z\" (UID: \"521e0e75-e94e-4541-a553-6ef6b6330eb4\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd6db6bcb-98q8z"
Aug 04 17:58:01.897700 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:01.897680    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klh5j\" (UniqueName: \"kubernetes.io/projected/521e0e75-e94e-4541-a553-6ef6b6330eb4-kube-api-access-klh5j\") pod \"opendatahub-operator-controller-manager-5fd6db6bcb-98q8z\" (UID: \"521e0e75-e94e-4541-a553-6ef6b6330eb4\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd6db6bcb-98q8z"
Aug 04 17:58:01.937804 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:01.937775    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5fd6db6bcb-98q8z"
Aug 04 17:58:02.078571 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:02.078541    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5fd6db6bcb-98q8z"]
Aug 04 17:58:02.081060 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:58:02.081029    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod521e0e75_e94e_4541_a553_6ef6b6330eb4.slice/crio-b426099bb4fc3426c4a7d0cc9e9a72699ce04c7c3e5ccad2d8cc232bc9954159 WatchSource:0}: Error finding container b426099bb4fc3426c4a7d0cc9e9a72699ce04c7c3e5ccad2d8cc232bc9954159: Status 404 returned error can't find the container with id b426099bb4fc3426c4a7d0cc9e9a72699ce04c7c3e5ccad2d8cc232bc9954159
Aug 04 17:58:02.082850 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:02.082835    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 17:58:02.914427 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:02.914349    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fd6db6bcb-98q8z" event={"ID":"521e0e75-e94e-4541-a553-6ef6b6330eb4","Type":"ContainerStarted","Data":"b426099bb4fc3426c4a7d0cc9e9a72699ce04c7c3e5ccad2d8cc232bc9954159"}
Aug 04 17:58:05.927574 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:05.927532    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fd6db6bcb-98q8z" event={"ID":"521e0e75-e94e-4541-a553-6ef6b6330eb4","Type":"ContainerStarted","Data":"6d5b9837c610881d0038006c1659583d6d518e84afdac8af65851c9fd059d5bc"}
Aug 04 17:58:06.933085 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:06.933049    2560 generic.go:356] "Generic (PLEG): container finished" podID="521e0e75-e94e-4541-a553-6ef6b6330eb4" containerID="6d5b9837c610881d0038006c1659583d6d518e84afdac8af65851c9fd059d5bc" exitCode=0
Aug 04 17:58:06.933475 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:06.933137    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fd6db6bcb-98q8z" event={"ID":"521e0e75-e94e-4541-a553-6ef6b6330eb4","Type":"ContainerDied","Data":"6d5b9837c610881d0038006c1659583d6d518e84afdac8af65851c9fd059d5bc"}
Aug 04 17:58:07.939809 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:07.939777    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fd6db6bcb-98q8z" event={"ID":"521e0e75-e94e-4541-a553-6ef6b6330eb4","Type":"ContainerStarted","Data":"0c89b773016082bcac7117530d1b59768a011ec81c2294854863e108e877bcb1"}
Aug 04 17:58:07.940288 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:07.939891    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5fd6db6bcb-98q8z"
Aug 04 17:58:07.993039 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:07.992972    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5fd6db6bcb-98q8z" podStartSLOduration=2.300021319 podStartE2EDuration="6.992955343s" totalImagesPullingTime="3.619731268s" totalInitContainerRuntime="1.073202756s" isStatefulPod=false podCreationTimestamp="2026-08-04 17:58:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:58:07.990833094 +0000 UTC m=+370.173993276" watchObservedRunningTime="2026-08-04 17:58:07.992955343 +0000 UTC m=+370.176115518"
Aug 04 17:58:18.946537 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:18.946502    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5fd6db6bcb-98q8z"
Aug 04 17:58:31.080712 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:31.080680    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-55bf5c9f58-7gdxs"]
Aug 04 17:58:31.083900 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:31.083882    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-55bf5c9f58-7gdxs"
Aug 04 17:58:31.098208 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:31.098185    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Aug 04 17:58:31.099081 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:31.099054    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Aug 04 17:58:31.099212 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:31.099128    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Aug 04 17:58:31.099212 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:31.099139    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Aug 04 17:58:31.099212 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:31.099140    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Aug 04 17:58:31.099212 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:31.099153    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-cbt2s\""
Aug 04 17:58:31.105172 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:31.105155    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b748fb50-d006-4dc4-900d-bbebd806fc17-manager-config\") pod \"lws-controller-manager-55bf5c9f58-7gdxs\" (UID: \"b748fb50-d006-4dc4-900d-bbebd806fc17\") " pod="openshift-lws-operator/lws-controller-manager-55bf5c9f58-7gdxs"
Aug 04 17:58:31.105253 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:31.105186    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b748fb50-d006-4dc4-900d-bbebd806fc17-metrics-cert\") pod \"lws-controller-manager-55bf5c9f58-7gdxs\" (UID: \"b748fb50-d006-4dc4-900d-bbebd806fc17\") " pod="openshift-lws-operator/lws-controller-manager-55bf5c9f58-7gdxs"
Aug 04 17:58:31.105253 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:31.105210    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b748fb50-d006-4dc4-900d-bbebd806fc17-cert\") pod \"lws-controller-manager-55bf5c9f58-7gdxs\" (UID: \"b748fb50-d006-4dc4-900d-bbebd806fc17\") " pod="openshift-lws-operator/lws-controller-manager-55bf5c9f58-7gdxs"
Aug 04 17:58:31.105360 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:31.105292    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s228r\" (UniqueName: \"kubernetes.io/projected/b748fb50-d006-4dc4-900d-bbebd806fc17-kube-api-access-s228r\") pod \"lws-controller-manager-55bf5c9f58-7gdxs\" (UID: \"b748fb50-d006-4dc4-900d-bbebd806fc17\") " pod="openshift-lws-operator/lws-controller-manager-55bf5c9f58-7gdxs"
Aug 04 17:58:31.121470 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:31.121437    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-55bf5c9f58-7gdxs"]
Aug 04 17:58:31.206002 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:31.205962    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b748fb50-d006-4dc4-900d-bbebd806fc17-metrics-cert\") pod \"lws-controller-manager-55bf5c9f58-7gdxs\" (UID: \"b748fb50-d006-4dc4-900d-bbebd806fc17\") " pod="openshift-lws-operator/lws-controller-manager-55bf5c9f58-7gdxs"
Aug 04 17:58:31.206103 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:31.206016    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b748fb50-d006-4dc4-900d-bbebd806fc17-cert\") pod \"lws-controller-manager-55bf5c9f58-7gdxs\" (UID: \"b748fb50-d006-4dc4-900d-bbebd806fc17\") " pod="openshift-lws-operator/lws-controller-manager-55bf5c9f58-7gdxs"
Aug 04 17:58:31.206103 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:31.206044    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s228r\" (UniqueName: \"kubernetes.io/projected/b748fb50-d006-4dc4-900d-bbebd806fc17-kube-api-access-s228r\") pod \"lws-controller-manager-55bf5c9f58-7gdxs\" (UID: \"b748fb50-d006-4dc4-900d-bbebd806fc17\") " pod="openshift-lws-operator/lws-controller-manager-55bf5c9f58-7gdxs"
Aug 04 17:58:31.206103 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:31.206083    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b748fb50-d006-4dc4-900d-bbebd806fc17-manager-config\") pod \"lws-controller-manager-55bf5c9f58-7gdxs\" (UID: \"b748fb50-d006-4dc4-900d-bbebd806fc17\") " pod="openshift-lws-operator/lws-controller-manager-55bf5c9f58-7gdxs"
Aug 04 17:58:31.206674 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:31.206647    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b748fb50-d006-4dc4-900d-bbebd806fc17-manager-config\") pod \"lws-controller-manager-55bf5c9f58-7gdxs\" (UID: \"b748fb50-d006-4dc4-900d-bbebd806fc17\") " pod="openshift-lws-operator/lws-controller-manager-55bf5c9f58-7gdxs"
Aug 04 17:58:31.208502 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:31.208482    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b748fb50-d006-4dc4-900d-bbebd806fc17-metrics-cert\") pod \"lws-controller-manager-55bf5c9f58-7gdxs\" (UID: \"b748fb50-d006-4dc4-900d-bbebd806fc17\") " pod="openshift-lws-operator/lws-controller-manager-55bf5c9f58-7gdxs"
Aug 04 17:58:31.208599 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:31.208524    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b748fb50-d006-4dc4-900d-bbebd806fc17-cert\") pod \"lws-controller-manager-55bf5c9f58-7gdxs\" (UID: \"b748fb50-d006-4dc4-900d-bbebd806fc17\") " pod="openshift-lws-operator/lws-controller-manager-55bf5c9f58-7gdxs"
Aug 04 17:58:31.238326 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:31.238297    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s228r\" (UniqueName: \"kubernetes.io/projected/b748fb50-d006-4dc4-900d-bbebd806fc17-kube-api-access-s228r\") pod \"lws-controller-manager-55bf5c9f58-7gdxs\" (UID: \"b748fb50-d006-4dc4-900d-bbebd806fc17\") " pod="openshift-lws-operator/lws-controller-manager-55bf5c9f58-7gdxs"
Aug 04 17:58:31.392831 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:31.392748    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-55bf5c9f58-7gdxs"
Aug 04 17:58:31.542095 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:31.542068    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-55bf5c9f58-7gdxs"]
Aug 04 17:58:31.545139 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:58:31.545107    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb748fb50_d006_4dc4_900d_bbebd806fc17.slice/crio-6b1c68fb82c4a878c56a2e48d7866ea5b5dda3fb9598eda0e4a9e829ca6b77f7 WatchSource:0}: Error finding container 6b1c68fb82c4a878c56a2e48d7866ea5b5dda3fb9598eda0e4a9e829ca6b77f7: Status 404 returned error can't find the container with id 6b1c68fb82c4a878c56a2e48d7866ea5b5dda3fb9598eda0e4a9e829ca6b77f7
Aug 04 17:58:32.041630 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:32.041598    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-55bf5c9f58-7gdxs" event={"ID":"b748fb50-d006-4dc4-900d-bbebd806fc17","Type":"ContainerStarted","Data":"6b1c68fb82c4a878c56a2e48d7866ea5b5dda3fb9598eda0e4a9e829ca6b77f7"}
Aug 04 17:58:35.054791 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:35.054744    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-55bf5c9f58-7gdxs" event={"ID":"b748fb50-d006-4dc4-900d-bbebd806fc17","Type":"ContainerStarted","Data":"9d1e7bc73abfe341ea34398d4b6bc47b051bee9b2f4dfa739c7178480bd88a64"}
Aug 04 17:58:35.055238 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:35.054868    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-55bf5c9f58-7gdxs"
Aug 04 17:58:36.716454 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:36.716406    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-55bf5c9f58-7gdxs" podStartSLOduration=3.194466562 podStartE2EDuration="5.716388028s" totalImagesPullingTime="2.521921466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 17:58:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:58:35.098943147 +0000 UTC m=+397.282103322" watchObservedRunningTime="2026-08-04 17:58:36.716388028 +0000 UTC m=+398.899548203"
Aug 04 17:58:36.717580 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:36.717563    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-srjp7"]
Aug 04 17:58:36.720965 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:36.720949    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-srjp7"
Aug 04 17:58:36.736806 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:36.736787    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Aug 04 17:58:36.741037 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:36.741016    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-srjp7"]
Aug 04 17:58:36.744095 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:36.744064    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-slslx\""
Aug 04 17:58:36.752314 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:36.752293    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58d7dbcb-b524-4c25-8112-ac3cb702e76c-cert\") pod \"kserve-controller-manager-d6f4b65b8-srjp7\" (UID: \"58d7dbcb-b524-4c25-8112-ac3cb702e76c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-srjp7"
Aug 04 17:58:36.752419 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:36.752359    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29f4c\" (UniqueName: \"kubernetes.io/projected/58d7dbcb-b524-4c25-8112-ac3cb702e76c-kube-api-access-29f4c\") pod \"kserve-controller-manager-d6f4b65b8-srjp7\" (UID: \"58d7dbcb-b524-4c25-8112-ac3cb702e76c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-srjp7"
Aug 04 17:58:36.853679 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:36.853654    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58d7dbcb-b524-4c25-8112-ac3cb702e76c-cert\") pod \"kserve-controller-manager-d6f4b65b8-srjp7\" (UID: \"58d7dbcb-b524-4c25-8112-ac3cb702e76c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-srjp7"
Aug 04 17:58:36.853788 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:36.853711    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29f4c\" (UniqueName: \"kubernetes.io/projected/58d7dbcb-b524-4c25-8112-ac3cb702e76c-kube-api-access-29f4c\") pod \"kserve-controller-manager-d6f4b65b8-srjp7\" (UID: \"58d7dbcb-b524-4c25-8112-ac3cb702e76c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-srjp7"
Aug 04 17:58:36.856229 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:36.856210    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58d7dbcb-b524-4c25-8112-ac3cb702e76c-cert\") pod \"kserve-controller-manager-d6f4b65b8-srjp7\" (UID: \"58d7dbcb-b524-4c25-8112-ac3cb702e76c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-srjp7"
Aug 04 17:58:36.934602 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:36.934581    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29f4c\" (UniqueName: \"kubernetes.io/projected/58d7dbcb-b524-4c25-8112-ac3cb702e76c-kube-api-access-29f4c\") pod \"kserve-controller-manager-d6f4b65b8-srjp7\" (UID: \"58d7dbcb-b524-4c25-8112-ac3cb702e76c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-srjp7"
Aug 04 17:58:37.030861 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:37.030769    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-srjp7"
Aug 04 17:58:37.176077 ip-10-0-128-172 kubenswrapper[2560]: W0804 17:58:37.176052    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58d7dbcb_b524_4c25_8112_ac3cb702e76c.slice/crio-9a2fc196f8e5dbd1926a82eb69795202e49232817b91306fc2514e6b9f3c1a7d WatchSource:0}: Error finding container 9a2fc196f8e5dbd1926a82eb69795202e49232817b91306fc2514e6b9f3c1a7d: Status 404 returned error can't find the container with id 9a2fc196f8e5dbd1926a82eb69795202e49232817b91306fc2514e6b9f3c1a7d
Aug 04 17:58:37.177218 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:37.177194    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-srjp7"]
Aug 04 17:58:38.070063 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:38.070010    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-srjp7" event={"ID":"58d7dbcb-b524-4c25-8112-ac3cb702e76c","Type":"ContainerStarted","Data":"9a2fc196f8e5dbd1926a82eb69795202e49232817b91306fc2514e6b9f3c1a7d"}
Aug 04 17:58:40.080560 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:40.080530    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-srjp7" event={"ID":"58d7dbcb-b524-4c25-8112-ac3cb702e76c","Type":"ContainerStarted","Data":"b23b8453b8071bb25f525b3dc1f0f65f0359d9eaf51e12982fb1ee7e6ec78b9f"}
Aug 04 17:58:40.080943 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:40.080665    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-srjp7"
Aug 04 17:58:40.102345 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:40.102303    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-srjp7" podStartSLOduration=1.37001572 podStartE2EDuration="4.102288571s" totalImagesPullingTime="2.732272851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 17:58:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 17:58:40.100764408 +0000 UTC m=+402.283924585" watchObservedRunningTime="2026-08-04 17:58:40.102288571 +0000 UTC m=+402.285448747"
Aug 04 17:58:46.061741 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:58:46.061709    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-55bf5c9f58-7gdxs"
Aug 04 17:59:11.090373 ip-10-0-128-172 kubenswrapper[2560]: I0804 17:59:11.090344    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-srjp7"
Aug 04 18:00:33.128626 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:33.128592    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-k2g6l"]
Aug 04 18:00:33.131800 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:33.131785    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-k2g6l"
Aug 04 18:00:33.134556 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:33.134524    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Aug 04 18:00:33.135061 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:33.135041    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Aug 04 18:00:33.135156 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:33.135070    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-g4hn2\""
Aug 04 18:00:33.135156 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:33.135050    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Aug 04 18:00:33.143543 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:33.143519    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-k2g6l"]
Aug 04 18:00:33.202120 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:33.202087    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1c7dad3c-4b98-4cd8-af74-d49bf73ed92a-config-file\") pod \"limitador-limitador-859c4d96d6-k2g6l\" (UID: \"1c7dad3c-4b98-4cd8-af74-d49bf73ed92a\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-k2g6l"
Aug 04 18:00:33.202277 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:33.202123    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnzmw\" (UniqueName: \"kubernetes.io/projected/1c7dad3c-4b98-4cd8-af74-d49bf73ed92a-kube-api-access-mnzmw\") pod \"limitador-limitador-859c4d96d6-k2g6l\" (UID: \"1c7dad3c-4b98-4cd8-af74-d49bf73ed92a\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-k2g6l"
Aug 04 18:00:33.219043 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:33.219016    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-k2g6l"]
Aug 04 18:00:33.303511 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:33.303484    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1c7dad3c-4b98-4cd8-af74-d49bf73ed92a-config-file\") pod \"limitador-limitador-859c4d96d6-k2g6l\" (UID: \"1c7dad3c-4b98-4cd8-af74-d49bf73ed92a\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-k2g6l"
Aug 04 18:00:33.303649 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:33.303518    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnzmw\" (UniqueName: \"kubernetes.io/projected/1c7dad3c-4b98-4cd8-af74-d49bf73ed92a-kube-api-access-mnzmw\") pod \"limitador-limitador-859c4d96d6-k2g6l\" (UID: \"1c7dad3c-4b98-4cd8-af74-d49bf73ed92a\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-k2g6l"
Aug 04 18:00:33.304135 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:33.304114    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1c7dad3c-4b98-4cd8-af74-d49bf73ed92a-config-file\") pod \"limitador-limitador-859c4d96d6-k2g6l\" (UID: \"1c7dad3c-4b98-4cd8-af74-d49bf73ed92a\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-k2g6l"
Aug 04 18:00:33.313240 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:33.313210    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnzmw\" (UniqueName: \"kubernetes.io/projected/1c7dad3c-4b98-4cd8-af74-d49bf73ed92a-kube-api-access-mnzmw\") pod \"limitador-limitador-859c4d96d6-k2g6l\" (UID: \"1c7dad3c-4b98-4cd8-af74-d49bf73ed92a\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-k2g6l"
Aug 04 18:00:33.447152 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:33.447131    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-k2g6l"
Aug 04 18:00:33.583197 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:33.583171    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-k2g6l"]
Aug 04 18:00:33.586710 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:00:33.586682    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c7dad3c_4b98_4cd8_af74_d49bf73ed92a.slice/crio-ad08b2dc58d11768e6e83d1c28523309f987320bb4f467fd89e1a9af780a637e WatchSource:0}: Error finding container ad08b2dc58d11768e6e83d1c28523309f987320bb4f467fd89e1a9af780a637e: Status 404 returned error can't find the container with id ad08b2dc58d11768e6e83d1c28523309f987320bb4f467fd89e1a9af780a637e
Aug 04 18:00:33.972600 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:33.972569    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b54df94b4-ntnr6"]
Aug 04 18:00:33.976925 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:33.976905    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-ntnr6"
Aug 04 18:00:33.979658 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:33.979639    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-bph79\""
Aug 04 18:00:33.986524 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:33.986501    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-ntnr6"]
Aug 04 18:00:34.009954 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:34.009934    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mkwr\" (UniqueName: \"kubernetes.io/projected/c32758f2-2d77-4d40-9cbf-e502d2811b88-kube-api-access-7mkwr\") pod \"authorino-b54df94b4-ntnr6\" (UID: \"c32758f2-2d77-4d40-9cbf-e502d2811b88\") " pod="kuadrant-system/authorino-b54df94b4-ntnr6"
Aug 04 18:00:34.111284 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:34.111254    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mkwr\" (UniqueName: \"kubernetes.io/projected/c32758f2-2d77-4d40-9cbf-e502d2811b88-kube-api-access-7mkwr\") pod \"authorino-b54df94b4-ntnr6\" (UID: \"c32758f2-2d77-4d40-9cbf-e502d2811b88\") " pod="kuadrant-system/authorino-b54df94b4-ntnr6"
Aug 04 18:00:34.125323 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:34.125300    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mkwr\" (UniqueName: \"kubernetes.io/projected/c32758f2-2d77-4d40-9cbf-e502d2811b88-kube-api-access-7mkwr\") pod \"authorino-b54df94b4-ntnr6\" (UID: \"c32758f2-2d77-4d40-9cbf-e502d2811b88\") " pod="kuadrant-system/authorino-b54df94b4-ntnr6"
Aug 04 18:00:34.130177 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:34.130158    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-gz7lp"]
Aug 04 18:00:34.133294 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:34.133277    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-gz7lp"
Aug 04 18:00:34.145888 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:34.145869    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-gz7lp"]
Aug 04 18:00:34.211917 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:34.211891    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plplx\" (UniqueName: \"kubernetes.io/projected/285e371e-1230-4e16-8c57-d9e93980ac5c-kube-api-access-plplx\") pod \"authorino-c548b5d7d-gz7lp\" (UID: \"285e371e-1230-4e16-8c57-d9e93980ac5c\") " pod="kuadrant-system/authorino-c548b5d7d-gz7lp"
Aug 04 18:00:34.287093 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:34.287041    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-ntnr6"
Aug 04 18:00:34.313242 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:34.313214    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plplx\" (UniqueName: \"kubernetes.io/projected/285e371e-1230-4e16-8c57-d9e93980ac5c-kube-api-access-plplx\") pod \"authorino-c548b5d7d-gz7lp\" (UID: \"285e371e-1230-4e16-8c57-d9e93980ac5c\") " pod="kuadrant-system/authorino-c548b5d7d-gz7lp"
Aug 04 18:00:34.324936 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:34.324903    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plplx\" (UniqueName: \"kubernetes.io/projected/285e371e-1230-4e16-8c57-d9e93980ac5c-kube-api-access-plplx\") pod \"authorino-c548b5d7d-gz7lp\" (UID: \"285e371e-1230-4e16-8c57-d9e93980ac5c\") " pod="kuadrant-system/authorino-c548b5d7d-gz7lp"
Aug 04 18:00:34.421109 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:34.421086    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-ntnr6"]
Aug 04 18:00:34.423011 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:00:34.422961    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc32758f2_2d77_4d40_9cbf_e502d2811b88.slice/crio-1fa8e944ef07a732cf5e4d243505a01d4a2da5c4d50e1a69572235b5528f7db4 WatchSource:0}: Error finding container 1fa8e944ef07a732cf5e4d243505a01d4a2da5c4d50e1a69572235b5528f7db4: Status 404 returned error can't find the container with id 1fa8e944ef07a732cf5e4d243505a01d4a2da5c4d50e1a69572235b5528f7db4
Aug 04 18:00:34.442158 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:34.442130    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-gz7lp"
Aug 04 18:00:34.559459 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:34.559380    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-k2g6l" event={"ID":"1c7dad3c-4b98-4cd8-af74-d49bf73ed92a","Type":"ContainerStarted","Data":"ad08b2dc58d11768e6e83d1c28523309f987320bb4f467fd89e1a9af780a637e"}
Aug 04 18:00:34.561034 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:34.561003    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-ntnr6" event={"ID":"c32758f2-2d77-4d40-9cbf-e502d2811b88","Type":"ContainerStarted","Data":"1fa8e944ef07a732cf5e4d243505a01d4a2da5c4d50e1a69572235b5528f7db4"}
Aug 04 18:00:34.593688 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:34.593662    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-gz7lp"]
Aug 04 18:00:34.595826 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:00:34.595796    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod285e371e_1230_4e16_8c57_d9e93980ac5c.slice/crio-3796107d55100197c217feb07db903f51fff62720a82bb1bbe62fdcca8f91078 WatchSource:0}: Error finding container 3796107d55100197c217feb07db903f51fff62720a82bb1bbe62fdcca8f91078: Status 404 returned error can't find the container with id 3796107d55100197c217feb07db903f51fff62720a82bb1bbe62fdcca8f91078
Aug 04 18:00:35.574691 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:35.574619    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-gz7lp" event={"ID":"285e371e-1230-4e16-8c57-d9e93980ac5c","Type":"ContainerStarted","Data":"3796107d55100197c217feb07db903f51fff62720a82bb1bbe62fdcca8f91078"}
Aug 04 18:00:37.586636 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:37.586597    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-k2g6l" event={"ID":"1c7dad3c-4b98-4cd8-af74-d49bf73ed92a","Type":"ContainerStarted","Data":"de59d5f027643502215426e007dc5241772a7efe37754719e0f081fd6508bb14"}
Aug 04 18:00:37.587109 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:37.586657    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-k2g6l"
Aug 04 18:00:37.588189 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:37.588165    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-gz7lp" event={"ID":"285e371e-1230-4e16-8c57-d9e93980ac5c","Type":"ContainerStarted","Data":"44331051d714f7480d8719ae2d293ebe411dab0ad4348ca701a90e1cab4d3a3e"}
Aug 04 18:00:37.589539 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:37.589514    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-ntnr6" event={"ID":"c32758f2-2d77-4d40-9cbf-e502d2811b88","Type":"ContainerStarted","Data":"5a23a51ff209e04b550e00d3dcdc602b5177733b8c9f72820b3cab8c9a433f3f"}
Aug 04 18:00:37.607262 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:37.607220    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-k2g6l" podStartSLOduration=1.483670343 podStartE2EDuration="4.607206163s" totalImagesPullingTime="3.12353582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:00:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:00:37.60526285 +0000 UTC m=+519.788423036" watchObservedRunningTime="2026-08-04 18:00:37.607206163 +0000 UTC m=+519.790366339"
Aug 04 18:00:37.624328 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:37.624291    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-gz7lp" podStartSLOduration=1.502532284 podStartE2EDuration="3.624279276s" totalImagesPullingTime="2.121746992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:00:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:00:37.621306338 +0000 UTC m=+519.804466515" watchObservedRunningTime="2026-08-04 18:00:37.624279276 +0000 UTC m=+519.807439452"
Aug 04 18:00:37.644307 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:37.644267    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b54df94b4-ntnr6" podStartSLOduration=2.355954706 podStartE2EDuration="4.644253698s" totalImagesPullingTime="2.288298992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:00:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:00:37.643149942 +0000 UTC m=+519.826310119" watchObservedRunningTime="2026-08-04 18:00:37.644253698 +0000 UTC m=+519.827413874"
Aug 04 18:00:37.676586 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:37.676553    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-ntnr6"]
Aug 04 18:00:39.597486 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:39.597450    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b54df94b4-ntnr6" podUID="c32758f2-2d77-4d40-9cbf-e502d2811b88" containerName="authorino" containerID="cri-o://5a23a51ff209e04b550e00d3dcdc602b5177733b8c9f72820b3cab8c9a433f3f" gracePeriod=30
Aug 04 18:00:39.832615 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:39.832588    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-ntnr6"
Aug 04 18:00:39.960810 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:39.960784    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mkwr\" (UniqueName: \"kubernetes.io/projected/c32758f2-2d77-4d40-9cbf-e502d2811b88-kube-api-access-7mkwr\") pod \"c32758f2-2d77-4d40-9cbf-e502d2811b88\" (UID: \"c32758f2-2d77-4d40-9cbf-e502d2811b88\") "
Aug 04 18:00:39.962982 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:39.962959    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c32758f2-2d77-4d40-9cbf-e502d2811b88-kube-api-access-7mkwr" (OuterVolumeSpecName: "kube-api-access-7mkwr") pod "c32758f2-2d77-4d40-9cbf-e502d2811b88" (UID: "c32758f2-2d77-4d40-9cbf-e502d2811b88"). InnerVolumeSpecName "kube-api-access-7mkwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:00:40.062049 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:40.062025    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mkwr\" (UniqueName: \"kubernetes.io/projected/c32758f2-2d77-4d40-9cbf-e502d2811b88-kube-api-access-7mkwr\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:00:40.602408 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:40.602375    2560 generic.go:356] "Generic (PLEG): container finished" podID="c32758f2-2d77-4d40-9cbf-e502d2811b88" containerID="5a23a51ff209e04b550e00d3dcdc602b5177733b8c9f72820b3cab8c9a433f3f" exitCode=0
Aug 04 18:00:40.602880 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:40.602425    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-ntnr6"
Aug 04 18:00:40.602880 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:40.602456    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-ntnr6" event={"ID":"c32758f2-2d77-4d40-9cbf-e502d2811b88","Type":"ContainerDied","Data":"5a23a51ff209e04b550e00d3dcdc602b5177733b8c9f72820b3cab8c9a433f3f"}
Aug 04 18:00:40.602880 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:40.602496    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-ntnr6" event={"ID":"c32758f2-2d77-4d40-9cbf-e502d2811b88","Type":"ContainerDied","Data":"1fa8e944ef07a732cf5e4d243505a01d4a2da5c4d50e1a69572235b5528f7db4"}
Aug 04 18:00:40.602880 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:40.602513    2560 scope.go:117] "RemoveContainer" containerID="5a23a51ff209e04b550e00d3dcdc602b5177733b8c9f72820b3cab8c9a433f3f"
Aug 04 18:00:40.611170 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:40.611146    2560 scope.go:117] "RemoveContainer" containerID="5a23a51ff209e04b550e00d3dcdc602b5177733b8c9f72820b3cab8c9a433f3f"
Aug 04 18:00:40.611428 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:00:40.611406    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a23a51ff209e04b550e00d3dcdc602b5177733b8c9f72820b3cab8c9a433f3f\": container with ID starting with 5a23a51ff209e04b550e00d3dcdc602b5177733b8c9f72820b3cab8c9a433f3f not found: ID does not exist" containerID="5a23a51ff209e04b550e00d3dcdc602b5177733b8c9f72820b3cab8c9a433f3f"
Aug 04 18:00:40.611489 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:40.611437    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a23a51ff209e04b550e00d3dcdc602b5177733b8c9f72820b3cab8c9a433f3f"} err="failed to get container status \"5a23a51ff209e04b550e00d3dcdc602b5177733b8c9f72820b3cab8c9a433f3f\": rpc error: code = NotFound desc = could not find container \"5a23a51ff209e04b550e00d3dcdc602b5177733b8c9f72820b3cab8c9a433f3f\": container with ID starting with 5a23a51ff209e04b550e00d3dcdc602b5177733b8c9f72820b3cab8c9a433f3f not found: ID does not exist"
Aug 04 18:00:40.627823 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:40.627789    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-ntnr6"]
Aug 04 18:00:40.631721 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:40.631701    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b54df94b4-ntnr6"]
Aug 04 18:00:42.352395 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:42.352358    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c32758f2-2d77-4d40-9cbf-e502d2811b88" path="/var/lib/kubelet/pods/c32758f2-2d77-4d40-9cbf-e502d2811b88/volumes"
Aug 04 18:00:47.047332 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:47.047294    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-k2g6l"]
Aug 04 18:00:47.047777 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:47.047482    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-k2g6l" podUID="1c7dad3c-4b98-4cd8-af74-d49bf73ed92a" containerName="limitador" containerID="cri-o://de59d5f027643502215426e007dc5241772a7efe37754719e0f081fd6508bb14" gracePeriod=30
Aug 04 18:00:47.048313 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:47.048292    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-k2g6l"
Aug 04 18:00:47.585889 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:47.585861    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-k2g6l"
Aug 04 18:00:47.633945 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:47.633878    2560 generic.go:356] "Generic (PLEG): container finished" podID="1c7dad3c-4b98-4cd8-af74-d49bf73ed92a" containerID="de59d5f027643502215426e007dc5241772a7efe37754719e0f081fd6508bb14" exitCode=0
Aug 04 18:00:47.633945 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:47.633917    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-k2g6l"
Aug 04 18:00:47.634122 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:47.633969    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-k2g6l" event={"ID":"1c7dad3c-4b98-4cd8-af74-d49bf73ed92a","Type":"ContainerDied","Data":"de59d5f027643502215426e007dc5241772a7efe37754719e0f081fd6508bb14"}
Aug 04 18:00:47.634122 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:47.634040    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-k2g6l" event={"ID":"1c7dad3c-4b98-4cd8-af74-d49bf73ed92a","Type":"ContainerDied","Data":"ad08b2dc58d11768e6e83d1c28523309f987320bb4f467fd89e1a9af780a637e"}
Aug 04 18:00:47.634122 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:47.634065    2560 scope.go:117] "RemoveContainer" containerID="de59d5f027643502215426e007dc5241772a7efe37754719e0f081fd6508bb14"
Aug 04 18:00:47.642184 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:47.642168    2560 scope.go:117] "RemoveContainer" containerID="de59d5f027643502215426e007dc5241772a7efe37754719e0f081fd6508bb14"
Aug 04 18:00:47.642438 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:00:47.642422    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de59d5f027643502215426e007dc5241772a7efe37754719e0f081fd6508bb14\": container with ID starting with de59d5f027643502215426e007dc5241772a7efe37754719e0f081fd6508bb14 not found: ID does not exist" containerID="de59d5f027643502215426e007dc5241772a7efe37754719e0f081fd6508bb14"
Aug 04 18:00:47.642488 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:47.642443    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de59d5f027643502215426e007dc5241772a7efe37754719e0f081fd6508bb14"} err="failed to get container status \"de59d5f027643502215426e007dc5241772a7efe37754719e0f081fd6508bb14\": rpc error: code = NotFound desc = could not find container \"de59d5f027643502215426e007dc5241772a7efe37754719e0f081fd6508bb14\": container with ID starting with de59d5f027643502215426e007dc5241772a7efe37754719e0f081fd6508bb14 not found: ID does not exist"
Aug 04 18:00:47.722251 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:47.722216    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1c7dad3c-4b98-4cd8-af74-d49bf73ed92a-config-file\") pod \"1c7dad3c-4b98-4cd8-af74-d49bf73ed92a\" (UID: \"1c7dad3c-4b98-4cd8-af74-d49bf73ed92a\") "
Aug 04 18:00:47.722354 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:47.722259    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mnzmw\" (UniqueName: \"kubernetes.io/projected/1c7dad3c-4b98-4cd8-af74-d49bf73ed92a-kube-api-access-mnzmw\") pod \"1c7dad3c-4b98-4cd8-af74-d49bf73ed92a\" (UID: \"1c7dad3c-4b98-4cd8-af74-d49bf73ed92a\") "
Aug 04 18:00:47.722519 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:47.722499    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c7dad3c-4b98-4cd8-af74-d49bf73ed92a-config-file" (OuterVolumeSpecName: "config-file") pod "1c7dad3c-4b98-4cd8-af74-d49bf73ed92a" (UID: "1c7dad3c-4b98-4cd8-af74-d49bf73ed92a"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:00:47.724286 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:47.724264    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c7dad3c-4b98-4cd8-af74-d49bf73ed92a-kube-api-access-mnzmw" (OuterVolumeSpecName: "kube-api-access-mnzmw") pod "1c7dad3c-4b98-4cd8-af74-d49bf73ed92a" (UID: "1c7dad3c-4b98-4cd8-af74-d49bf73ed92a"). InnerVolumeSpecName "kube-api-access-mnzmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:00:47.823411 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:47.823379    2560 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1c7dad3c-4b98-4cd8-af74-d49bf73ed92a-config-file\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:00:47.823411 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:47.823409    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnzmw\" (UniqueName: \"kubernetes.io/projected/1c7dad3c-4b98-4cd8-af74-d49bf73ed92a-kube-api-access-mnzmw\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:00:47.970253 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:47.970228    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-k2g6l"]
Aug 04 18:00:47.973126 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:47.973105    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-k2g6l"]
Aug 04 18:00:48.353444 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:48.353368    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c7dad3c-4b98-4cd8-af74-d49bf73ed92a" path="/var/lib/kubelet/pods/1c7dad3c-4b98-4cd8-af74-d49bf73ed92a/volumes"
Aug 04 18:00:49.761030 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:49.760917    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5q8m5"]
Aug 04 18:00:49.761453 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:49.761412    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c7dad3c-4b98-4cd8-af74-d49bf73ed92a" containerName="limitador"
Aug 04 18:00:49.761453 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:49.761430    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c7dad3c-4b98-4cd8-af74-d49bf73ed92a" containerName="limitador"
Aug 04 18:00:49.761453 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:49.761441    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c32758f2-2d77-4d40-9cbf-e502d2811b88" containerName="authorino"
Aug 04 18:00:49.761453 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:49.761450    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c32758f2-2d77-4d40-9cbf-e502d2811b88" containerName="authorino"
Aug 04 18:00:49.761640 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:49.761541    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c32758f2-2d77-4d40-9cbf-e502d2811b88" containerName="authorino"
Aug 04 18:00:49.761640 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:49.761554    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c7dad3c-4b98-4cd8-af74-d49bf73ed92a" containerName="limitador"
Aug 04 18:00:49.766106 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:49.766084    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5q8m5"
Aug 04 18:00:49.768812 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:49.768785    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 04 18:00:49.768919 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:49.768831    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Aug 04 18:00:49.769605 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:49.769588    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-wdzng\""
Aug 04 18:00:49.769681 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:49.769590    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Aug 04 18:00:49.774655 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:49.774335    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5q8m5"]
Aug 04 18:00:49.840296 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:49.840264    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/be680e73-4411-45f1-9849-3ad182348acf-data\") pod \"postgres-69fb4594cf-5q8m5\" (UID: \"be680e73-4411-45f1-9849-3ad182348acf\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5q8m5"
Aug 04 18:00:49.840404 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:49.840302    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76s9b\" (UniqueName: \"kubernetes.io/projected/be680e73-4411-45f1-9849-3ad182348acf-kube-api-access-76s9b\") pod \"postgres-69fb4594cf-5q8m5\" (UID: \"be680e73-4411-45f1-9849-3ad182348acf\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5q8m5"
Aug 04 18:00:49.941650 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:49.941618    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/be680e73-4411-45f1-9849-3ad182348acf-data\") pod \"postgres-69fb4594cf-5q8m5\" (UID: \"be680e73-4411-45f1-9849-3ad182348acf\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5q8m5"
Aug 04 18:00:49.941776 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:49.941664    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76s9b\" (UniqueName: \"kubernetes.io/projected/be680e73-4411-45f1-9849-3ad182348acf-kube-api-access-76s9b\") pod \"postgres-69fb4594cf-5q8m5\" (UID: \"be680e73-4411-45f1-9849-3ad182348acf\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5q8m5"
Aug 04 18:00:49.942013 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:49.941968    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/be680e73-4411-45f1-9849-3ad182348acf-data\") pod \"postgres-69fb4594cf-5q8m5\" (UID: \"be680e73-4411-45f1-9849-3ad182348acf\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5q8m5"
Aug 04 18:00:49.952440 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:49.952420    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76s9b\" (UniqueName: \"kubernetes.io/projected/be680e73-4411-45f1-9849-3ad182348acf-kube-api-access-76s9b\") pod \"postgres-69fb4594cf-5q8m5\" (UID: \"be680e73-4411-45f1-9849-3ad182348acf\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5q8m5"
Aug 04 18:00:50.078499 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:50.078443    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5q8m5"
Aug 04 18:00:50.213222 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:50.213201    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5q8m5"]
Aug 04 18:00:50.216070 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:00:50.216043    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe680e73_4411_45f1_9849_3ad182348acf.slice/crio-228f983eaf7614cac49cfeab2e29d0c3f1f51474ba29ff12f7275bb03b50bf25 WatchSource:0}: Error finding container 228f983eaf7614cac49cfeab2e29d0c3f1f51474ba29ff12f7275bb03b50bf25: Status 404 returned error can't find the container with id 228f983eaf7614cac49cfeab2e29d0c3f1f51474ba29ff12f7275bb03b50bf25
Aug 04 18:00:50.648776 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:50.648739    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5q8m5" event={"ID":"be680e73-4411-45f1-9849-3ad182348acf","Type":"ContainerStarted","Data":"228f983eaf7614cac49cfeab2e29d0c3f1f51474ba29ff12f7275bb03b50bf25"}
Aug 04 18:00:55.678511 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:55.678476    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5q8m5" event={"ID":"be680e73-4411-45f1-9849-3ad182348acf","Type":"ContainerStarted","Data":"09253701f55c3921fa6075bdbd1f071763eac8517b781a10679a41fb68a465a6"}
Aug 04 18:00:55.678967 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:55.678602    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5q8m5"
Aug 04 18:00:55.697872 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:00:55.697828    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5q8m5" podStartSLOduration=1.632824148 podStartE2EDuration="6.697815071s" totalImagesPullingTime="5.064990923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:00:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:00:55.69625606 +0000 UTC m=+537.879416250" watchObservedRunningTime="2026-08-04 18:00:55.697815071 +0000 UTC m=+537.880975247"
Aug 04 18:01:01.711726 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:01.711691    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5q8m5"
Aug 04 18:01:02.287463 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.287434    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-rhk5j"]
Aug 04 18:01:02.293588 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.293565    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-rhk5j"
Aug 04 18:01:02.299010 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.298965    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-rhk5j"]
Aug 04 18:01:02.427635 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.427604    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-rhk5j"]
Aug 04 18:01:02.427840 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:01:02.427819    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-96x57], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5dc7ccb5f-rhk5j" podUID="80ccd01b-e1f6-4a2c-896f-e848adc2a3fa"
Aug 04 18:01:02.444712 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.444684    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96x57\" (UniqueName: \"kubernetes.io/projected/80ccd01b-e1f6-4a2c-896f-e848adc2a3fa-kube-api-access-96x57\") pod \"authorino-5dc7ccb5f-rhk5j\" (UID: \"80ccd01b-e1f6-4a2c-896f-e848adc2a3fa\") " pod="kuadrant-system/authorino-5dc7ccb5f-rhk5j"
Aug 04 18:01:02.464562 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.464534    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-687bd45585-ggfzv"]
Aug 04 18:01:02.468092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.468074    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-ggfzv"
Aug 04 18:01:02.471010 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.470982    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Aug 04 18:01:02.481631 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.481608    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-687bd45585-ggfzv"]
Aug 04 18:01:02.545550 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.545478    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q5md\" (UniqueName: \"kubernetes.io/projected/d8838434-e37d-444f-a78e-a3291152778e-kube-api-access-4q5md\") pod \"authorino-687bd45585-ggfzv\" (UID: \"d8838434-e37d-444f-a78e-a3291152778e\") " pod="kuadrant-system/authorino-687bd45585-ggfzv"
Aug 04 18:01:02.545655 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.545557    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96x57\" (UniqueName: \"kubernetes.io/projected/80ccd01b-e1f6-4a2c-896f-e848adc2a3fa-kube-api-access-96x57\") pod \"authorino-5dc7ccb5f-rhk5j\" (UID: \"80ccd01b-e1f6-4a2c-896f-e848adc2a3fa\") " pod="kuadrant-system/authorino-5dc7ccb5f-rhk5j"
Aug 04 18:01:02.545655 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.545620    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d8838434-e37d-444f-a78e-a3291152778e-tls-cert\") pod \"authorino-687bd45585-ggfzv\" (UID: \"d8838434-e37d-444f-a78e-a3291152778e\") " pod="kuadrant-system/authorino-687bd45585-ggfzv"
Aug 04 18:01:02.547045 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.547024    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-687bd45585-ggfzv"]
Aug 04 18:01:02.547222 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:01:02.547205    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-4q5md tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-687bd45585-ggfzv" podUID="d8838434-e37d-444f-a78e-a3291152778e"
Aug 04 18:01:02.561013 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.560978    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96x57\" (UniqueName: \"kubernetes.io/projected/80ccd01b-e1f6-4a2c-896f-e848adc2a3fa-kube-api-access-96x57\") pod \"authorino-5dc7ccb5f-rhk5j\" (UID: \"80ccd01b-e1f6-4a2c-896f-e848adc2a3fa\") " pod="kuadrant-system/authorino-5dc7ccb5f-rhk5j"
Aug 04 18:01:02.580605 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.580582    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59b65697db-twb97"]
Aug 04 18:01:02.584225 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.584209    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59b65697db-twb97"
Aug 04 18:01:02.593032 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.593015    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59b65697db-twb97"]
Aug 04 18:01:02.646038 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.646012    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d8838434-e37d-444f-a78e-a3291152778e-tls-cert\") pod \"authorino-687bd45585-ggfzv\" (UID: \"d8838434-e37d-444f-a78e-a3291152778e\") " pod="kuadrant-system/authorino-687bd45585-ggfzv"
Aug 04 18:01:02.646140 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.646077    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q5md\" (UniqueName: \"kubernetes.io/projected/d8838434-e37d-444f-a78e-a3291152778e-kube-api-access-4q5md\") pod \"authorino-687bd45585-ggfzv\" (UID: \"d8838434-e37d-444f-a78e-a3291152778e\") " pod="kuadrant-system/authorino-687bd45585-ggfzv"
Aug 04 18:01:02.648561 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.648536    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d8838434-e37d-444f-a78e-a3291152778e-tls-cert\") pod \"authorino-687bd45585-ggfzv\" (UID: \"d8838434-e37d-444f-a78e-a3291152778e\") " pod="kuadrant-system/authorino-687bd45585-ggfzv"
Aug 04 18:01:02.656467 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.656445    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q5md\" (UniqueName: \"kubernetes.io/projected/d8838434-e37d-444f-a78e-a3291152778e-kube-api-access-4q5md\") pod \"authorino-687bd45585-ggfzv\" (UID: \"d8838434-e37d-444f-a78e-a3291152778e\") " pod="kuadrant-system/authorino-687bd45585-ggfzv"
Aug 04 18:01:02.710380 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.710355    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-ggfzv"
Aug 04 18:01:02.710470 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.710362    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-rhk5j"
Aug 04 18:01:02.717623 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.717605    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-ggfzv"
Aug 04 18:01:02.720497 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.720481    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-rhk5j"
Aug 04 18:01:02.746615 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.746581    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d91129c1-d186-48d5-b15c-088f2c747814-tls-cert\") pod \"authorino-59b65697db-twb97\" (UID: \"d91129c1-d186-48d5-b15c-088f2c747814\") " pod="kuadrant-system/authorino-59b65697db-twb97"
Aug 04 18:01:02.746711 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.746635    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8rj5\" (UniqueName: \"kubernetes.io/projected/d91129c1-d186-48d5-b15c-088f2c747814-kube-api-access-h8rj5\") pod \"authorino-59b65697db-twb97\" (UID: \"d91129c1-d186-48d5-b15c-088f2c747814\") " pod="kuadrant-system/authorino-59b65697db-twb97"
Aug 04 18:01:02.847926 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.847859    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4q5md\" (UniqueName: \"kubernetes.io/projected/d8838434-e37d-444f-a78e-a3291152778e-kube-api-access-4q5md\") pod \"d8838434-e37d-444f-a78e-a3291152778e\" (UID: \"d8838434-e37d-444f-a78e-a3291152778e\") "
Aug 04 18:01:02.847926 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.847894    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d8838434-e37d-444f-a78e-a3291152778e-tls-cert\") pod \"d8838434-e37d-444f-a78e-a3291152778e\" (UID: \"d8838434-e37d-444f-a78e-a3291152778e\") "
Aug 04 18:01:02.847926 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.847926    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96x57\" (UniqueName: \"kubernetes.io/projected/80ccd01b-e1f6-4a2c-896f-e848adc2a3fa-kube-api-access-96x57\") pod \"80ccd01b-e1f6-4a2c-896f-e848adc2a3fa\" (UID: \"80ccd01b-e1f6-4a2c-896f-e848adc2a3fa\") "
Aug 04 18:01:02.848174 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.848149    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d91129c1-d186-48d5-b15c-088f2c747814-tls-cert\") pod \"authorino-59b65697db-twb97\" (UID: \"d91129c1-d186-48d5-b15c-088f2c747814\") " pod="kuadrant-system/authorino-59b65697db-twb97"
Aug 04 18:01:02.848234 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.848178    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8rj5\" (UniqueName: \"kubernetes.io/projected/d91129c1-d186-48d5-b15c-088f2c747814-kube-api-access-h8rj5\") pod \"authorino-59b65697db-twb97\" (UID: \"d91129c1-d186-48d5-b15c-088f2c747814\") " pod="kuadrant-system/authorino-59b65697db-twb97"
Aug 04 18:01:02.850171 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.850134    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8838434-e37d-444f-a78e-a3291152778e-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "d8838434-e37d-444f-a78e-a3291152778e" (UID: "d8838434-e37d-444f-a78e-a3291152778e"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:01:02.850603 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.850580    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80ccd01b-e1f6-4a2c-896f-e848adc2a3fa-kube-api-access-96x57" (OuterVolumeSpecName: "kube-api-access-96x57") pod "80ccd01b-e1f6-4a2c-896f-e848adc2a3fa" (UID: "80ccd01b-e1f6-4a2c-896f-e848adc2a3fa"). InnerVolumeSpecName "kube-api-access-96x57". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:01:02.850698 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.850617    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8838434-e37d-444f-a78e-a3291152778e-kube-api-access-4q5md" (OuterVolumeSpecName: "kube-api-access-4q5md") pod "d8838434-e37d-444f-a78e-a3291152778e" (UID: "d8838434-e37d-444f-a78e-a3291152778e"). InnerVolumeSpecName "kube-api-access-4q5md". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:01:02.850764 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.850744    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d91129c1-d186-48d5-b15c-088f2c747814-tls-cert\") pod \"authorino-59b65697db-twb97\" (UID: \"d91129c1-d186-48d5-b15c-088f2c747814\") " pod="kuadrant-system/authorino-59b65697db-twb97"
Aug 04 18:01:02.859446 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.859422    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8rj5\" (UniqueName: \"kubernetes.io/projected/d91129c1-d186-48d5-b15c-088f2c747814-kube-api-access-h8rj5\") pod \"authorino-59b65697db-twb97\" (UID: \"d91129c1-d186-48d5-b15c-088f2c747814\") " pod="kuadrant-system/authorino-59b65697db-twb97"
Aug 04 18:01:02.893064 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.893041    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59b65697db-twb97"
Aug 04 18:01:02.949538 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.949467    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4q5md\" (UniqueName: \"kubernetes.io/projected/d8838434-e37d-444f-a78e-a3291152778e-kube-api-access-4q5md\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:01:02.949538 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.949498    2560 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d8838434-e37d-444f-a78e-a3291152778e-tls-cert\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:01:02.949538 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:02.949513    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96x57\" (UniqueName: \"kubernetes.io/projected/80ccd01b-e1f6-4a2c-896f-e848adc2a3fa-kube-api-access-96x57\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:01:03.039956 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:03.035163    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59b65697db-twb97"]
Aug 04 18:01:03.715750 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:03.715712    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59b65697db-twb97" event={"ID":"d91129c1-d186-48d5-b15c-088f2c747814","Type":"ContainerStarted","Data":"8108bd082d0e4d49ed1b6b25fc685c3cd7c6cc428e9ce8471423bd18c5daba4a"}
Aug 04 18:01:03.715750 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:03.715744    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-ggfzv"
Aug 04 18:01:03.715964 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:03.715742    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-rhk5j"
Aug 04 18:01:03.751373 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:03.751349    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-rhk5j"]
Aug 04 18:01:03.756097 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:03.756074    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-rhk5j"]
Aug 04 18:01:03.779724 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:03.779701    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-687bd45585-ggfzv"]
Aug 04 18:01:03.784291 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:03.784270    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-687bd45585-ggfzv"]
Aug 04 18:01:04.353719 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:04.353682    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80ccd01b-e1f6-4a2c-896f-e848adc2a3fa" path="/var/lib/kubelet/pods/80ccd01b-e1f6-4a2c-896f-e848adc2a3fa/volumes"
Aug 04 18:01:04.353968 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:04.353955    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8838434-e37d-444f-a78e-a3291152778e" path="/var/lib/kubelet/pods/d8838434-e37d-444f-a78e-a3291152778e/volumes"
Aug 04 18:01:04.722017 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:04.721973    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59b65697db-twb97" event={"ID":"d91129c1-d186-48d5-b15c-088f2c747814","Type":"ContainerStarted","Data":"c9b65b9369e0d4b96c8c67e143e0fab9693fd4fa9a405803fd3d65b4ccdca2d5"}
Aug 04 18:01:04.740917 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:04.740876    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59b65697db-twb97" podStartSLOduration=1.935113112 podStartE2EDuration="2.740863007s" totalImagesPullingTime="805.749895ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:01:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:04.739824222 +0000 UTC m=+546.922984400" watchObservedRunningTime="2026-08-04 18:01:04.740863007 +0000 UTC m=+546.924023183"
Aug 04 18:01:04.769859 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:04.769832    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-gz7lp"]
Aug 04 18:01:04.770405 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:04.770004    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-gz7lp" podUID="285e371e-1230-4e16-8c57-d9e93980ac5c" containerName="authorino" containerID="cri-o://44331051d714f7480d8719ae2d293ebe411dab0ad4348ca701a90e1cab4d3a3e" gracePeriod=30
Aug 04 18:01:05.059502 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:05.059477    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-gz7lp"
Aug 04 18:01:05.167905 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:05.167877    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-plplx\" (UniqueName: \"kubernetes.io/projected/285e371e-1230-4e16-8c57-d9e93980ac5c-kube-api-access-plplx\") pod \"285e371e-1230-4e16-8c57-d9e93980ac5c\" (UID: \"285e371e-1230-4e16-8c57-d9e93980ac5c\") "
Aug 04 18:01:05.170029 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:05.169983    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/285e371e-1230-4e16-8c57-d9e93980ac5c-kube-api-access-plplx" (OuterVolumeSpecName: "kube-api-access-plplx") pod "285e371e-1230-4e16-8c57-d9e93980ac5c" (UID: "285e371e-1230-4e16-8c57-d9e93980ac5c"). InnerVolumeSpecName "kube-api-access-plplx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:01:05.269411 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:05.269355    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plplx\" (UniqueName: \"kubernetes.io/projected/285e371e-1230-4e16-8c57-d9e93980ac5c-kube-api-access-plplx\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:01:05.727056 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:05.727022    2560 generic.go:356] "Generic (PLEG): container finished" podID="285e371e-1230-4e16-8c57-d9e93980ac5c" containerID="44331051d714f7480d8719ae2d293ebe411dab0ad4348ca701a90e1cab4d3a3e" exitCode=0
Aug 04 18:01:05.727324 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:05.727062    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-gz7lp"
Aug 04 18:01:05.727324 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:05.727118    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-gz7lp" event={"ID":"285e371e-1230-4e16-8c57-d9e93980ac5c","Type":"ContainerDied","Data":"44331051d714f7480d8719ae2d293ebe411dab0ad4348ca701a90e1cab4d3a3e"}
Aug 04 18:01:05.727324 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:05.727164    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-gz7lp" event={"ID":"285e371e-1230-4e16-8c57-d9e93980ac5c","Type":"ContainerDied","Data":"3796107d55100197c217feb07db903f51fff62720a82bb1bbe62fdcca8f91078"}
Aug 04 18:01:05.727324 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:05.727187    2560 scope.go:117] "RemoveContainer" containerID="44331051d714f7480d8719ae2d293ebe411dab0ad4348ca701a90e1cab4d3a3e"
Aug 04 18:01:05.735902 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:05.735884    2560 scope.go:117] "RemoveContainer" containerID="44331051d714f7480d8719ae2d293ebe411dab0ad4348ca701a90e1cab4d3a3e"
Aug 04 18:01:05.736161 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:01:05.736140    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44331051d714f7480d8719ae2d293ebe411dab0ad4348ca701a90e1cab4d3a3e\": container with ID starting with 44331051d714f7480d8719ae2d293ebe411dab0ad4348ca701a90e1cab4d3a3e not found: ID does not exist" containerID="44331051d714f7480d8719ae2d293ebe411dab0ad4348ca701a90e1cab4d3a3e"
Aug 04 18:01:05.736223 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:05.736169    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44331051d714f7480d8719ae2d293ebe411dab0ad4348ca701a90e1cab4d3a3e"} err="failed to get container status \"44331051d714f7480d8719ae2d293ebe411dab0ad4348ca701a90e1cab4d3a3e\": rpc error: code = NotFound desc = could not find container \"44331051d714f7480d8719ae2d293ebe411dab0ad4348ca701a90e1cab4d3a3e\": container with ID starting with 44331051d714f7480d8719ae2d293ebe411dab0ad4348ca701a90e1cab4d3a3e not found: ID does not exist"
Aug 04 18:01:05.747896 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:05.747876    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-gz7lp"]
Aug 04 18:01:05.752800 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:05.752781    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-gz7lp"]
Aug 04 18:01:06.355454 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:06.355420    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="285e371e-1230-4e16-8c57-d9e93980ac5c" path="/var/lib/kubelet/pods/285e371e-1230-4e16-8c57-d9e93980ac5c/volumes"
Aug 04 18:01:09.515491 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:09.515460    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-579cdc86b5-4d47b"]
Aug 04 18:01:09.515929 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:09.515917    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="285e371e-1230-4e16-8c57-d9e93980ac5c" containerName="authorino"
Aug 04 18:01:09.516015 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:09.515934    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="285e371e-1230-4e16-8c57-d9e93980ac5c" containerName="authorino"
Aug 04 18:01:09.516090 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:09.516073    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="285e371e-1230-4e16-8c57-d9e93980ac5c" containerName="authorino"
Aug 04 18:01:09.542612 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:09.542585    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579cdc86b5-4d47b"]
Aug 04 18:01:09.542777 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:09.542703    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cdc86b5-4d47b"
Aug 04 18:01:09.548743 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:09.548723    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 04 18:01:09.548931 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:09.548889    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-lmgls\""
Aug 04 18:01:09.548931 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:09.548915    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 04 18:01:09.703319 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:09.703290    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f1163da-3a07-4a30-95eb-b50480da8792-webhook-cert\") pod \"maas-controller-579cdc86b5-4d47b\" (UID: \"8f1163da-3a07-4a30-95eb-b50480da8792\") " pod="opendatahub/maas-controller-579cdc86b5-4d47b"
Aug 04 18:01:09.703319 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:09.703319    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbj45\" (UniqueName: \"kubernetes.io/projected/8f1163da-3a07-4a30-95eb-b50480da8792-kube-api-access-pbj45\") pod \"maas-controller-579cdc86b5-4d47b\" (UID: \"8f1163da-3a07-4a30-95eb-b50480da8792\") " pod="opendatahub/maas-controller-579cdc86b5-4d47b"
Aug 04 18:01:09.715454 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:09.715429    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-57c8d79cc7-xkmx9"]
Aug 04 18:01:09.717599 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:09.717585    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57c8d79cc7-xkmx9"
Aug 04 18:01:09.735031 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:09.735010    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57c8d79cc7-xkmx9"]
Aug 04 18:01:09.803808 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:09.803750    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ww7t\" (UniqueName: \"kubernetes.io/projected/23539736-56e8-4000-afd2-9ea4fce9a012-kube-api-access-2ww7t\") pod \"maas-controller-57c8d79cc7-xkmx9\" (UID: \"23539736-56e8-4000-afd2-9ea4fce9a012\") " pod="opendatahub/maas-controller-57c8d79cc7-xkmx9"
Aug 04 18:01:09.803808 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:09.803804    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f1163da-3a07-4a30-95eb-b50480da8792-webhook-cert\") pod \"maas-controller-579cdc86b5-4d47b\" (UID: \"8f1163da-3a07-4a30-95eb-b50480da8792\") " pod="opendatahub/maas-controller-579cdc86b5-4d47b"
Aug 04 18:01:09.803947 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:09.803823    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbj45\" (UniqueName: \"kubernetes.io/projected/8f1163da-3a07-4a30-95eb-b50480da8792-kube-api-access-pbj45\") pod \"maas-controller-579cdc86b5-4d47b\" (UID: \"8f1163da-3a07-4a30-95eb-b50480da8792\") " pod="opendatahub/maas-controller-579cdc86b5-4d47b"
Aug 04 18:01:09.803947 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:09.803881    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/23539736-56e8-4000-afd2-9ea4fce9a012-webhook-cert\") pod \"maas-controller-57c8d79cc7-xkmx9\" (UID: \"23539736-56e8-4000-afd2-9ea4fce9a012\") " pod="opendatahub/maas-controller-57c8d79cc7-xkmx9"
Aug 04 18:01:09.804050 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:01:09.803969    2560 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Aug 04 18:01:09.804050 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:01:09.804031    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f1163da-3a07-4a30-95eb-b50480da8792-webhook-cert podName:8f1163da-3a07-4a30-95eb-b50480da8792 nodeName:}" failed. No retries permitted until 2026-08-04 18:01:10.304017513 +0000 UTC m=+552.487177667 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/8f1163da-3a07-4a30-95eb-b50480da8792-webhook-cert") pod "maas-controller-579cdc86b5-4d47b" (UID: "8f1163da-3a07-4a30-95eb-b50480da8792") : secret "maas-controller-webhook-cert" not found
Aug 04 18:01:09.820024 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:09.820005    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbj45\" (UniqueName: \"kubernetes.io/projected/8f1163da-3a07-4a30-95eb-b50480da8792-kube-api-access-pbj45\") pod \"maas-controller-579cdc86b5-4d47b\" (UID: \"8f1163da-3a07-4a30-95eb-b50480da8792\") " pod="opendatahub/maas-controller-579cdc86b5-4d47b"
Aug 04 18:01:09.904945 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:09.904915    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ww7t\" (UniqueName: \"kubernetes.io/projected/23539736-56e8-4000-afd2-9ea4fce9a012-kube-api-access-2ww7t\") pod \"maas-controller-57c8d79cc7-xkmx9\" (UID: \"23539736-56e8-4000-afd2-9ea4fce9a012\") " pod="opendatahub/maas-controller-57c8d79cc7-xkmx9"
Aug 04 18:01:09.905104 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:09.905023    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/23539736-56e8-4000-afd2-9ea4fce9a012-webhook-cert\") pod \"maas-controller-57c8d79cc7-xkmx9\" (UID: \"23539736-56e8-4000-afd2-9ea4fce9a012\") " pod="opendatahub/maas-controller-57c8d79cc7-xkmx9"
Aug 04 18:01:09.907501 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:09.907483    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/23539736-56e8-4000-afd2-9ea4fce9a012-webhook-cert\") pod \"maas-controller-57c8d79cc7-xkmx9\" (UID: \"23539736-56e8-4000-afd2-9ea4fce9a012\") " pod="opendatahub/maas-controller-57c8d79cc7-xkmx9"
Aug 04 18:01:09.917945 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:09.917922    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ww7t\" (UniqueName: \"kubernetes.io/projected/23539736-56e8-4000-afd2-9ea4fce9a012-kube-api-access-2ww7t\") pod \"maas-controller-57c8d79cc7-xkmx9\" (UID: \"23539736-56e8-4000-afd2-9ea4fce9a012\") " pod="opendatahub/maas-controller-57c8d79cc7-xkmx9"
Aug 04 18:01:10.026769 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:10.026743    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57c8d79cc7-xkmx9"
Aug 04 18:01:10.164834 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:10.164811    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57c8d79cc7-xkmx9"]
Aug 04 18:01:10.166895 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:01:10.166868    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23539736_56e8_4000_afd2_9ea4fce9a012.slice/crio-ff5e532d3de8984169a4d858adacc9b112212ce441c6f988206630a550ddd69d WatchSource:0}: Error finding container ff5e532d3de8984169a4d858adacc9b112212ce441c6f988206630a550ddd69d: Status 404 returned error can't find the container with id ff5e532d3de8984169a4d858adacc9b112212ce441c6f988206630a550ddd69d
Aug 04 18:01:10.307902 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:10.307873    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f1163da-3a07-4a30-95eb-b50480da8792-webhook-cert\") pod \"maas-controller-579cdc86b5-4d47b\" (UID: \"8f1163da-3a07-4a30-95eb-b50480da8792\") " pod="opendatahub/maas-controller-579cdc86b5-4d47b"
Aug 04 18:01:10.310309 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:10.310290    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f1163da-3a07-4a30-95eb-b50480da8792-webhook-cert\") pod \"maas-controller-579cdc86b5-4d47b\" (UID: \"8f1163da-3a07-4a30-95eb-b50480da8792\") " pod="opendatahub/maas-controller-579cdc86b5-4d47b"
Aug 04 18:01:10.453409 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:10.453384    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cdc86b5-4d47b"
Aug 04 18:01:10.595289 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:10.595254    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579cdc86b5-4d47b"]
Aug 04 18:01:10.597876 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:01:10.597848    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f1163da_3a07_4a30_95eb_b50480da8792.slice/crio-89be18ec08db57587fcdc902c517db8419957587d25b912878d404a5bd96d59d WatchSource:0}: Error finding container 89be18ec08db57587fcdc902c517db8419957587d25b912878d404a5bd96d59d: Status 404 returned error can't find the container with id 89be18ec08db57587fcdc902c517db8419957587d25b912878d404a5bd96d59d
Aug 04 18:01:10.751797 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:10.751726    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cdc86b5-4d47b" event={"ID":"8f1163da-3a07-4a30-95eb-b50480da8792","Type":"ContainerStarted","Data":"89be18ec08db57587fcdc902c517db8419957587d25b912878d404a5bd96d59d"}
Aug 04 18:01:10.752895 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:10.752871    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57c8d79cc7-xkmx9" event={"ID":"23539736-56e8-4000-afd2-9ea4fce9a012","Type":"ContainerStarted","Data":"ff5e532d3de8984169a4d858adacc9b112212ce441c6f988206630a550ddd69d"}
Aug 04 18:01:13.769975 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:13.769922    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cdc86b5-4d47b" event={"ID":"8f1163da-3a07-4a30-95eb-b50480da8792","Type":"ContainerStarted","Data":"8bc376399cf755dfd9518ab3eac72df54a43b8840aa76b720fa6ed56efdc10bc"}
Aug 04 18:01:13.770481 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:13.770081    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-579cdc86b5-4d47b"
Aug 04 18:01:13.771899 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:13.771841    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57c8d79cc7-xkmx9" event={"ID":"23539736-56e8-4000-afd2-9ea4fce9a012","Type":"ContainerStarted","Data":"edbb1d6059d60cada1391baddf05af02270619edab6c3cdd41329ed760582933"}
Aug 04 18:01:13.772057 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:13.771940    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-57c8d79cc7-xkmx9"
Aug 04 18:01:13.794082 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:13.794037    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-579cdc86b5-4d47b" podStartSLOduration=2.007443771 podStartE2EDuration="4.794019771s" totalImagesPullingTime="2.786576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:01:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:13.792798969 +0000 UTC m=+555.975959143" watchObservedRunningTime="2026-08-04 18:01:13.794019771 +0000 UTC m=+555.977179950"
Aug 04 18:01:13.817209 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:13.817170    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-57c8d79cc7-xkmx9" podStartSLOduration=1.597882243 podStartE2EDuration="4.817159734s" totalImagesPullingTime="3.219277491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:01:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:13.816589121 +0000 UTC m=+555.999749297" watchObservedRunningTime="2026-08-04 18:01:13.817159734 +0000 UTC m=+556.000319909"
Aug 04 18:01:24.779578 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:24.779548    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-57c8d79cc7-xkmx9"
Aug 04 18:01:24.780138 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:24.779628    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-579cdc86b5-4d47b"
Aug 04 18:01:24.853559 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:24.853520    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579cdc86b5-4d47b"]
Aug 04 18:01:24.853743 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:24.853713    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-579cdc86b5-4d47b" podUID="8f1163da-3a07-4a30-95eb-b50480da8792" containerName="manager" containerID="cri-o://8bc376399cf755dfd9518ab3eac72df54a43b8840aa76b720fa6ed56efdc10bc" gracePeriod=30
Aug 04 18:01:25.096507 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:25.096483    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cdc86b5-4d47b"
Aug 04 18:01:25.133294 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:25.133267    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pbj45\" (UniqueName: \"kubernetes.io/projected/8f1163da-3a07-4a30-95eb-b50480da8792-kube-api-access-pbj45\") pod \"8f1163da-3a07-4a30-95eb-b50480da8792\" (UID: \"8f1163da-3a07-4a30-95eb-b50480da8792\") "
Aug 04 18:01:25.133474 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:25.133353    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f1163da-3a07-4a30-95eb-b50480da8792-webhook-cert\") pod \"8f1163da-3a07-4a30-95eb-b50480da8792\" (UID: \"8f1163da-3a07-4a30-95eb-b50480da8792\") "
Aug 04 18:01:25.135578 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:25.135545    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f1163da-3a07-4a30-95eb-b50480da8792-kube-api-access-pbj45" (OuterVolumeSpecName: "kube-api-access-pbj45") pod "8f1163da-3a07-4a30-95eb-b50480da8792" (UID: "8f1163da-3a07-4a30-95eb-b50480da8792"). InnerVolumeSpecName "kube-api-access-pbj45". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:01:25.135685 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:25.135609    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f1163da-3a07-4a30-95eb-b50480da8792-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8f1163da-3a07-4a30-95eb-b50480da8792" (UID: "8f1163da-3a07-4a30-95eb-b50480da8792"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:01:25.234466 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:25.234431    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbj45\" (UniqueName: \"kubernetes.io/projected/8f1163da-3a07-4a30-95eb-b50480da8792-kube-api-access-pbj45\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:01:25.234466 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:25.234460    2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f1163da-3a07-4a30-95eb-b50480da8792-webhook-cert\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:01:25.834757 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:25.834722    2560 generic.go:356] "Generic (PLEG): container finished" podID="8f1163da-3a07-4a30-95eb-b50480da8792" containerID="8bc376399cf755dfd9518ab3eac72df54a43b8840aa76b720fa6ed56efdc10bc" exitCode=0
Aug 04 18:01:25.834757 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:25.834768    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cdc86b5-4d47b" event={"ID":"8f1163da-3a07-4a30-95eb-b50480da8792","Type":"ContainerDied","Data":"8bc376399cf755dfd9518ab3eac72df54a43b8840aa76b720fa6ed56efdc10bc"}
Aug 04 18:01:25.835276 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:25.834794    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cdc86b5-4d47b"
Aug 04 18:01:25.835276 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:25.834835    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cdc86b5-4d47b" event={"ID":"8f1163da-3a07-4a30-95eb-b50480da8792","Type":"ContainerDied","Data":"89be18ec08db57587fcdc902c517db8419957587d25b912878d404a5bd96d59d"}
Aug 04 18:01:25.835276 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:25.834857    2560 scope.go:117] "RemoveContainer" containerID="8bc376399cf755dfd9518ab3eac72df54a43b8840aa76b720fa6ed56efdc10bc"
Aug 04 18:01:25.844305 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:25.844289    2560 scope.go:117] "RemoveContainer" containerID="8bc376399cf755dfd9518ab3eac72df54a43b8840aa76b720fa6ed56efdc10bc"
Aug 04 18:01:25.844558 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:01:25.844540    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bc376399cf755dfd9518ab3eac72df54a43b8840aa76b720fa6ed56efdc10bc\": container with ID starting with 8bc376399cf755dfd9518ab3eac72df54a43b8840aa76b720fa6ed56efdc10bc not found: ID does not exist" containerID="8bc376399cf755dfd9518ab3eac72df54a43b8840aa76b720fa6ed56efdc10bc"
Aug 04 18:01:25.844619 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:25.844566    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bc376399cf755dfd9518ab3eac72df54a43b8840aa76b720fa6ed56efdc10bc"} err="failed to get container status \"8bc376399cf755dfd9518ab3eac72df54a43b8840aa76b720fa6ed56efdc10bc\": rpc error: code = NotFound desc = could not find container \"8bc376399cf755dfd9518ab3eac72df54a43b8840aa76b720fa6ed56efdc10bc\": container with ID starting with 8bc376399cf755dfd9518ab3eac72df54a43b8840aa76b720fa6ed56efdc10bc not found: ID does not exist"
Aug 04 18:01:25.859769 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:25.859747    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579cdc86b5-4d47b"]
Aug 04 18:01:25.864846 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:25.864824    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-579cdc86b5-4d47b"]
Aug 04 18:01:26.353359 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:26.353328    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f1163da-3a07-4a30-95eb-b50480da8792" path="/var/lib/kubelet/pods/8f1163da-3a07-4a30-95eb-b50480da8792/volumes"
Aug 04 18:01:47.012780 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:47.012749    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-67c8968587-5cqm2"]
Aug 04 18:01:47.013231 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:47.013137    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f1163da-3a07-4a30-95eb-b50480da8792" containerName="manager"
Aug 04 18:01:47.013231 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:47.013149    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f1163da-3a07-4a30-95eb-b50480da8792" containerName="manager"
Aug 04 18:01:47.013231 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:47.013210    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f1163da-3a07-4a30-95eb-b50480da8792" containerName="manager"
Aug 04 18:01:47.017460 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:47.017444    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-67c8968587-5cqm2"
Aug 04 18:01:47.034411 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:47.034386    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Aug 04 18:01:47.036416 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:47.036394    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vrzn8\""
Aug 04 18:01:47.037637 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:47.037614    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-67c8968587-5cqm2"]
Aug 04 18:01:47.106052 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:47.106027    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96d4bdeb-9a3c-41e1-9568-3fab780c0469-maas-api-tls\") pod \"maas-api-67c8968587-5cqm2\" (UID: \"96d4bdeb-9a3c-41e1-9568-3fab780c0469\") " pod="odh-ai-gateway-infra/maas-api-67c8968587-5cqm2"
Aug 04 18:01:47.106159 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:47.106070    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96d4bdeb-9a3c-41e1-9568-3fab780c0469-openshift-service-ca\") pod \"maas-api-67c8968587-5cqm2\" (UID: \"96d4bdeb-9a3c-41e1-9568-3fab780c0469\") " pod="odh-ai-gateway-infra/maas-api-67c8968587-5cqm2"
Aug 04 18:01:47.106159 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:47.106121    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5bbv\" (UniqueName: \"kubernetes.io/projected/96d4bdeb-9a3c-41e1-9568-3fab780c0469-kube-api-access-v5bbv\") pod \"maas-api-67c8968587-5cqm2\" (UID: \"96d4bdeb-9a3c-41e1-9568-3fab780c0469\") " pod="odh-ai-gateway-infra/maas-api-67c8968587-5cqm2"
Aug 04 18:01:47.207281 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:47.207254    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5bbv\" (UniqueName: \"kubernetes.io/projected/96d4bdeb-9a3c-41e1-9568-3fab780c0469-kube-api-access-v5bbv\") pod \"maas-api-67c8968587-5cqm2\" (UID: \"96d4bdeb-9a3c-41e1-9568-3fab780c0469\") " pod="odh-ai-gateway-infra/maas-api-67c8968587-5cqm2"
Aug 04 18:01:47.207417 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:47.207318    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96d4bdeb-9a3c-41e1-9568-3fab780c0469-maas-api-tls\") pod \"maas-api-67c8968587-5cqm2\" (UID: \"96d4bdeb-9a3c-41e1-9568-3fab780c0469\") " pod="odh-ai-gateway-infra/maas-api-67c8968587-5cqm2"
Aug 04 18:01:47.207417 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:47.207355    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96d4bdeb-9a3c-41e1-9568-3fab780c0469-openshift-service-ca\") pod \"maas-api-67c8968587-5cqm2\" (UID: \"96d4bdeb-9a3c-41e1-9568-3fab780c0469\") " pod="odh-ai-gateway-infra/maas-api-67c8968587-5cqm2"
Aug 04 18:01:47.208119 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:47.208024    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96d4bdeb-9a3c-41e1-9568-3fab780c0469-openshift-service-ca\") pod \"maas-api-67c8968587-5cqm2\" (UID: \"96d4bdeb-9a3c-41e1-9568-3fab780c0469\") " pod="odh-ai-gateway-infra/maas-api-67c8968587-5cqm2"
Aug 04 18:01:47.209907 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:47.209888    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96d4bdeb-9a3c-41e1-9568-3fab780c0469-maas-api-tls\") pod \"maas-api-67c8968587-5cqm2\" (UID: \"96d4bdeb-9a3c-41e1-9568-3fab780c0469\") " pod="odh-ai-gateway-infra/maas-api-67c8968587-5cqm2"
Aug 04 18:01:47.220376 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:47.220356    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5bbv\" (UniqueName: \"kubernetes.io/projected/96d4bdeb-9a3c-41e1-9568-3fab780c0469-kube-api-access-v5bbv\") pod \"maas-api-67c8968587-5cqm2\" (UID: \"96d4bdeb-9a3c-41e1-9568-3fab780c0469\") " pod="odh-ai-gateway-infra/maas-api-67c8968587-5cqm2"
Aug 04 18:01:47.327674 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:47.327555    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-67c8968587-5cqm2"
Aug 04 18:01:47.478060 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:47.478035    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-67c8968587-5cqm2"]
Aug 04 18:01:47.480358 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:01:47.480334    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96d4bdeb_9a3c_41e1_9568_3fab780c0469.slice/crio-7af11f782e5960e4acce9207d76f58bba5ca71644d3bdac399ede017cf9e00d8 WatchSource:0}: Error finding container 7af11f782e5960e4acce9207d76f58bba5ca71644d3bdac399ede017cf9e00d8: Status 404 returned error can't find the container with id 7af11f782e5960e4acce9207d76f58bba5ca71644d3bdac399ede017cf9e00d8
Aug 04 18:01:47.936008 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:47.935952    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-67c8968587-5cqm2" event={"ID":"96d4bdeb-9a3c-41e1-9568-3fab780c0469","Type":"ContainerStarted","Data":"7af11f782e5960e4acce9207d76f58bba5ca71644d3bdac399ede017cf9e00d8"}
Aug 04 18:01:47.941681 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:47.941658    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-4kddb"]
Aug 04 18:01:47.944411 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:47.944396    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4kddb"
Aug 04 18:01:47.947758 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:47.947740    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Aug 04 18:01:47.948344 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:47.948328    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-kcdhm\""
Aug 04 18:01:47.963159 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:47.963136    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-4kddb"]
Aug 04 18:01:48.014787 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:48.014763    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6zjs\" (UniqueName: \"kubernetes.io/projected/7fff5555-9d9f-475d-82af-84fcf3e5a7b6-kube-api-access-g6zjs\") pod \"payload-pre-processing-687dbb6cf6-4kddb\" (UID: \"7fff5555-9d9f-475d-82af-84fcf3e5a7b6\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4kddb"
Aug 04 18:01:48.015168 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:48.014812    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fff5555-9d9f-475d-82af-84fcf3e5a7b6-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-4kddb\" (UID: \"7fff5555-9d9f-475d-82af-84fcf3e5a7b6\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4kddb"
Aug 04 18:01:48.025634 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:48.025615    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-6wqzc"]
Aug 04 18:01:48.027826 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:48.027810    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-6wqzc"
Aug 04 18:01:48.041162 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:48.041140    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-6wqzc"]
Aug 04 18:01:48.115924 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:48.115902    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6zjs\" (UniqueName: \"kubernetes.io/projected/7fff5555-9d9f-475d-82af-84fcf3e5a7b6-kube-api-access-g6zjs\") pod \"payload-pre-processing-687dbb6cf6-4kddb\" (UID: \"7fff5555-9d9f-475d-82af-84fcf3e5a7b6\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4kddb"
Aug 04 18:01:48.116052 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:48.115938    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fff5555-9d9f-475d-82af-84fcf3e5a7b6-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-4kddb\" (UID: \"7fff5555-9d9f-475d-82af-84fcf3e5a7b6\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4kddb"
Aug 04 18:01:48.116052 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:48.115969    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwmkz\" (UniqueName: \"kubernetes.io/projected/12873b28-6874-43e7-b589-16b8f7f7fef4-kube-api-access-rwmkz\") pod \"payload-processing-5cc6cd444b-6wqzc\" (UID: \"12873b28-6874-43e7-b589-16b8f7f7fef4\") " pod="openshift-ingress/payload-processing-5cc6cd444b-6wqzc"
Aug 04 18:01:48.116142 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:48.116122    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12873b28-6874-43e7-b589-16b8f7f7fef4-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-6wqzc\" (UID: \"12873b28-6874-43e7-b589-16b8f7f7fef4\") " pod="openshift-ingress/payload-processing-5cc6cd444b-6wqzc"
Aug 04 18:01:48.116573 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:48.116556    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fff5555-9d9f-475d-82af-84fcf3e5a7b6-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-4kddb\" (UID: \"7fff5555-9d9f-475d-82af-84fcf3e5a7b6\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4kddb"
Aug 04 18:01:48.126138 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:48.126121    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6zjs\" (UniqueName: \"kubernetes.io/projected/7fff5555-9d9f-475d-82af-84fcf3e5a7b6-kube-api-access-g6zjs\") pod \"payload-pre-processing-687dbb6cf6-4kddb\" (UID: \"7fff5555-9d9f-475d-82af-84fcf3e5a7b6\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4kddb"
Aug 04 18:01:48.216903 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:48.216844    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwmkz\" (UniqueName: \"kubernetes.io/projected/12873b28-6874-43e7-b589-16b8f7f7fef4-kube-api-access-rwmkz\") pod \"payload-processing-5cc6cd444b-6wqzc\" (UID: \"12873b28-6874-43e7-b589-16b8f7f7fef4\") " pod="openshift-ingress/payload-processing-5cc6cd444b-6wqzc"
Aug 04 18:01:48.216903 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:48.216880    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12873b28-6874-43e7-b589-16b8f7f7fef4-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-6wqzc\" (UID: \"12873b28-6874-43e7-b589-16b8f7f7fef4\") " pod="openshift-ingress/payload-processing-5cc6cd444b-6wqzc"
Aug 04 18:01:48.217528 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:48.217512    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12873b28-6874-43e7-b589-16b8f7f7fef4-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-6wqzc\" (UID: \"12873b28-6874-43e7-b589-16b8f7f7fef4\") " pod="openshift-ingress/payload-processing-5cc6cd444b-6wqzc"
Aug 04 18:01:48.227929 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:48.227909    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwmkz\" (UniqueName: \"kubernetes.io/projected/12873b28-6874-43e7-b589-16b8f7f7fef4-kube-api-access-rwmkz\") pod \"payload-processing-5cc6cd444b-6wqzc\" (UID: \"12873b28-6874-43e7-b589-16b8f7f7fef4\") " pod="openshift-ingress/payload-processing-5cc6cd444b-6wqzc"
Aug 04 18:01:48.255433 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:48.255411    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4kddb"
Aug 04 18:01:48.339623 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:48.339589    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-6wqzc"
Aug 04 18:01:48.393282 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:48.393249    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-4kddb"]
Aug 04 18:01:48.398501 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:01:48.398428    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fff5555_9d9f_475d_82af_84fcf3e5a7b6.slice/crio-ebb2b385e536fbc664b15c549dc06d816aa61e0c5225e6be12a26a50fef01387 WatchSource:0}: Error finding container ebb2b385e536fbc664b15c549dc06d816aa61e0c5225e6be12a26a50fef01387: Status 404 returned error can't find the container with id ebb2b385e536fbc664b15c549dc06d816aa61e0c5225e6be12a26a50fef01387
Aug 04 18:01:48.486819 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:48.485061    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-6wqzc"]
Aug 04 18:01:48.488671 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:01:48.488647    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12873b28_6874_43e7_b589_16b8f7f7fef4.slice/crio-0153ae14b0a2be0a66b47c91d4d04be487ed5c2391ab49c24c680948d4b40023 WatchSource:0}: Error finding container 0153ae14b0a2be0a66b47c91d4d04be487ed5c2391ab49c24c680948d4b40023: Status 404 returned error can't find the container with id 0153ae14b0a2be0a66b47c91d4d04be487ed5c2391ab49c24c680948d4b40023
Aug 04 18:01:48.942636 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:48.942599    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4kddb" event={"ID":"7fff5555-9d9f-475d-82af-84fcf3e5a7b6","Type":"ContainerStarted","Data":"ebb2b385e536fbc664b15c549dc06d816aa61e0c5225e6be12a26a50fef01387"}
Aug 04 18:01:48.943964 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:48.943935    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-6wqzc" event={"ID":"12873b28-6874-43e7-b589-16b8f7f7fef4","Type":"ContainerStarted","Data":"0153ae14b0a2be0a66b47c91d4d04be487ed5c2391ab49c24c680948d4b40023"}
Aug 04 18:01:51.960403 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:51.960364    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4kddb" event={"ID":"7fff5555-9d9f-475d-82af-84fcf3e5a7b6","Type":"ContainerStarted","Data":"54640e27efe9c9ccc3cb5533bed05699d692f3794ccd0fbdab2025276beef7b6"}
Aug 04 18:01:51.960817 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:51.960449    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4kddb"
Aug 04 18:01:51.961975 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:51.961950    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-6wqzc" event={"ID":"12873b28-6874-43e7-b589-16b8f7f7fef4","Type":"ContainerStarted","Data":"db921c837fa19e3b3ae2c3d3a69c39a0220d07551469312fc38c13778d232697"}
Aug 04 18:01:51.962132 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:51.962027    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-6wqzc"
Aug 04 18:01:51.963640 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:51.963610    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-67c8968587-5cqm2" event={"ID":"96d4bdeb-9a3c-41e1-9568-3fab780c0469","Type":"ContainerStarted","Data":"7d372c4210b0299ef7d4f445859add3ce4553863d9407736c52318c521b8dbaa"}
Aug 04 18:01:51.963746 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:51.963693    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-67c8968587-5cqm2"
Aug 04 18:01:51.980624 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:51.980586    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4kddb" podStartSLOduration=1.56624688 podStartE2EDuration="4.98057459s" totalImagesPullingTime="3.41432771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:01:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:51.979643057 +0000 UTC m=+594.162803229" watchObservedRunningTime="2026-08-04 18:01:51.98057459 +0000 UTC m=+594.163734765"
Aug 04 18:01:52.001519 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:52.001476    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-6wqzc" podStartSLOduration=0.920659169 podStartE2EDuration="4.001463596s" totalImagesPullingTime="3.080804427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:01:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:51.999466828 +0000 UTC m=+594.182627003" watchObservedRunningTime="2026-08-04 18:01:52.001463596 +0000 UTC m=+594.184623810"
Aug 04 18:01:52.018407 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:52.018370    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-67c8968587-5cqm2" podStartSLOduration=1.929950717 podStartE2EDuration="6.018359078s" totalImagesPullingTime="4.088408361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:01:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:01:52.017450808 +0000 UTC m=+594.200610984" watchObservedRunningTime="2026-08-04 18:01:52.018359078 +0000 UTC m=+594.201519253"
Aug 04 18:01:57.974154 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:57.974128    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-67c8968587-5cqm2"
Aug 04 18:01:58.279741 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:58.279673    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/2.log"
Aug 04 18:01:58.281726 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:58.281705    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/2.log"
Aug 04 18:01:58.285019 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:58.284982    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 18:01:58.286949 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:01:58.286932    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 18:02:02.970410 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:02.970381    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-6wqzc"
Aug 04 18:02:02.970898 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:02.970675    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4kddb"
Aug 04 18:02:17.633094 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:17.633059    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg"]
Aug 04 18:02:17.636414 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:17.636395    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg"
Aug 04 18:02:17.648276 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:17.648258    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Aug 04 18:02:17.648376 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:17.648300    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Aug 04 18:02:17.649174 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:17.649155    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Aug 04 18:02:17.649275 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:17.649190    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-dk8xl\""
Aug 04 18:02:17.669629 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:17.669605    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg"]
Aug 04 18:02:17.756965 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:17.756938    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/997763e6-8ea2-4603-be9f-123e78c2b87e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg\" (UID: \"997763e6-8ea2-4603-be9f-123e78c2b87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg"
Aug 04 18:02:17.756965 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:17.756969    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwljd\" (UniqueName: \"kubernetes.io/projected/997763e6-8ea2-4603-be9f-123e78c2b87e-kube-api-access-bwljd\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg\" (UID: \"997763e6-8ea2-4603-be9f-123e78c2b87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg"
Aug 04 18:02:17.757165 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:17.757078    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/997763e6-8ea2-4603-be9f-123e78c2b87e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg\" (UID: \"997763e6-8ea2-4603-be9f-123e78c2b87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg"
Aug 04 18:02:17.757165 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:17.757106    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/997763e6-8ea2-4603-be9f-123e78c2b87e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg\" (UID: \"997763e6-8ea2-4603-be9f-123e78c2b87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg"
Aug 04 18:02:17.757165 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:17.757130    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/997763e6-8ea2-4603-be9f-123e78c2b87e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg\" (UID: \"997763e6-8ea2-4603-be9f-123e78c2b87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg"
Aug 04 18:02:17.757270 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:17.757173    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/997763e6-8ea2-4603-be9f-123e78c2b87e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg\" (UID: \"997763e6-8ea2-4603-be9f-123e78c2b87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg"
Aug 04 18:02:17.858594 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:17.858569    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/997763e6-8ea2-4603-be9f-123e78c2b87e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg\" (UID: \"997763e6-8ea2-4603-be9f-123e78c2b87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg"
Aug 04 18:02:17.858724 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:17.858600    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/997763e6-8ea2-4603-be9f-123e78c2b87e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg\" (UID: \"997763e6-8ea2-4603-be9f-123e78c2b87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg"
Aug 04 18:02:17.858724 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:17.858621    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/997763e6-8ea2-4603-be9f-123e78c2b87e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg\" (UID: \"997763e6-8ea2-4603-be9f-123e78c2b87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg"
Aug 04 18:02:17.858724 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:17.858640    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/997763e6-8ea2-4603-be9f-123e78c2b87e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg\" (UID: \"997763e6-8ea2-4603-be9f-123e78c2b87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg"
Aug 04 18:02:17.858724 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:17.858692    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/997763e6-8ea2-4603-be9f-123e78c2b87e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg\" (UID: \"997763e6-8ea2-4603-be9f-123e78c2b87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg"
Aug 04 18:02:17.858724 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:17.858721    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwljd\" (UniqueName: \"kubernetes.io/projected/997763e6-8ea2-4603-be9f-123e78c2b87e-kube-api-access-bwljd\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg\" (UID: \"997763e6-8ea2-4603-be9f-123e78c2b87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg"
Aug 04 18:02:17.859176 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:17.859149    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/997763e6-8ea2-4603-be9f-123e78c2b87e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg\" (UID: \"997763e6-8ea2-4603-be9f-123e78c2b87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg"
Aug 04 18:02:17.859176 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:17.859171    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/997763e6-8ea2-4603-be9f-123e78c2b87e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg\" (UID: \"997763e6-8ea2-4603-be9f-123e78c2b87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg"
Aug 04 18:02:17.859406 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:17.859187    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/997763e6-8ea2-4603-be9f-123e78c2b87e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg\" (UID: \"997763e6-8ea2-4603-be9f-123e78c2b87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg"
Aug 04 18:02:17.861208 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:17.861183    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/997763e6-8ea2-4603-be9f-123e78c2b87e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg\" (UID: \"997763e6-8ea2-4603-be9f-123e78c2b87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg"
Aug 04 18:02:17.861288 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:17.861225    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/997763e6-8ea2-4603-be9f-123e78c2b87e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg\" (UID: \"997763e6-8ea2-4603-be9f-123e78c2b87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg"
Aug 04 18:02:17.872511 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:17.872483    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwljd\" (UniqueName: \"kubernetes.io/projected/997763e6-8ea2-4603-be9f-123e78c2b87e-kube-api-access-bwljd\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg\" (UID: \"997763e6-8ea2-4603-be9f-123e78c2b87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg"
Aug 04 18:02:17.946160 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:17.946133    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg"
Aug 04 18:02:18.094852 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:18.094831    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg"]
Aug 04 18:02:18.097130 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:02:18.097105    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod997763e6_8ea2_4603_be9f_123e78c2b87e.slice/crio-10d4577ddbd7b13a6402a0ef44694894283581d0591cd31aa7f8b72d82376582 WatchSource:0}: Error finding container 10d4577ddbd7b13a6402a0ef44694894283581d0591cd31aa7f8b72d82376582: Status 404 returned error can't find the container with id 10d4577ddbd7b13a6402a0ef44694894283581d0591cd31aa7f8b72d82376582
Aug 04 18:02:19.090172 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:19.090111    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg" event={"ID":"997763e6-8ea2-4603-be9f-123e78c2b87e","Type":"ContainerStarted","Data":"10d4577ddbd7b13a6402a0ef44694894283581d0591cd31aa7f8b72d82376582"}
Aug 04 18:02:20.095655 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:20.095619    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg" event={"ID":"997763e6-8ea2-4603-be9f-123e78c2b87e","Type":"ContainerStarted","Data":"72a967e9db2b839e74ce3985678215dad8ec18cfc6028e86b825923aa53ebd0b"}
Aug 04 18:02:20.123519 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:20.123474    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg" podStartSLOduration=2.040485766 podStartE2EDuration="3.123459039s" totalImagesPullingTime="1.082973273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:02:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:02:20.122346964 +0000 UTC m=+622.305507142" watchObservedRunningTime="2026-08-04 18:02:20.123459039 +0000 UTC m=+622.306619232"
Aug 04 18:02:27.946395 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:27.946361    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg"
Aug 04 18:02:27.946968 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:27.946428    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg"
Aug 04 18:02:27.959342 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:27.959321    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg"
Aug 04 18:02:28.142478 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:28.142453    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg"
Aug 04 18:02:31.926213 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:31.924492    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp"]
Aug 04 18:02:31.929798 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:31.929773    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp"
Aug 04 18:02:31.935539 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:31.935516    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Aug 04 18:02:31.944114 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:31.944093    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp"]
Aug 04 18:02:31.971878 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:31.971850    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/336edf09-c429-4f98-a313-edd0af9ace0f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zclgp\" (UID: \"336edf09-c429-4f98-a313-edd0af9ace0f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp"
Aug 04 18:02:31.971981 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:31.971892    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/336edf09-c429-4f98-a313-edd0af9ace0f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zclgp\" (UID: \"336edf09-c429-4f98-a313-edd0af9ace0f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp"
Aug 04 18:02:31.971981 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:31.971917    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/336edf09-c429-4f98-a313-edd0af9ace0f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zclgp\" (UID: \"336edf09-c429-4f98-a313-edd0af9ace0f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp"
Aug 04 18:02:31.972126 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:31.972059    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/336edf09-c429-4f98-a313-edd0af9ace0f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zclgp\" (UID: \"336edf09-c429-4f98-a313-edd0af9ace0f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp"
Aug 04 18:02:31.972126 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:31.972101    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/336edf09-c429-4f98-a313-edd0af9ace0f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zclgp\" (UID: \"336edf09-c429-4f98-a313-edd0af9ace0f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp"
Aug 04 18:02:31.972195 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:31.972132    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l72dx\" (UniqueName: \"kubernetes.io/projected/336edf09-c429-4f98-a313-edd0af9ace0f-kube-api-access-l72dx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zclgp\" (UID: \"336edf09-c429-4f98-a313-edd0af9ace0f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp"
Aug 04 18:02:32.073431 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:32.073407    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/336edf09-c429-4f98-a313-edd0af9ace0f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zclgp\" (UID: \"336edf09-c429-4f98-a313-edd0af9ace0f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp"
Aug 04 18:02:32.073582 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:32.073436    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/336edf09-c429-4f98-a313-edd0af9ace0f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zclgp\" (UID: \"336edf09-c429-4f98-a313-edd0af9ace0f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp"
Aug 04 18:02:32.073582 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:32.073457    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/336edf09-c429-4f98-a313-edd0af9ace0f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zclgp\" (UID: \"336edf09-c429-4f98-a313-edd0af9ace0f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp"
Aug 04 18:02:32.073582 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:32.073513    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/336edf09-c429-4f98-a313-edd0af9ace0f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zclgp\" (UID: \"336edf09-c429-4f98-a313-edd0af9ace0f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp"
Aug 04 18:02:32.073582 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:32.073549    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/336edf09-c429-4f98-a313-edd0af9ace0f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zclgp\" (UID: \"336edf09-c429-4f98-a313-edd0af9ace0f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp"
Aug 04 18:02:32.073804 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:32.073608    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l72dx\" (UniqueName: \"kubernetes.io/projected/336edf09-c429-4f98-a313-edd0af9ace0f-kube-api-access-l72dx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zclgp\" (UID: \"336edf09-c429-4f98-a313-edd0af9ace0f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp"
Aug 04 18:02:32.074046 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:32.074025    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/336edf09-c429-4f98-a313-edd0af9ace0f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zclgp\" (UID: \"336edf09-c429-4f98-a313-edd0af9ace0f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp"
Aug 04 18:02:32.074138 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:32.074085    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/336edf09-c429-4f98-a313-edd0af9ace0f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zclgp\" (UID: \"336edf09-c429-4f98-a313-edd0af9ace0f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp"
Aug 04 18:02:32.074211 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:32.074198    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/336edf09-c429-4f98-a313-edd0af9ace0f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zclgp\" (UID: \"336edf09-c429-4f98-a313-edd0af9ace0f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp"
Aug 04 18:02:32.076140 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:32.076111    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/336edf09-c429-4f98-a313-edd0af9ace0f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zclgp\" (UID: \"336edf09-c429-4f98-a313-edd0af9ace0f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp"
Aug 04 18:02:32.076517 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:32.076497    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/336edf09-c429-4f98-a313-edd0af9ace0f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zclgp\" (UID: \"336edf09-c429-4f98-a313-edd0af9ace0f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp"
Aug 04 18:02:32.090817 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:32.090794    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l72dx\" (UniqueName: \"kubernetes.io/projected/336edf09-c429-4f98-a313-edd0af9ace0f-kube-api-access-l72dx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zclgp\" (UID: \"336edf09-c429-4f98-a313-edd0af9ace0f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp"
Aug 04 18:02:32.240645 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:32.240575    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp"
Aug 04 18:02:32.386199 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:32.386178    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp"]
Aug 04 18:02:32.386917 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:02:32.386894    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod336edf09_c429_4f98_a313_edd0af9ace0f.slice/crio-ab353fbe572b042d93baceb16cd74eb90d26f1cb6b9e06097e8827d6ce264353 WatchSource:0}: Error finding container ab353fbe572b042d93baceb16cd74eb90d26f1cb6b9e06097e8827d6ce264353: Status 404 returned error can't find the container with id ab353fbe572b042d93baceb16cd74eb90d26f1cb6b9e06097e8827d6ce264353
Aug 04 18:02:33.155706 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:33.155656    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp" event={"ID":"336edf09-c429-4f98-a313-edd0af9ace0f","Type":"ContainerStarted","Data":"e984db83c57e9853348ebc4ea8e4649c50d1da220eea1aaa14f60dd2a52f99a0"}
Aug 04 18:02:33.156169 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:33.155717    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp" event={"ID":"336edf09-c429-4f98-a313-edd0af9ace0f","Type":"ContainerStarted","Data":"ab353fbe572b042d93baceb16cd74eb90d26f1cb6b9e06097e8827d6ce264353"}
Aug 04 18:02:33.220908 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:33.220854    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp" podStartSLOduration=1.992226523 podStartE2EDuration="2.220836278s" totalImagesPullingTime="228.609755ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:02:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:02:33.206827541 +0000 UTC m=+635.389987716" watchObservedRunningTime="2026-08-04 18:02:33.220836278 +0000 UTC m=+635.403996455"
Aug 04 18:02:42.241416 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:42.241379    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp"
Aug 04 18:02:42.244038 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:42.241448    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp"
Aug 04 18:02:42.254167 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:42.254144    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp"
Aug 04 18:02:43.214182 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:43.214154    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zclgp"
Aug 04 18:02:44.436227 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:44.436198    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w"]
Aug 04 18:02:44.439871 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:44.439850    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w"
Aug 04 18:02:44.447016 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:44.446977    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Aug 04 18:02:44.460671 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:44.460646    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w"]
Aug 04 18:02:44.475207 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:44.475163    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0d794f7-bbdb-4dba-b945-61b1f8535542-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4rt8w\" (UID: \"d0d794f7-bbdb-4dba-b945-61b1f8535542\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w"
Aug 04 18:02:44.475311 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:44.475206    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d662x\" (UniqueName: \"kubernetes.io/projected/d0d794f7-bbdb-4dba-b945-61b1f8535542-kube-api-access-d662x\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4rt8w\" (UID: \"d0d794f7-bbdb-4dba-b945-61b1f8535542\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w"
Aug 04 18:02:44.475311 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:44.475270    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0d794f7-bbdb-4dba-b945-61b1f8535542-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4rt8w\" (UID: \"d0d794f7-bbdb-4dba-b945-61b1f8535542\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w"
Aug 04 18:02:44.475399 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:44.475332    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0d794f7-bbdb-4dba-b945-61b1f8535542-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4rt8w\" (UID: \"d0d794f7-bbdb-4dba-b945-61b1f8535542\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w"
Aug 04 18:02:44.475399 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:44.475347    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0d794f7-bbdb-4dba-b945-61b1f8535542-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4rt8w\" (UID: \"d0d794f7-bbdb-4dba-b945-61b1f8535542\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w"
Aug 04 18:02:44.475476 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:44.475460    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0d794f7-bbdb-4dba-b945-61b1f8535542-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4rt8w\" (UID: \"d0d794f7-bbdb-4dba-b945-61b1f8535542\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w"
Aug 04 18:02:44.576460 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:44.576426    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0d794f7-bbdb-4dba-b945-61b1f8535542-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4rt8w\" (UID: \"d0d794f7-bbdb-4dba-b945-61b1f8535542\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w"
Aug 04 18:02:44.576572 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:44.576495    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0d794f7-bbdb-4dba-b945-61b1f8535542-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4rt8w\" (UID: \"d0d794f7-bbdb-4dba-b945-61b1f8535542\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w"
Aug 04 18:02:44.576572 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:44.576518    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d662x\" (UniqueName: \"kubernetes.io/projected/d0d794f7-bbdb-4dba-b945-61b1f8535542-kube-api-access-d662x\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4rt8w\" (UID: \"d0d794f7-bbdb-4dba-b945-61b1f8535542\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w"
Aug 04 18:02:44.576572 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:44.576543    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0d794f7-bbdb-4dba-b945-61b1f8535542-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4rt8w\" (UID: \"d0d794f7-bbdb-4dba-b945-61b1f8535542\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w"
Aug 04 18:02:44.576723 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:44.576594    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0d794f7-bbdb-4dba-b945-61b1f8535542-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4rt8w\" (UID: \"d0d794f7-bbdb-4dba-b945-61b1f8535542\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w"
Aug 04 18:02:44.576723 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:44.576615    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0d794f7-bbdb-4dba-b945-61b1f8535542-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4rt8w\" (UID: \"d0d794f7-bbdb-4dba-b945-61b1f8535542\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w"
Aug 04 18:02:44.576825 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:44.576749    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0d794f7-bbdb-4dba-b945-61b1f8535542-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4rt8w\" (UID: \"d0d794f7-bbdb-4dba-b945-61b1f8535542\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w"
Aug 04 18:02:44.576907 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:44.576884    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0d794f7-bbdb-4dba-b945-61b1f8535542-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4rt8w\" (UID: \"d0d794f7-bbdb-4dba-b945-61b1f8535542\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w"
Aug 04 18:02:44.577026 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:44.576972    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0d794f7-bbdb-4dba-b945-61b1f8535542-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4rt8w\" (UID: \"d0d794f7-bbdb-4dba-b945-61b1f8535542\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w"
Aug 04 18:02:44.578927 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:44.578907    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0d794f7-bbdb-4dba-b945-61b1f8535542-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4rt8w\" (UID: \"d0d794f7-bbdb-4dba-b945-61b1f8535542\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w"
Aug 04 18:02:44.579097 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:44.579081    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0d794f7-bbdb-4dba-b945-61b1f8535542-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4rt8w\" (UID: \"d0d794f7-bbdb-4dba-b945-61b1f8535542\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w"
Aug 04 18:02:44.589240 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:44.589208    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d662x\" (UniqueName: \"kubernetes.io/projected/d0d794f7-bbdb-4dba-b945-61b1f8535542-kube-api-access-d662x\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4rt8w\" (UID: \"d0d794f7-bbdb-4dba-b945-61b1f8535542\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w"
Aug 04 18:02:44.750352 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:44.750297    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w"
Aug 04 18:02:44.897138 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:44.897108    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w"]
Aug 04 18:02:44.900470 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:02:44.900443    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0d794f7_bbdb_4dba_b945_61b1f8535542.slice/crio-9713a40fcba1c91bf4760a70839990960170f6e954e0d1307c3a18dacd90ba06 WatchSource:0}: Error finding container 9713a40fcba1c91bf4760a70839990960170f6e954e0d1307c3a18dacd90ba06: Status 404 returned error can't find the container with id 9713a40fcba1c91bf4760a70839990960170f6e954e0d1307c3a18dacd90ba06
Aug 04 18:02:45.213882 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:45.213845    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w" event={"ID":"d0d794f7-bbdb-4dba-b945-61b1f8535542","Type":"ContainerStarted","Data":"9713a40fcba1c91bf4760a70839990960170f6e954e0d1307c3a18dacd90ba06"}
Aug 04 18:02:46.219825 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:46.219789    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w" event={"ID":"d0d794f7-bbdb-4dba-b945-61b1f8535542","Type":"ContainerStarted","Data":"d4a157557815133d6f8ab4837d95f55c7ae51f892476ab8012400425a12e8ca1"}
Aug 04 18:02:46.246970 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:46.246928    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w" podStartSLOduration=1.981450323 podStartE2EDuration="2.246914197s" totalImagesPullingTime="265.463874ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:02:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:02:46.244031048 +0000 UTC m=+648.427191225" watchObservedRunningTime="2026-08-04 18:02:46.246914197 +0000 UTC m=+648.430074351"
Aug 04 18:02:54.751309 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:54.751278    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w"
Aug 04 18:02:54.751782 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:54.751333    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w"
Aug 04 18:02:54.764111 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:54.764091    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w"
Aug 04 18:02:55.269468 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:02:55.269444    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4rt8w"
Aug 04 18:04:12.194978 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:12.194948    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-54dcfb4cb7-5lwnq"]
Aug 04 18:04:12.198241 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:12.198224    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54dcfb4cb7-5lwnq"
Aug 04 18:04:12.209294 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:12.209274    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54dcfb4cb7-5lwnq"]
Aug 04 18:04:12.255726 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:12.255704    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65mhp\" (UniqueName: \"kubernetes.io/projected/4cda36d4-0d07-4bf9-a53d-64d18263c5fd-kube-api-access-65mhp\") pod \"authorino-54dcfb4cb7-5lwnq\" (UID: \"4cda36d4-0d07-4bf9-a53d-64d18263c5fd\") " pod="kuadrant-system/authorino-54dcfb4cb7-5lwnq"
Aug 04 18:04:12.255824 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:12.255737    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4cda36d4-0d07-4bf9-a53d-64d18263c5fd-tls-cert\") pod \"authorino-54dcfb4cb7-5lwnq\" (UID: \"4cda36d4-0d07-4bf9-a53d-64d18263c5fd\") " pod="kuadrant-system/authorino-54dcfb4cb7-5lwnq"
Aug 04 18:04:12.356606 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:12.356580    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65mhp\" (UniqueName: \"kubernetes.io/projected/4cda36d4-0d07-4bf9-a53d-64d18263c5fd-kube-api-access-65mhp\") pod \"authorino-54dcfb4cb7-5lwnq\" (UID: \"4cda36d4-0d07-4bf9-a53d-64d18263c5fd\") " pod="kuadrant-system/authorino-54dcfb4cb7-5lwnq"
Aug 04 18:04:12.356740 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:12.356613    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4cda36d4-0d07-4bf9-a53d-64d18263c5fd-tls-cert\") pod \"authorino-54dcfb4cb7-5lwnq\" (UID: \"4cda36d4-0d07-4bf9-a53d-64d18263c5fd\") " pod="kuadrant-system/authorino-54dcfb4cb7-5lwnq"
Aug 04 18:04:12.359234 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:12.359210    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4cda36d4-0d07-4bf9-a53d-64d18263c5fd-tls-cert\") pod \"authorino-54dcfb4cb7-5lwnq\" (UID: \"4cda36d4-0d07-4bf9-a53d-64d18263c5fd\") " pod="kuadrant-system/authorino-54dcfb4cb7-5lwnq"
Aug 04 18:04:12.369700 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:12.369677    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65mhp\" (UniqueName: \"kubernetes.io/projected/4cda36d4-0d07-4bf9-a53d-64d18263c5fd-kube-api-access-65mhp\") pod \"authorino-54dcfb4cb7-5lwnq\" (UID: \"4cda36d4-0d07-4bf9-a53d-64d18263c5fd\") " pod="kuadrant-system/authorino-54dcfb4cb7-5lwnq"
Aug 04 18:04:12.506814 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:12.506739    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54dcfb4cb7-5lwnq"
Aug 04 18:04:12.638175 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:12.638133    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54dcfb4cb7-5lwnq"]
Aug 04 18:04:12.643263 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:12.643241    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 18:04:13.623936 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:13.623903    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54dcfb4cb7-5lwnq" event={"ID":"4cda36d4-0d07-4bf9-a53d-64d18263c5fd","Type":"ContainerStarted","Data":"a9581f9527f203b5c033ca7a00cfb2ed25402696a1eb4de50892beaae8bfe924"}
Aug 04 18:04:15.642468 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:15.642429    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54dcfb4cb7-5lwnq" event={"ID":"4cda36d4-0d07-4bf9-a53d-64d18263c5fd","Type":"ContainerStarted","Data":"a74ea36319a4e010dd81dbb11ff70e9718e7069cc63974c0b4c48725a068daf5"}
Aug 04 18:04:15.679894 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:15.679845    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-54dcfb4cb7-5lwnq" podStartSLOduration=1.306172821 podStartE2EDuration="3.679828638s" totalImagesPullingTime="2.373655817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:04:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:04:15.677294509 +0000 UTC m=+737.860454688" watchObservedRunningTime="2026-08-04 18:04:15.679828638 +0000 UTC m=+737.862988813"
Aug 04 18:04:15.734838 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:15.734804    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59b65697db-twb97"]
Aug 04 18:04:15.735112 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:15.735084    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59b65697db-twb97" podUID="d91129c1-d186-48d5-b15c-088f2c747814" containerName="authorino" containerID="cri-o://c9b65b9369e0d4b96c8c67e143e0fab9693fd4fa9a405803fd3d65b4ccdca2d5" gracePeriod=30
Aug 04 18:04:16.001671 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:16.001644    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59b65697db-twb97"
Aug 04 18:04:16.086307 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:16.086274    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d91129c1-d186-48d5-b15c-088f2c747814-tls-cert\") pod \"d91129c1-d186-48d5-b15c-088f2c747814\" (UID: \"d91129c1-d186-48d5-b15c-088f2c747814\") "
Aug 04 18:04:16.086493 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:16.086326    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8rj5\" (UniqueName: \"kubernetes.io/projected/d91129c1-d186-48d5-b15c-088f2c747814-kube-api-access-h8rj5\") pod \"d91129c1-d186-48d5-b15c-088f2c747814\" (UID: \"d91129c1-d186-48d5-b15c-088f2c747814\") "
Aug 04 18:04:16.088859 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:16.088829    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d91129c1-d186-48d5-b15c-088f2c747814-kube-api-access-h8rj5" (OuterVolumeSpecName: "kube-api-access-h8rj5") pod "d91129c1-d186-48d5-b15c-088f2c747814" (UID: "d91129c1-d186-48d5-b15c-088f2c747814"). InnerVolumeSpecName "kube-api-access-h8rj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:04:16.098276 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:16.098249    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d91129c1-d186-48d5-b15c-088f2c747814-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "d91129c1-d186-48d5-b15c-088f2c747814" (UID: "d91129c1-d186-48d5-b15c-088f2c747814"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:04:16.187258 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:16.187222    2560 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d91129c1-d186-48d5-b15c-088f2c747814-tls-cert\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:04:16.187258 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:16.187252    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8rj5\" (UniqueName: \"kubernetes.io/projected/d91129c1-d186-48d5-b15c-088f2c747814-kube-api-access-h8rj5\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:04:16.649215 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:16.649117    2560 generic.go:356] "Generic (PLEG): container finished" podID="d91129c1-d186-48d5-b15c-088f2c747814" containerID="c9b65b9369e0d4b96c8c67e143e0fab9693fd4fa9a405803fd3d65b4ccdca2d5" exitCode=0
Aug 04 18:04:16.649215 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:16.649158    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59b65697db-twb97"
Aug 04 18:04:16.649215 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:16.649195    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59b65697db-twb97" event={"ID":"d91129c1-d186-48d5-b15c-088f2c747814","Type":"ContainerDied","Data":"c9b65b9369e0d4b96c8c67e143e0fab9693fd4fa9a405803fd3d65b4ccdca2d5"}
Aug 04 18:04:16.649810 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:16.649240    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59b65697db-twb97" event={"ID":"d91129c1-d186-48d5-b15c-088f2c747814","Type":"ContainerDied","Data":"8108bd082d0e4d49ed1b6b25fc685c3cd7c6cc428e9ce8471423bd18c5daba4a"}
Aug 04 18:04:16.649810 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:16.649261    2560 scope.go:117] "RemoveContainer" containerID="c9b65b9369e0d4b96c8c67e143e0fab9693fd4fa9a405803fd3d65b4ccdca2d5"
Aug 04 18:04:16.659048 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:16.659025    2560 scope.go:117] "RemoveContainer" containerID="c9b65b9369e0d4b96c8c67e143e0fab9693fd4fa9a405803fd3d65b4ccdca2d5"
Aug 04 18:04:16.659331 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:04:16.659313    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9b65b9369e0d4b96c8c67e143e0fab9693fd4fa9a405803fd3d65b4ccdca2d5\": container with ID starting with c9b65b9369e0d4b96c8c67e143e0fab9693fd4fa9a405803fd3d65b4ccdca2d5 not found: ID does not exist" containerID="c9b65b9369e0d4b96c8c67e143e0fab9693fd4fa9a405803fd3d65b4ccdca2d5"
Aug 04 18:04:16.659398 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:16.659342    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9b65b9369e0d4b96c8c67e143e0fab9693fd4fa9a405803fd3d65b4ccdca2d5"} err="failed to get container status \"c9b65b9369e0d4b96c8c67e143e0fab9693fd4fa9a405803fd3d65b4ccdca2d5\": rpc error: code = NotFound desc = could not find container \"c9b65b9369e0d4b96c8c67e143e0fab9693fd4fa9a405803fd3d65b4ccdca2d5\": container with ID starting with c9b65b9369e0d4b96c8c67e143e0fab9693fd4fa9a405803fd3d65b4ccdca2d5 not found: ID does not exist"
Aug 04 18:04:16.762953 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:16.762919    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59b65697db-twb97"]
Aug 04 18:04:16.764811 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:16.764783    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59b65697db-twb97"]
Aug 04 18:04:18.354268 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:04:18.354226    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d91129c1-d186-48d5-b15c-088f2c747814" path="/var/lib/kubelet/pods/d91129c1-d186-48d5-b15c-088f2c747814/volumes"
Aug 04 18:05:05.113716 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:05:05.113677    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57c8d79cc7-xkmx9"]
Aug 04 18:05:05.114282 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:05:05.113926    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-57c8d79cc7-xkmx9" podUID="23539736-56e8-4000-afd2-9ea4fce9a012" containerName="manager" containerID="cri-o://edbb1d6059d60cada1391baddf05af02270619edab6c3cdd41329ed760582933" gracePeriod=30
Aug 04 18:05:05.372705 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:05:05.372651    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57c8d79cc7-xkmx9"
Aug 04 18:05:05.412258 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:05:05.412237    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ww7t\" (UniqueName: \"kubernetes.io/projected/23539736-56e8-4000-afd2-9ea4fce9a012-kube-api-access-2ww7t\") pod \"23539736-56e8-4000-afd2-9ea4fce9a012\" (UID: \"23539736-56e8-4000-afd2-9ea4fce9a012\") "
Aug 04 18:05:05.412400 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:05:05.412341    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/23539736-56e8-4000-afd2-9ea4fce9a012-webhook-cert\") pod \"23539736-56e8-4000-afd2-9ea4fce9a012\" (UID: \"23539736-56e8-4000-afd2-9ea4fce9a012\") "
Aug 04 18:05:05.414454 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:05:05.414430    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23539736-56e8-4000-afd2-9ea4fce9a012-kube-api-access-2ww7t" (OuterVolumeSpecName: "kube-api-access-2ww7t") pod "23539736-56e8-4000-afd2-9ea4fce9a012" (UID: "23539736-56e8-4000-afd2-9ea4fce9a012"). InnerVolumeSpecName "kube-api-access-2ww7t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:05:05.414609 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:05:05.414595    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23539736-56e8-4000-afd2-9ea4fce9a012-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "23539736-56e8-4000-afd2-9ea4fce9a012" (UID: "23539736-56e8-4000-afd2-9ea4fce9a012"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:05:05.513885 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:05:05.513853    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ww7t\" (UniqueName: \"kubernetes.io/projected/23539736-56e8-4000-afd2-9ea4fce9a012-kube-api-access-2ww7t\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:05:05.514025 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:05:05.513890    2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/23539736-56e8-4000-afd2-9ea4fce9a012-webhook-cert\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:05:05.900521 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:05:05.900488    2560 generic.go:356] "Generic (PLEG): container finished" podID="23539736-56e8-4000-afd2-9ea4fce9a012" containerID="edbb1d6059d60cada1391baddf05af02270619edab6c3cdd41329ed760582933" exitCode=0
Aug 04 18:05:05.900710 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:05:05.900534    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57c8d79cc7-xkmx9"
Aug 04 18:05:05.900710 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:05:05.900575    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57c8d79cc7-xkmx9" event={"ID":"23539736-56e8-4000-afd2-9ea4fce9a012","Type":"ContainerDied","Data":"edbb1d6059d60cada1391baddf05af02270619edab6c3cdd41329ed760582933"}
Aug 04 18:05:05.900710 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:05:05.900641    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57c8d79cc7-xkmx9" event={"ID":"23539736-56e8-4000-afd2-9ea4fce9a012","Type":"ContainerDied","Data":"ff5e532d3de8984169a4d858adacc9b112212ce441c6f988206630a550ddd69d"}
Aug 04 18:05:05.900710 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:05:05.900671    2560 scope.go:117] "RemoveContainer" containerID="edbb1d6059d60cada1391baddf05af02270619edab6c3cdd41329ed760582933"
Aug 04 18:05:05.909674 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:05:05.909658    2560 scope.go:117] "RemoveContainer" containerID="edbb1d6059d60cada1391baddf05af02270619edab6c3cdd41329ed760582933"
Aug 04 18:05:05.909907 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:05:05.909892    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edbb1d6059d60cada1391baddf05af02270619edab6c3cdd41329ed760582933\": container with ID starting with edbb1d6059d60cada1391baddf05af02270619edab6c3cdd41329ed760582933 not found: ID does not exist" containerID="edbb1d6059d60cada1391baddf05af02270619edab6c3cdd41329ed760582933"
Aug 04 18:05:05.909973 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:05:05.909912    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edbb1d6059d60cada1391baddf05af02270619edab6c3cdd41329ed760582933"} err="failed to get container status \"edbb1d6059d60cada1391baddf05af02270619edab6c3cdd41329ed760582933\": rpc error: code = NotFound desc = could not find container \"edbb1d6059d60cada1391baddf05af02270619edab6c3cdd41329ed760582933\": container with ID starting with edbb1d6059d60cada1391baddf05af02270619edab6c3cdd41329ed760582933 not found: ID does not exist"
Aug 04 18:05:05.926018 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:05:05.925975    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57c8d79cc7-xkmx9"]
Aug 04 18:05:05.930112 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:05:05.930087    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-57c8d79cc7-xkmx9"]
Aug 04 18:05:06.353701 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:05:06.353666    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23539736-56e8-4000-afd2-9ea4fce9a012" path="/var/lib/kubelet/pods/23539736-56e8-4000-afd2-9ea4fce9a012/volumes"
Aug 04 18:06:06.815022 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:06.814971    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-57c8d79cc7-pwvpk"]
Aug 04 18:06:06.815453 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:06.815392    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d91129c1-d186-48d5-b15c-088f2c747814" containerName="authorino"
Aug 04 18:06:06.815453 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:06.815405    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d91129c1-d186-48d5-b15c-088f2c747814" containerName="authorino"
Aug 04 18:06:06.815453 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:06.815413    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23539736-56e8-4000-afd2-9ea4fce9a012" containerName="manager"
Aug 04 18:06:06.815453 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:06.815419    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="23539736-56e8-4000-afd2-9ea4fce9a012" containerName="manager"
Aug 04 18:06:06.815585 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:06.815486    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d91129c1-d186-48d5-b15c-088f2c747814" containerName="authorino"
Aug 04 18:06:06.815585 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:06.815498    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="23539736-56e8-4000-afd2-9ea4fce9a012" containerName="manager"
Aug 04 18:06:06.819627 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:06.819610    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57c8d79cc7-pwvpk"
Aug 04 18:06:06.825182 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:06.825163    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 04 18:06:06.826171 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:06.826153    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 04 18:06:06.826259 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:06.826193    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-lmgls\""
Aug 04 18:06:06.835528 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:06.835507    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57c8d79cc7-pwvpk"]
Aug 04 18:06:06.996221 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:06.996192    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phk9j\" (UniqueName: \"kubernetes.io/projected/a606d0f1-46f8-4347-9eda-327a5d782ec1-kube-api-access-phk9j\") pod \"maas-controller-57c8d79cc7-pwvpk\" (UID: \"a606d0f1-46f8-4347-9eda-327a5d782ec1\") " pod="opendatahub/maas-controller-57c8d79cc7-pwvpk"
Aug 04 18:06:06.996352 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:06.996269    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a606d0f1-46f8-4347-9eda-327a5d782ec1-webhook-cert\") pod \"maas-controller-57c8d79cc7-pwvpk\" (UID: \"a606d0f1-46f8-4347-9eda-327a5d782ec1\") " pod="opendatahub/maas-controller-57c8d79cc7-pwvpk"
Aug 04 18:06:07.096913 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:07.096844    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a606d0f1-46f8-4347-9eda-327a5d782ec1-webhook-cert\") pod \"maas-controller-57c8d79cc7-pwvpk\" (UID: \"a606d0f1-46f8-4347-9eda-327a5d782ec1\") " pod="opendatahub/maas-controller-57c8d79cc7-pwvpk"
Aug 04 18:06:07.096913 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:07.096892    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phk9j\" (UniqueName: \"kubernetes.io/projected/a606d0f1-46f8-4347-9eda-327a5d782ec1-kube-api-access-phk9j\") pod \"maas-controller-57c8d79cc7-pwvpk\" (UID: \"a606d0f1-46f8-4347-9eda-327a5d782ec1\") " pod="opendatahub/maas-controller-57c8d79cc7-pwvpk"
Aug 04 18:06:07.099569 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:07.099548    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a606d0f1-46f8-4347-9eda-327a5d782ec1-webhook-cert\") pod \"maas-controller-57c8d79cc7-pwvpk\" (UID: \"a606d0f1-46f8-4347-9eda-327a5d782ec1\") " pod="opendatahub/maas-controller-57c8d79cc7-pwvpk"
Aug 04 18:06:07.107685 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:07.107660    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phk9j\" (UniqueName: \"kubernetes.io/projected/a606d0f1-46f8-4347-9eda-327a5d782ec1-kube-api-access-phk9j\") pod \"maas-controller-57c8d79cc7-pwvpk\" (UID: \"a606d0f1-46f8-4347-9eda-327a5d782ec1\") " pod="opendatahub/maas-controller-57c8d79cc7-pwvpk"
Aug 04 18:06:07.129132 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:07.129107    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57c8d79cc7-pwvpk"
Aug 04 18:06:07.268129 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:07.268104    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57c8d79cc7-pwvpk"]
Aug 04 18:06:07.269551 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:06:07.269529    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda606d0f1_46f8_4347_9eda_327a5d782ec1.slice/crio-879a84eb4f7d05d8206866b973ad58370af2d04b23608ea609f3fb4d4dff1cf4 WatchSource:0}: Error finding container 879a84eb4f7d05d8206866b973ad58370af2d04b23608ea609f3fb4d4dff1cf4: Status 404 returned error can't find the container with id 879a84eb4f7d05d8206866b973ad58370af2d04b23608ea609f3fb4d4dff1cf4
Aug 04 18:06:08.191824 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:08.191795    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57c8d79cc7-pwvpk" event={"ID":"a606d0f1-46f8-4347-9eda-327a5d782ec1","Type":"ContainerStarted","Data":"85742f09942fdf8c468dcd3589af8d2e7634b5e8515a28ddc7f510d5110f182a"}
Aug 04 18:06:08.192340 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:08.191844    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57c8d79cc7-pwvpk" event={"ID":"a606d0f1-46f8-4347-9eda-327a5d782ec1","Type":"ContainerStarted","Data":"879a84eb4f7d05d8206866b973ad58370af2d04b23608ea609f3fb4d4dff1cf4"}
Aug 04 18:06:08.192340 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:08.191884    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-57c8d79cc7-pwvpk"
Aug 04 18:06:08.215693 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:08.215653    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-57c8d79cc7-pwvpk" podStartSLOduration=1.755931167 podStartE2EDuration="2.215640139s" totalImagesPullingTime="459.708972ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:06:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:06:08.213762817 +0000 UTC m=+850.396922992" watchObservedRunningTime="2026-08-04 18:06:08.215640139 +0000 UTC m=+850.398800314"
Aug 04 18:06:19.198807 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:19.198778    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-57c8d79cc7-pwvpk"
Aug 04 18:06:23.837537 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:23.837506    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57c8d79cc7-pwvpk"]
Aug 04 18:06:23.837953 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:23.837746    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-57c8d79cc7-pwvpk" podUID="a606d0f1-46f8-4347-9eda-327a5d782ec1" containerName="manager" containerID="cri-o://85742f09942fdf8c468dcd3589af8d2e7634b5e8515a28ddc7f510d5110f182a" gracePeriod=30
Aug 04 18:06:24.088163 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:24.088111    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57c8d79cc7-pwvpk"
Aug 04 18:06:24.140907 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:24.140884    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a606d0f1-46f8-4347-9eda-327a5d782ec1-webhook-cert\") pod \"a606d0f1-46f8-4347-9eda-327a5d782ec1\" (UID: \"a606d0f1-46f8-4347-9eda-327a5d782ec1\") "
Aug 04 18:06:24.141082 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:24.140949    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phk9j\" (UniqueName: \"kubernetes.io/projected/a606d0f1-46f8-4347-9eda-327a5d782ec1-kube-api-access-phk9j\") pod \"a606d0f1-46f8-4347-9eda-327a5d782ec1\" (UID: \"a606d0f1-46f8-4347-9eda-327a5d782ec1\") "
Aug 04 18:06:24.143129 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:24.143099    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a606d0f1-46f8-4347-9eda-327a5d782ec1-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a606d0f1-46f8-4347-9eda-327a5d782ec1" (UID: "a606d0f1-46f8-4347-9eda-327a5d782ec1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:06:24.143242 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:24.143102    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a606d0f1-46f8-4347-9eda-327a5d782ec1-kube-api-access-phk9j" (OuterVolumeSpecName: "kube-api-access-phk9j") pod "a606d0f1-46f8-4347-9eda-327a5d782ec1" (UID: "a606d0f1-46f8-4347-9eda-327a5d782ec1"). InnerVolumeSpecName "kube-api-access-phk9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:06:24.241881 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:24.241855    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phk9j\" (UniqueName: \"kubernetes.io/projected/a606d0f1-46f8-4347-9eda-327a5d782ec1-kube-api-access-phk9j\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:06:24.241881 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:24.241877    2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a606d0f1-46f8-4347-9eda-327a5d782ec1-webhook-cert\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:06:24.274914 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:24.274882    2560 generic.go:356] "Generic (PLEG): container finished" podID="a606d0f1-46f8-4347-9eda-327a5d782ec1" containerID="85742f09942fdf8c468dcd3589af8d2e7634b5e8515a28ddc7f510d5110f182a" exitCode=0
Aug 04 18:06:24.275055 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:24.274938    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57c8d79cc7-pwvpk"
Aug 04 18:06:24.275055 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:24.274973    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57c8d79cc7-pwvpk" event={"ID":"a606d0f1-46f8-4347-9eda-327a5d782ec1","Type":"ContainerDied","Data":"85742f09942fdf8c468dcd3589af8d2e7634b5e8515a28ddc7f510d5110f182a"}
Aug 04 18:06:24.275055 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:24.275045    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57c8d79cc7-pwvpk" event={"ID":"a606d0f1-46f8-4347-9eda-327a5d782ec1","Type":"ContainerDied","Data":"879a84eb4f7d05d8206866b973ad58370af2d04b23608ea609f3fb4d4dff1cf4"}
Aug 04 18:06:24.275216 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:24.275068    2560 scope.go:117] "RemoveContainer" containerID="85742f09942fdf8c468dcd3589af8d2e7634b5e8515a28ddc7f510d5110f182a"
Aug 04 18:06:24.284036 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:24.284020    2560 scope.go:117] "RemoveContainer" containerID="85742f09942fdf8c468dcd3589af8d2e7634b5e8515a28ddc7f510d5110f182a"
Aug 04 18:06:24.284290 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:06:24.284274    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85742f09942fdf8c468dcd3589af8d2e7634b5e8515a28ddc7f510d5110f182a\": container with ID starting with 85742f09942fdf8c468dcd3589af8d2e7634b5e8515a28ddc7f510d5110f182a not found: ID does not exist" containerID="85742f09942fdf8c468dcd3589af8d2e7634b5e8515a28ddc7f510d5110f182a"
Aug 04 18:06:24.284350 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:24.284297    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85742f09942fdf8c468dcd3589af8d2e7634b5e8515a28ddc7f510d5110f182a"} err="failed to get container status \"85742f09942fdf8c468dcd3589af8d2e7634b5e8515a28ddc7f510d5110f182a\": rpc error: code = NotFound desc = could not find container \"85742f09942fdf8c468dcd3589af8d2e7634b5e8515a28ddc7f510d5110f182a\": container with ID starting with 85742f09942fdf8c468dcd3589af8d2e7634b5e8515a28ddc7f510d5110f182a not found: ID does not exist"
Aug 04 18:06:24.296692 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:24.296669    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57c8d79cc7-pwvpk"]
Aug 04 18:06:24.302511 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:24.302493    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-57c8d79cc7-pwvpk"]
Aug 04 18:06:24.353577 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:24.353529    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a606d0f1-46f8-4347-9eda-327a5d782ec1" path="/var/lib/kubelet/pods/a606d0f1-46f8-4347-9eda-327a5d782ec1/volumes"
Aug 04 18:06:25.107197 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:25.107168    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-57c8d79cc7-97k8r"]
Aug 04 18:06:25.107639 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:25.107527    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a606d0f1-46f8-4347-9eda-327a5d782ec1" containerName="manager"
Aug 04 18:06:25.107639 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:25.107537    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a606d0f1-46f8-4347-9eda-327a5d782ec1" containerName="manager"
Aug 04 18:06:25.107639 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:25.107601    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a606d0f1-46f8-4347-9eda-327a5d782ec1" containerName="manager"
Aug 04 18:06:25.111800 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:25.111782    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57c8d79cc7-97k8r"
Aug 04 18:06:25.114293 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:25.114273    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 04 18:06:25.115017 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:25.115002    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-lmgls\""
Aug 04 18:06:25.115098 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:25.115065    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 04 18:06:25.121079 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:25.121062    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57c8d79cc7-97k8r"]
Aug 04 18:06:25.149220 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:25.149197    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqnbm\" (UniqueName: \"kubernetes.io/projected/76b5a999-8b81-45a0-9663-9daabaca87fa-kube-api-access-hqnbm\") pod \"maas-controller-57c8d79cc7-97k8r\" (UID: \"76b5a999-8b81-45a0-9663-9daabaca87fa\") " pod="opendatahub/maas-controller-57c8d79cc7-97k8r"
Aug 04 18:06:25.149322 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:25.149241    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/76b5a999-8b81-45a0-9663-9daabaca87fa-webhook-cert\") pod \"maas-controller-57c8d79cc7-97k8r\" (UID: \"76b5a999-8b81-45a0-9663-9daabaca87fa\") " pod="opendatahub/maas-controller-57c8d79cc7-97k8r"
Aug 04 18:06:25.249842 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:25.249813    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqnbm\" (UniqueName: \"kubernetes.io/projected/76b5a999-8b81-45a0-9663-9daabaca87fa-kube-api-access-hqnbm\") pod \"maas-controller-57c8d79cc7-97k8r\" (UID: \"76b5a999-8b81-45a0-9663-9daabaca87fa\") " pod="opendatahub/maas-controller-57c8d79cc7-97k8r"
Aug 04 18:06:25.249974 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:25.249855    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/76b5a999-8b81-45a0-9663-9daabaca87fa-webhook-cert\") pod \"maas-controller-57c8d79cc7-97k8r\" (UID: \"76b5a999-8b81-45a0-9663-9daabaca87fa\") " pod="opendatahub/maas-controller-57c8d79cc7-97k8r"
Aug 04 18:06:25.252379 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:25.252353    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/76b5a999-8b81-45a0-9663-9daabaca87fa-webhook-cert\") pod \"maas-controller-57c8d79cc7-97k8r\" (UID: \"76b5a999-8b81-45a0-9663-9daabaca87fa\") " pod="opendatahub/maas-controller-57c8d79cc7-97k8r"
Aug 04 18:06:25.259774 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:25.259739    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqnbm\" (UniqueName: \"kubernetes.io/projected/76b5a999-8b81-45a0-9663-9daabaca87fa-kube-api-access-hqnbm\") pod \"maas-controller-57c8d79cc7-97k8r\" (UID: \"76b5a999-8b81-45a0-9663-9daabaca87fa\") " pod="opendatahub/maas-controller-57c8d79cc7-97k8r"
Aug 04 18:06:25.422713 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:25.422678    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57c8d79cc7-97k8r"
Aug 04 18:06:25.569354 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:25.569332    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57c8d79cc7-97k8r"]
Aug 04 18:06:25.571311 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:06:25.571278    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76b5a999_8b81_45a0_9663_9daabaca87fa.slice/crio-06c31ef8a745c91de266a9fbef319877753897a6e8ae3a431f46661057b71625 WatchSource:0}: Error finding container 06c31ef8a745c91de266a9fbef319877753897a6e8ae3a431f46661057b71625: Status 404 returned error can't find the container with id 06c31ef8a745c91de266a9fbef319877753897a6e8ae3a431f46661057b71625
Aug 04 18:06:26.286038 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:26.285958    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57c8d79cc7-97k8r" event={"ID":"76b5a999-8b81-45a0-9663-9daabaca87fa","Type":"ContainerStarted","Data":"e4ce1740c2fce74f1e28f58d8795dd058ab90623c8914afb8bfb857ddf5702a9"}
Aug 04 18:06:26.286392 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:26.286046    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57c8d79cc7-97k8r" event={"ID":"76b5a999-8b81-45a0-9663-9daabaca87fa","Type":"ContainerStarted","Data":"06c31ef8a745c91de266a9fbef319877753897a6e8ae3a431f46661057b71625"}
Aug 04 18:06:26.286392 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:26.286101    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-57c8d79cc7-97k8r"
Aug 04 18:06:26.313268 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:26.313206    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-57c8d79cc7-97k8r" podStartSLOduration=0.873965598 podStartE2EDuration="1.313184795s" totalImagesPullingTime="439.219197ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:06:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:06:26.310128901 +0000 UTC m=+868.493289087" watchObservedRunningTime="2026-08-04 18:06:26.313184795 +0000 UTC m=+868.496344972"
Aug 04 18:06:37.292760 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:37.292726    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-57c8d79cc7-97k8r"
Aug 04 18:06:58.311013 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:58.310915    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/2.log"
Aug 04 18:06:58.311522 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:58.311334    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/2.log"
Aug 04 18:06:58.317839 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:58.317806    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 18:06:58.318661 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:06:58.318645    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 18:11:58.342046 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:11:58.341939    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/2.log"
Aug 04 18:11:58.347969 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:11:58.347941    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 18:11:58.349305 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:11:58.349284    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/2.log"
Aug 04 18:11:58.356180 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:11:58.356161    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 18:16:58.389633 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:16:58.389524    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/2.log"
Aug 04 18:16:58.396434 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:16:58.396406    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/2.log"
Aug 04 18:16:58.396813 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:16:58.396797    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 18:16:58.404324 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:16:58.404298    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 18:17:25.284885 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:25.284807    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"]
Aug 04 18:17:25.288869 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:25.288840    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"
Aug 04 18:17:25.291635 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:25.291602    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-b751fcdc-kserve-self-signed-certs\""
Aug 04 18:17:25.300299 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:25.300275    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"]
Aug 04 18:17:25.371182 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:25.371153    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8522b204-e346-4378-bc3b-fb2e350d6a8c-dshm\") pod \"e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx\" (UID: \"8522b204-e346-4378-bc3b-fb2e350d6a8c\") " pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"
Aug 04 18:17:25.371313 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:25.371193    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8522b204-e346-4378-bc3b-fb2e350d6a8c-tmp-dir\") pod \"e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx\" (UID: \"8522b204-e346-4378-bc3b-fb2e350d6a8c\") " pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"
Aug 04 18:17:25.371313 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:25.371216    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk5tq\" (UniqueName: \"kubernetes.io/projected/8522b204-e346-4378-bc3b-fb2e350d6a8c-kube-api-access-hk5tq\") pod \"e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx\" (UID: \"8522b204-e346-4378-bc3b-fb2e350d6a8c\") " pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"
Aug 04 18:17:25.371313 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:25.371281    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8522b204-e346-4378-bc3b-fb2e350d6a8c-model-cache\") pod \"e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx\" (UID: \"8522b204-e346-4378-bc3b-fb2e350d6a8c\") " pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"
Aug 04 18:17:25.371461 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:25.371330    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8522b204-e346-4378-bc3b-fb2e350d6a8c-tls-certs\") pod \"e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx\" (UID: \"8522b204-e346-4378-bc3b-fb2e350d6a8c\") " pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"
Aug 04 18:17:25.371461 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:25.371359    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8522b204-e346-4378-bc3b-fb2e350d6a8c-home\") pod \"e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx\" (UID: \"8522b204-e346-4378-bc3b-fb2e350d6a8c\") " pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"
Aug 04 18:17:25.471884 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:25.471853    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8522b204-e346-4378-bc3b-fb2e350d6a8c-dshm\") pod \"e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx\" (UID: \"8522b204-e346-4378-bc3b-fb2e350d6a8c\") " pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"
Aug 04 18:17:25.472054 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:25.471897    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8522b204-e346-4378-bc3b-fb2e350d6a8c-tmp-dir\") pod \"e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx\" (UID: \"8522b204-e346-4378-bc3b-fb2e350d6a8c\") " pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"
Aug 04 18:17:25.472054 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:25.471917    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk5tq\" (UniqueName: \"kubernetes.io/projected/8522b204-e346-4378-bc3b-fb2e350d6a8c-kube-api-access-hk5tq\") pod \"e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx\" (UID: \"8522b204-e346-4378-bc3b-fb2e350d6a8c\") " pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"
Aug 04 18:17:25.472054 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:25.471943    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8522b204-e346-4378-bc3b-fb2e350d6a8c-model-cache\") pod \"e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx\" (UID: \"8522b204-e346-4378-bc3b-fb2e350d6a8c\") " pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"
Aug 04 18:17:25.472214 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:25.472058    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8522b204-e346-4378-bc3b-fb2e350d6a8c-tls-certs\") pod \"e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx\" (UID: \"8522b204-e346-4378-bc3b-fb2e350d6a8c\") " pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"
Aug 04 18:17:25.472214 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:25.472111    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8522b204-e346-4378-bc3b-fb2e350d6a8c-home\") pod \"e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx\" (UID: \"8522b204-e346-4378-bc3b-fb2e350d6a8c\") " pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"
Aug 04 18:17:25.472327 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:25.472308    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8522b204-e346-4378-bc3b-fb2e350d6a8c-model-cache\") pod \"e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx\" (UID: \"8522b204-e346-4378-bc3b-fb2e350d6a8c\") " pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"
Aug 04 18:17:25.472386 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:25.472349    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8522b204-e346-4378-bc3b-fb2e350d6a8c-tmp-dir\") pod \"e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx\" (UID: \"8522b204-e346-4378-bc3b-fb2e350d6a8c\") " pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"
Aug 04 18:17:25.472386 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:25.472372    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8522b204-e346-4378-bc3b-fb2e350d6a8c-home\") pod \"e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx\" (UID: \"8522b204-e346-4378-bc3b-fb2e350d6a8c\") " pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"
Aug 04 18:17:25.474170 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:25.474144    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8522b204-e346-4378-bc3b-fb2e350d6a8c-dshm\") pod \"e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx\" (UID: \"8522b204-e346-4378-bc3b-fb2e350d6a8c\") " pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"
Aug 04 18:17:25.474550 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:25.474533    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8522b204-e346-4378-bc3b-fb2e350d6a8c-tls-certs\") pod \"e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx\" (UID: \"8522b204-e346-4378-bc3b-fb2e350d6a8c\") " pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"
Aug 04 18:17:25.490293 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:25.490271    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk5tq\" (UniqueName: \"kubernetes.io/projected/8522b204-e346-4378-bc3b-fb2e350d6a8c-kube-api-access-hk5tq\") pod \"e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx\" (UID: \"8522b204-e346-4378-bc3b-fb2e350d6a8c\") " pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"
Aug 04 18:17:25.599051 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:25.598972    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"
Aug 04 18:17:25.738383 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:25.738361    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"]
Aug 04 18:17:25.740950 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:17:25.740923    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8522b204_e346_4378_bc3b_fb2e350d6a8c.slice/crio-c8ccd6d4cd139a1d6a59725223cee8d86038f918abca9dc407654697646980ce WatchSource:0}: Error finding container c8ccd6d4cd139a1d6a59725223cee8d86038f918abca9dc407654697646980ce: Status 404 returned error can't find the container with id c8ccd6d4cd139a1d6a59725223cee8d86038f918abca9dc407654697646980ce
Aug 04 18:17:26.390657 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:26.390627    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx" event={"ID":"8522b204-e346-4378-bc3b-fb2e350d6a8c","Type":"ContainerStarted","Data":"d73658d28d3f5946ea7ee9ea8cd9e5ae533f05109eabc7058a9d113b09cbd3e6"}
Aug 04 18:17:26.390657 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:26.390665    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx" event={"ID":"8522b204-e346-4378-bc3b-fb2e350d6a8c","Type":"ContainerStarted","Data":"c8ccd6d4cd139a1d6a59725223cee8d86038f918abca9dc407654697646980ce"}
Aug 04 18:17:26.409519 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:26.409475    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx" podStartSLOduration=1.409460717 podStartE2EDuration="1.409460717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:17:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:17:26.408077946 +0000 UTC m=+1528.591238129" watchObservedRunningTime="2026-08-04 18:17:26.409460717 +0000 UTC m=+1528.592620893"
Aug 04 18:17:35.600067 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:35.600024    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"
Aug 04 18:17:35.600562 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:35.600083    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"
Aug 04 18:17:35.612789 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:35.612767    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"
Aug 04 18:17:36.446967 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:36.446942    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"
Aug 04 18:17:59.143512 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:17:59.143479    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"]
Aug 04 18:18:14.163174 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.163107    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx" podUID="8522b204-e346-4378-bc3b-fb2e350d6a8c" containerName="main" containerID="cri-o://d73658d28d3f5946ea7ee9ea8cd9e5ae533f05109eabc7058a9d113b09cbd3e6" gracePeriod=45
Aug 04 18:18:14.419826 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.419801    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"
Aug 04 18:18:14.581343 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.581314    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8522b204-e346-4378-bc3b-fb2e350d6a8c-model-cache\") pod \"8522b204-e346-4378-bc3b-fb2e350d6a8c\" (UID: \"8522b204-e346-4378-bc3b-fb2e350d6a8c\") "
Aug 04 18:18:14.581343 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.581348    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8522b204-e346-4378-bc3b-fb2e350d6a8c-home\") pod \"8522b204-e346-4378-bc3b-fb2e350d6a8c\" (UID: \"8522b204-e346-4378-bc3b-fb2e350d6a8c\") "
Aug 04 18:18:14.581591 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.581404    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8522b204-e346-4378-bc3b-fb2e350d6a8c-tmp-dir\") pod \"8522b204-e346-4378-bc3b-fb2e350d6a8c\" (UID: \"8522b204-e346-4378-bc3b-fb2e350d6a8c\") "
Aug 04 18:18:14.581591 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.581429    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8522b204-e346-4378-bc3b-fb2e350d6a8c-dshm\") pod \"8522b204-e346-4378-bc3b-fb2e350d6a8c\" (UID: \"8522b204-e346-4378-bc3b-fb2e350d6a8c\") "
Aug 04 18:18:14.581591 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.581445    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hk5tq\" (UniqueName: \"kubernetes.io/projected/8522b204-e346-4378-bc3b-fb2e350d6a8c-kube-api-access-hk5tq\") pod \"8522b204-e346-4378-bc3b-fb2e350d6a8c\" (UID: \"8522b204-e346-4378-bc3b-fb2e350d6a8c\") "
Aug 04 18:18:14.581591 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.581471    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8522b204-e346-4378-bc3b-fb2e350d6a8c-tls-certs\") pod \"8522b204-e346-4378-bc3b-fb2e350d6a8c\" (UID: \"8522b204-e346-4378-bc3b-fb2e350d6a8c\") "
Aug 04 18:18:14.581814 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.581614    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8522b204-e346-4378-bc3b-fb2e350d6a8c-model-cache" (OuterVolumeSpecName: "model-cache") pod "8522b204-e346-4378-bc3b-fb2e350d6a8c" (UID: "8522b204-e346-4378-bc3b-fb2e350d6a8c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:18:14.581814 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.581633    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8522b204-e346-4378-bc3b-fb2e350d6a8c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8522b204-e346-4378-bc3b-fb2e350d6a8c" (UID: "8522b204-e346-4378-bc3b-fb2e350d6a8c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:18:14.581814 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.581705    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8522b204-e346-4378-bc3b-fb2e350d6a8c-home" (OuterVolumeSpecName: "home") pod "8522b204-e346-4378-bc3b-fb2e350d6a8c" (UID: "8522b204-e346-4378-bc3b-fb2e350d6a8c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:18:14.581974 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.581857    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8522b204-e346-4378-bc3b-fb2e350d6a8c-model-cache\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:18:14.581974 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.581875    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8522b204-e346-4378-bc3b-fb2e350d6a8c-home\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:18:14.581974 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.581887    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8522b204-e346-4378-bc3b-fb2e350d6a8c-tmp-dir\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:18:14.583878 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.583845    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8522b204-e346-4378-bc3b-fb2e350d6a8c-dshm" (OuterVolumeSpecName: "dshm") pod "8522b204-e346-4378-bc3b-fb2e350d6a8c" (UID: "8522b204-e346-4378-bc3b-fb2e350d6a8c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:18:14.583878 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.583851    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8522b204-e346-4378-bc3b-fb2e350d6a8c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8522b204-e346-4378-bc3b-fb2e350d6a8c" (UID: "8522b204-e346-4378-bc3b-fb2e350d6a8c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:18:14.584067 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.583955    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8522b204-e346-4378-bc3b-fb2e350d6a8c-kube-api-access-hk5tq" (OuterVolumeSpecName: "kube-api-access-hk5tq") pod "8522b204-e346-4378-bc3b-fb2e350d6a8c" (UID: "8522b204-e346-4378-bc3b-fb2e350d6a8c"). InnerVolumeSpecName "kube-api-access-hk5tq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:18:14.616265 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.616237    2560 generic.go:356] "Generic (PLEG): container finished" podID="8522b204-e346-4378-bc3b-fb2e350d6a8c" containerID="d73658d28d3f5946ea7ee9ea8cd9e5ae533f05109eabc7058a9d113b09cbd3e6" exitCode=0
Aug 04 18:18:14.616376 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.616278    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"
Aug 04 18:18:14.616376 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.616326    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx" event={"ID":"8522b204-e346-4378-bc3b-fb2e350d6a8c","Type":"ContainerDied","Data":"d73658d28d3f5946ea7ee9ea8cd9e5ae533f05109eabc7058a9d113b09cbd3e6"}
Aug 04 18:18:14.616445 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.616377    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx" event={"ID":"8522b204-e346-4378-bc3b-fb2e350d6a8c","Type":"ContainerDied","Data":"c8ccd6d4cd139a1d6a59725223cee8d86038f918abca9dc407654697646980ce"}
Aug 04 18:18:14.616445 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.616395    2560 scope.go:117] "RemoveContainer" containerID="d73658d28d3f5946ea7ee9ea8cd9e5ae533f05109eabc7058a9d113b09cbd3e6"
Aug 04 18:18:14.625292 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.625271    2560 scope.go:117] "RemoveContainer" containerID="d73658d28d3f5946ea7ee9ea8cd9e5ae533f05109eabc7058a9d113b09cbd3e6"
Aug 04 18:18:14.625609 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:18:14.625522    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d73658d28d3f5946ea7ee9ea8cd9e5ae533f05109eabc7058a9d113b09cbd3e6\": container with ID starting with d73658d28d3f5946ea7ee9ea8cd9e5ae533f05109eabc7058a9d113b09cbd3e6 not found: ID does not exist" containerID="d73658d28d3f5946ea7ee9ea8cd9e5ae533f05109eabc7058a9d113b09cbd3e6"
Aug 04 18:18:14.625609 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.625552    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d73658d28d3f5946ea7ee9ea8cd9e5ae533f05109eabc7058a9d113b09cbd3e6"} err="failed to get container status \"d73658d28d3f5946ea7ee9ea8cd9e5ae533f05109eabc7058a9d113b09cbd3e6\": rpc error: code = NotFound desc = could not find container \"d73658d28d3f5946ea7ee9ea8cd9e5ae533f05109eabc7058a9d113b09cbd3e6\": container with ID starting with d73658d28d3f5946ea7ee9ea8cd9e5ae533f05109eabc7058a9d113b09cbd3e6 not found: ID does not exist"
Aug 04 18:18:14.643880 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.643856    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"]
Aug 04 18:18:14.645523 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.645504    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-b751fcdc-kserve-554dd5b687-bnvmx"]
Aug 04 18:18:14.682762 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.682740    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8522b204-e346-4378-bc3b-fb2e350d6a8c-dshm\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:18:14.682762 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.682760    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hk5tq\" (UniqueName: \"kubernetes.io/projected/8522b204-e346-4378-bc3b-fb2e350d6a8c-kube-api-access-hk5tq\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:18:14.682894 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:14.682770    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8522b204-e346-4378-bc3b-fb2e350d6a8c-tls-certs\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:18:16.353215 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:18:16.353178    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8522b204-e346-4378-bc3b-fb2e350d6a8c" path="/var/lib/kubelet/pods/8522b204-e346-4378-bc3b-fb2e350d6a8c/volumes"
Aug 04 18:21:58.419933 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:21:58.419902    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/2.log"
Aug 04 18:21:58.425760 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:21:58.425738    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 18:21:58.428274 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:21:58.428257    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/2.log"
Aug 04 18:21:58.432957 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:21:58.432943    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 18:22:06.615650 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:06.615620    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx"]
Aug 04 18:22:06.616097 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:06.616034    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8522b204-e346-4378-bc3b-fb2e350d6a8c" containerName="main"
Aug 04 18:22:06.616097 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:06.616047    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8522b204-e346-4378-bc3b-fb2e350d6a8c" containerName="main"
Aug 04 18:22:06.616223 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:06.616124    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8522b204-e346-4378-bc3b-fb2e350d6a8c" containerName="main"
Aug 04 18:22:06.619185 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:06.619165    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx"
Aug 04 18:22:06.621290 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:06.621270    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-745b1b91-serving-cert\""
Aug 04 18:22:06.633683 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:06.633654    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx"]
Aug 04 18:22:06.648794 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:06.648759    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e1d568d-dfa0-45e9-9e6b-6c33be9816e1-maas-api-tls\") pod \"maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx\" (UID: \"2e1d568d-dfa0-45e9-9e6b-6c33be9816e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx"
Aug 04 18:22:06.648919 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:06.648794    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpl7m\" (UniqueName: \"kubernetes.io/projected/2e1d568d-dfa0-45e9-9e6b-6c33be9816e1-kube-api-access-fpl7m\") pod \"maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx\" (UID: \"2e1d568d-dfa0-45e9-9e6b-6c33be9816e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx"
Aug 04 18:22:06.648919 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:06.648903    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e1d568d-dfa0-45e9-9e6b-6c33be9816e1-openshift-service-ca\") pod \"maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx\" (UID: \"2e1d568d-dfa0-45e9-9e6b-6c33be9816e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx"
Aug 04 18:22:06.749341 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:06.749313    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e1d568d-dfa0-45e9-9e6b-6c33be9816e1-openshift-service-ca\") pod \"maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx\" (UID: \"2e1d568d-dfa0-45e9-9e6b-6c33be9816e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx"
Aug 04 18:22:06.749502 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:06.749360    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e1d568d-dfa0-45e9-9e6b-6c33be9816e1-maas-api-tls\") pod \"maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx\" (UID: \"2e1d568d-dfa0-45e9-9e6b-6c33be9816e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx"
Aug 04 18:22:06.749502 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:06.749378    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpl7m\" (UniqueName: \"kubernetes.io/projected/2e1d568d-dfa0-45e9-9e6b-6c33be9816e1-kube-api-access-fpl7m\") pod \"maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx\" (UID: \"2e1d568d-dfa0-45e9-9e6b-6c33be9816e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx"
Aug 04 18:22:06.749502 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:22:06.749492    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-serving-cert: secret "maas-api-e2e-ait-745b1b91-serving-cert" not found
Aug 04 18:22:06.749672 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:22:06.749576    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e1d568d-dfa0-45e9-9e6b-6c33be9816e1-maas-api-tls podName:2e1d568d-dfa0-45e9-9e6b-6c33be9816e1 nodeName:}" failed. No retries permitted until 2026-08-04 18:22:07.249553547 +0000 UTC m=+1809.432713714 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2e1d568d-dfa0-45e9-9e6b-6c33be9816e1-maas-api-tls") pod "maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx" (UID: "2e1d568d-dfa0-45e9-9e6b-6c33be9816e1") : secret "maas-api-e2e-ait-745b1b91-serving-cert" not found
Aug 04 18:22:06.750064 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:06.750040    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e1d568d-dfa0-45e9-9e6b-6c33be9816e1-openshift-service-ca\") pod \"maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx\" (UID: \"2e1d568d-dfa0-45e9-9e6b-6c33be9816e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx"
Aug 04 18:22:06.758598 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:06.758579    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpl7m\" (UniqueName: \"kubernetes.io/projected/2e1d568d-dfa0-45e9-9e6b-6c33be9816e1-kube-api-access-fpl7m\") pod \"maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx\" (UID: \"2e1d568d-dfa0-45e9-9e6b-6c33be9816e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx"
Aug 04 18:22:07.254453 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.254419    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e1d568d-dfa0-45e9-9e6b-6c33be9816e1-maas-api-tls\") pod \"maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx\" (UID: \"2e1d568d-dfa0-45e9-9e6b-6c33be9816e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx"
Aug 04 18:22:07.256901 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.256875    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e1d568d-dfa0-45e9-9e6b-6c33be9816e1-maas-api-tls\") pod \"maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx\" (UID: \"2e1d568d-dfa0-45e9-9e6b-6c33be9816e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx"
Aug 04 18:22:07.322499 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.322467    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv"]
Aug 04 18:22:07.325859 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.325844    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv"
Aug 04 18:22:07.328130 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.328111    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-745b1b91\""
Aug 04 18:22:07.328240 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.328150    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-745b1b91-dockercfg-hkjfw\""
Aug 04 18:22:07.335672 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.335653    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv"]
Aug 04 18:22:07.355412 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.355390    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6bqz\" (UniqueName: \"kubernetes.io/projected/a2ece725-2d43-4eaf-a080-03e69eee3a68-kube-api-access-f6bqz\") pod \"payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv\" (UID: \"a2ece725-2d43-4eaf-a080-03e69eee3a68\") " pod="openshift-ingress/payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv"
Aug 04 18:22:07.355511 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.355442    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2ece725-2d43-4eaf-a080-03e69eee3a68-plugins-config-volume\") pod \"payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv\" (UID: \"a2ece725-2d43-4eaf-a080-03e69eee3a68\") " pod="openshift-ingress/payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv"
Aug 04 18:22:07.360884 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.360860    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q"]
Aug 04 18:22:07.364730 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.364713    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q"
Aug 04 18:22:07.374482 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.374457    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q"]
Aug 04 18:22:07.456779 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.456752    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2ece725-2d43-4eaf-a080-03e69eee3a68-plugins-config-volume\") pod \"payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv\" (UID: \"a2ece725-2d43-4eaf-a080-03e69eee3a68\") " pod="openshift-ingress/payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv"
Aug 04 18:22:07.456956 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.456803    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl42v\" (UniqueName: \"kubernetes.io/projected/12eaa26c-86f5-4938-b886-5d3ec6f192f2-kube-api-access-cl42v\") pod \"payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q\" (UID: \"12eaa26c-86f5-4938-b886-5d3ec6f192f2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q"
Aug 04 18:22:07.456956 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.456925    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6bqz\" (UniqueName: \"kubernetes.io/projected/a2ece725-2d43-4eaf-a080-03e69eee3a68-kube-api-access-f6bqz\") pod \"payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv\" (UID: \"a2ece725-2d43-4eaf-a080-03e69eee3a68\") " pod="openshift-ingress/payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv"
Aug 04 18:22:07.457276 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.457253    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12eaa26c-86f5-4938-b886-5d3ec6f192f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q\" (UID: \"12eaa26c-86f5-4938-b886-5d3ec6f192f2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q"
Aug 04 18:22:07.457782 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.457756    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2ece725-2d43-4eaf-a080-03e69eee3a68-plugins-config-volume\") pod \"payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv\" (UID: \"a2ece725-2d43-4eaf-a080-03e69eee3a68\") " pod="openshift-ingress/payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv"
Aug 04 18:22:07.466318 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.466294    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6bqz\" (UniqueName: \"kubernetes.io/projected/a2ece725-2d43-4eaf-a080-03e69eee3a68-kube-api-access-f6bqz\") pod \"payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv\" (UID: \"a2ece725-2d43-4eaf-a080-03e69eee3a68\") " pod="openshift-ingress/payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv"
Aug 04 18:22:07.529474 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.529425    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx"
Aug 04 18:22:07.558861 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.558830    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl42v\" (UniqueName: \"kubernetes.io/projected/12eaa26c-86f5-4938-b886-5d3ec6f192f2-kube-api-access-cl42v\") pod \"payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q\" (UID: \"12eaa26c-86f5-4938-b886-5d3ec6f192f2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q"
Aug 04 18:22:07.558983 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.558911    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12eaa26c-86f5-4938-b886-5d3ec6f192f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q\" (UID: \"12eaa26c-86f5-4938-b886-5d3ec6f192f2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q"
Aug 04 18:22:07.559657 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.559633    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12eaa26c-86f5-4938-b886-5d3ec6f192f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q\" (UID: \"12eaa26c-86f5-4938-b886-5d3ec6f192f2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q"
Aug 04 18:22:07.567611 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.567589    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl42v\" (UniqueName: \"kubernetes.io/projected/12eaa26c-86f5-4938-b886-5d3ec6f192f2-kube-api-access-cl42v\") pod \"payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q\" (UID: \"12eaa26c-86f5-4938-b886-5d3ec6f192f2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q"
Aug 04 18:22:07.638398 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.638366    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv"
Aug 04 18:22:07.667745 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.667719    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx"]
Aug 04 18:22:07.669967 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:22:07.669934    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e1d568d_dfa0_45e9_9e6b_6c33be9816e1.slice/crio-d05807d6945905e904e41c9667c7640e8a580469704e0e0283576dfc903a17d9 WatchSource:0}: Error finding container d05807d6945905e904e41c9667c7640e8a580469704e0e0283576dfc903a17d9: Status 404 returned error can't find the container with id d05807d6945905e904e41c9667c7640e8a580469704e0e0283576dfc903a17d9
Aug 04 18:22:07.671549 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.671521    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 18:22:07.675340 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.675319    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q"
Aug 04 18:22:07.728175 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.728130    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx" event={"ID":"2e1d568d-dfa0-45e9-9e6b-6c33be9816e1","Type":"ContainerStarted","Data":"d05807d6945905e904e41c9667c7640e8a580469704e0e0283576dfc903a17d9"}
Aug 04 18:22:07.799220 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.798502    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv"]
Aug 04 18:22:07.801239 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:22:07.801213    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2ece725_2d43_4eaf_a080_03e69eee3a68.slice/crio-977bba02f3789a6d2e2159904f24b15edbd3ac5aba07eeaf0fd497065a5ff6f2 WatchSource:0}: Error finding container 977bba02f3789a6d2e2159904f24b15edbd3ac5aba07eeaf0fd497065a5ff6f2: Status 404 returned error can't find the container with id 977bba02f3789a6d2e2159904f24b15edbd3ac5aba07eeaf0fd497065a5ff6f2
Aug 04 18:22:07.834620 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:07.834599    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q"]
Aug 04 18:22:07.837085 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:22:07.837059    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12eaa26c_86f5_4938_b886_5d3ec6f192f2.slice/crio-916114617da067ba92a575ae7c60b2bb5c678f4658043d22c362743134effc16 WatchSource:0}: Error finding container 916114617da067ba92a575ae7c60b2bb5c678f4658043d22c362743134effc16: Status 404 returned error can't find the container with id 916114617da067ba92a575ae7c60b2bb5c678f4658043d22c362743134effc16
Aug 04 18:22:08.735509 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:08.735469    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx" event={"ID":"2e1d568d-dfa0-45e9-9e6b-6c33be9816e1","Type":"ContainerStarted","Data":"10966d93a50660172f986b3fdc952534348fb365dfd41dfcade7c8bc54367e04"}
Aug 04 18:22:08.735978 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:08.735538    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx"
Aug 04 18:22:08.737139 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:08.737111    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv" event={"ID":"a2ece725-2d43-4eaf-a080-03e69eee3a68","Type":"ContainerStarted","Data":"9b4f8b336c4a75bc6be194d8ad8cf6d4a420d1a7294c9f4f6a693bb494e7c069"}
Aug 04 18:22:08.737268 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:08.737165    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv"
Aug 04 18:22:08.737268 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:08.737188    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv" event={"ID":"a2ece725-2d43-4eaf-a080-03e69eee3a68","Type":"ContainerStarted","Data":"977bba02f3789a6d2e2159904f24b15edbd3ac5aba07eeaf0fd497065a5ff6f2"}
Aug 04 18:22:08.738699 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:08.738675    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q" event={"ID":"12eaa26c-86f5-4938-b886-5d3ec6f192f2","Type":"ContainerStarted","Data":"87ef2fc4d60bf7d5e30bdf110135cfcb6ca3e858b2aeb23354cf93d83f0b5a63"}
Aug 04 18:22:08.738827 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:08.738719    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q" event={"ID":"12eaa26c-86f5-4938-b886-5d3ec6f192f2","Type":"ContainerStarted","Data":"916114617da067ba92a575ae7c60b2bb5c678f4658043d22c362743134effc16"}
Aug 04 18:22:08.738827 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:08.738764    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q"
Aug 04 18:22:08.753728 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:08.753684    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx" podStartSLOduration=2.175445427 podStartE2EDuration="2.75366835s" totalImagesPullingTime="578.222923ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:22:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:22:08.75218417 +0000 UTC m=+1810.935344346" watchObservedRunningTime="2026-08-04 18:22:08.75366835 +0000 UTC m=+1810.936828526"
Aug 04 18:22:08.769644 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:08.769599    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv" podStartSLOduration=1.7695810079999998 podStartE2EDuration="1.769581008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:22:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:22:08.767614965 +0000 UTC m=+1810.950775145" watchObservedRunningTime="2026-08-04 18:22:08.769581008 +0000 UTC m=+1810.952741185"
Aug 04 18:22:08.785256 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:08.785220    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q" podStartSLOduration=1.7852084559999999 podStartE2EDuration="1.785208456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:22:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:22:08.782812168 +0000 UTC m=+1810.965972344" watchObservedRunningTime="2026-08-04 18:22:08.785208456 +0000 UTC m=+1810.968368631"
Aug 04 18:22:14.749442 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:14.749417    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx"
Aug 04 18:22:19.746064 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:19.746015    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q" podUID="12eaa26c-86f5-4938-b886-5d3ec6f192f2" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.49:9004: connect: connection refused"
Aug 04 18:22:19.746571 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:19.746552    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv"
Aug 04 18:22:29.745346 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:29.745310    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q"
Aug 04 18:22:37.379385 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.379338    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx"]
Aug 04 18:22:37.379870 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.379568    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx" podUID="2e1d568d-dfa0-45e9-9e6b-6c33be9816e1" containerName="maas-api" containerID="cri-o://10966d93a50660172f986b3fdc952534348fb365dfd41dfcade7c8bc54367e04" gracePeriod=30
Aug 04 18:22:37.448705 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.448669    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv"]
Aug 04 18:22:37.448932 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.448904    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv" podUID="a2ece725-2d43-4eaf-a080-03e69eee3a68" containerName="payload-processing" containerID="cri-o://9b4f8b336c4a75bc6be194d8ad8cf6d4a420d1a7294c9f4f6a693bb494e7c069" gracePeriod=30
Aug 04 18:22:37.455077 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.455046    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q"]
Aug 04 18:22:37.455353 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.455292    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q" podUID="12eaa26c-86f5-4938-b886-5d3ec6f192f2" containerName="payload-pre-processing" containerID="cri-o://87ef2fc4d60bf7d5e30bdf110135cfcb6ca3e858b2aeb23354cf93d83f0b5a63" gracePeriod=30
Aug 04 18:22:37.742094 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.742059    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx"
Aug 04 18:22:37.744355 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.744308    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv"
Aug 04 18:22:37.785636 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.785607    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q"
Aug 04 18:22:37.891597 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.891564    2560 generic.go:356] "Generic (PLEG): container finished" podID="12eaa26c-86f5-4938-b886-5d3ec6f192f2" containerID="87ef2fc4d60bf7d5e30bdf110135cfcb6ca3e858b2aeb23354cf93d83f0b5a63" exitCode=0
Aug 04 18:22:37.891791 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.891607    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q"
Aug 04 18:22:37.891791 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.891649    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q" event={"ID":"12eaa26c-86f5-4938-b886-5d3ec6f192f2","Type":"ContainerDied","Data":"87ef2fc4d60bf7d5e30bdf110135cfcb6ca3e858b2aeb23354cf93d83f0b5a63"}
Aug 04 18:22:37.891791 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.891699    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q" event={"ID":"12eaa26c-86f5-4938-b886-5d3ec6f192f2","Type":"ContainerDied","Data":"916114617da067ba92a575ae7c60b2bb5c678f4658043d22c362743134effc16"}
Aug 04 18:22:37.891791 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.891729    2560 scope.go:117] "RemoveContainer" containerID="87ef2fc4d60bf7d5e30bdf110135cfcb6ca3e858b2aeb23354cf93d83f0b5a63"
Aug 04 18:22:37.893078 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.893055    2560 generic.go:356] "Generic (PLEG): container finished" podID="2e1d568d-dfa0-45e9-9e6b-6c33be9816e1" containerID="10966d93a50660172f986b3fdc952534348fb365dfd41dfcade7c8bc54367e04" exitCode=0
Aug 04 18:22:37.893165 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.893099    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx"
Aug 04 18:22:37.893165 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.893116    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx" event={"ID":"2e1d568d-dfa0-45e9-9e6b-6c33be9816e1","Type":"ContainerDied","Data":"10966d93a50660172f986b3fdc952534348fb365dfd41dfcade7c8bc54367e04"}
Aug 04 18:22:37.893267 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.893162    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx" event={"ID":"2e1d568d-dfa0-45e9-9e6b-6c33be9816e1","Type":"ContainerDied","Data":"d05807d6945905e904e41c9667c7640e8a580469704e0e0283576dfc903a17d9"}
Aug 04 18:22:37.894409 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.894382    2560 generic.go:356] "Generic (PLEG): container finished" podID="a2ece725-2d43-4eaf-a080-03e69eee3a68" containerID="9b4f8b336c4a75bc6be194d8ad8cf6d4a420d1a7294c9f4f6a693bb494e7c069" exitCode=0
Aug 04 18:22:37.894503 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.894424    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv" event={"ID":"a2ece725-2d43-4eaf-a080-03e69eee3a68","Type":"ContainerDied","Data":"9b4f8b336c4a75bc6be194d8ad8cf6d4a420d1a7294c9f4f6a693bb494e7c069"}
Aug 04 18:22:37.894503 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.894468    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv" event={"ID":"a2ece725-2d43-4eaf-a080-03e69eee3a68","Type":"ContainerDied","Data":"977bba02f3789a6d2e2159904f24b15edbd3ac5aba07eeaf0fd497065a5ff6f2"}
Aug 04 18:22:37.894503 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.894430    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv"
Aug 04 18:22:37.900781 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.900762    2560 scope.go:117] "RemoveContainer" containerID="87ef2fc4d60bf7d5e30bdf110135cfcb6ca3e858b2aeb23354cf93d83f0b5a63"
Aug 04 18:22:37.901075 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:22:37.901055    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87ef2fc4d60bf7d5e30bdf110135cfcb6ca3e858b2aeb23354cf93d83f0b5a63\": container with ID starting with 87ef2fc4d60bf7d5e30bdf110135cfcb6ca3e858b2aeb23354cf93d83f0b5a63 not found: ID does not exist" containerID="87ef2fc4d60bf7d5e30bdf110135cfcb6ca3e858b2aeb23354cf93d83f0b5a63"
Aug 04 18:22:37.901146 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.901089    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87ef2fc4d60bf7d5e30bdf110135cfcb6ca3e858b2aeb23354cf93d83f0b5a63"} err="failed to get container status \"87ef2fc4d60bf7d5e30bdf110135cfcb6ca3e858b2aeb23354cf93d83f0b5a63\": rpc error: code = NotFound desc = could not find container \"87ef2fc4d60bf7d5e30bdf110135cfcb6ca3e858b2aeb23354cf93d83f0b5a63\": container with ID starting with 87ef2fc4d60bf7d5e30bdf110135cfcb6ca3e858b2aeb23354cf93d83f0b5a63 not found: ID does not exist"
Aug 04 18:22:37.901146 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.901106    2560 scope.go:117] "RemoveContainer" containerID="10966d93a50660172f986b3fdc952534348fb365dfd41dfcade7c8bc54367e04"
Aug 04 18:22:37.909778 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.909754    2560 scope.go:117] "RemoveContainer" containerID="10966d93a50660172f986b3fdc952534348fb365dfd41dfcade7c8bc54367e04"
Aug 04 18:22:37.910144 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:22:37.910124    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10966d93a50660172f986b3fdc952534348fb365dfd41dfcade7c8bc54367e04\": container with ID starting with 10966d93a50660172f986b3fdc952534348fb365dfd41dfcade7c8bc54367e04 not found: ID does not exist" containerID="10966d93a50660172f986b3fdc952534348fb365dfd41dfcade7c8bc54367e04"
Aug 04 18:22:37.910210 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.910153    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10966d93a50660172f986b3fdc952534348fb365dfd41dfcade7c8bc54367e04"} err="failed to get container status \"10966d93a50660172f986b3fdc952534348fb365dfd41dfcade7c8bc54367e04\": rpc error: code = NotFound desc = could not find container \"10966d93a50660172f986b3fdc952534348fb365dfd41dfcade7c8bc54367e04\": container with ID starting with 10966d93a50660172f986b3fdc952534348fb365dfd41dfcade7c8bc54367e04 not found: ID does not exist"
Aug 04 18:22:37.910210 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.910171    2560 scope.go:117] "RemoveContainer" containerID="9b4f8b336c4a75bc6be194d8ad8cf6d4a420d1a7294c9f4f6a693bb494e7c069"
Aug 04 18:22:37.918617 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.918597    2560 scope.go:117] "RemoveContainer" containerID="9b4f8b336c4a75bc6be194d8ad8cf6d4a420d1a7294c9f4f6a693bb494e7c069"
Aug 04 18:22:37.918920 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:22:37.918900    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b4f8b336c4a75bc6be194d8ad8cf6d4a420d1a7294c9f4f6a693bb494e7c069\": container with ID starting with 9b4f8b336c4a75bc6be194d8ad8cf6d4a420d1a7294c9f4f6a693bb494e7c069 not found: ID does not exist" containerID="9b4f8b336c4a75bc6be194d8ad8cf6d4a420d1a7294c9f4f6a693bb494e7c069"
Aug 04 18:22:37.918981 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.918931    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b4f8b336c4a75bc6be194d8ad8cf6d4a420d1a7294c9f4f6a693bb494e7c069"} err="failed to get container status \"9b4f8b336c4a75bc6be194d8ad8cf6d4a420d1a7294c9f4f6a693bb494e7c069\": rpc error: code = NotFound desc = could not find container \"9b4f8b336c4a75bc6be194d8ad8cf6d4a420d1a7294c9f4f6a693bb494e7c069\": container with ID starting with 9b4f8b336c4a75bc6be194d8ad8cf6d4a420d1a7294c9f4f6a693bb494e7c069 not found: ID does not exist"
Aug 04 18:22:37.936296 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.936262    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e1d568d-dfa0-45e9-9e6b-6c33be9816e1-maas-api-tls\") pod \"2e1d568d-dfa0-45e9-9e6b-6c33be9816e1\" (UID: \"2e1d568d-dfa0-45e9-9e6b-6c33be9816e1\") "
Aug 04 18:22:37.936428 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.936306    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6bqz\" (UniqueName: \"kubernetes.io/projected/a2ece725-2d43-4eaf-a080-03e69eee3a68-kube-api-access-f6bqz\") pod \"a2ece725-2d43-4eaf-a080-03e69eee3a68\" (UID: \"a2ece725-2d43-4eaf-a080-03e69eee3a68\") "
Aug 04 18:22:37.936428 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.936338    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2ece725-2d43-4eaf-a080-03e69eee3a68-plugins-config-volume\") pod \"a2ece725-2d43-4eaf-a080-03e69eee3a68\" (UID: \"a2ece725-2d43-4eaf-a080-03e69eee3a68\") "
Aug 04 18:22:37.936428 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.936385    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12eaa26c-86f5-4938-b886-5d3ec6f192f2-plugins-config-volume\") pod \"12eaa26c-86f5-4938-b886-5d3ec6f192f2\" (UID: \"12eaa26c-86f5-4938-b886-5d3ec6f192f2\") "
Aug 04 18:22:37.936428 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.936412    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fpl7m\" (UniqueName: \"kubernetes.io/projected/2e1d568d-dfa0-45e9-9e6b-6c33be9816e1-kube-api-access-fpl7m\") pod \"2e1d568d-dfa0-45e9-9e6b-6c33be9816e1\" (UID: \"2e1d568d-dfa0-45e9-9e6b-6c33be9816e1\") "
Aug 04 18:22:37.936624 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.936474    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cl42v\" (UniqueName: \"kubernetes.io/projected/12eaa26c-86f5-4938-b886-5d3ec6f192f2-kube-api-access-cl42v\") pod \"12eaa26c-86f5-4938-b886-5d3ec6f192f2\" (UID: \"12eaa26c-86f5-4938-b886-5d3ec6f192f2\") "
Aug 04 18:22:37.936624 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.936533    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e1d568d-dfa0-45e9-9e6b-6c33be9816e1-openshift-service-ca\") pod \"2e1d568d-dfa0-45e9-9e6b-6c33be9816e1\" (UID: \"2e1d568d-dfa0-45e9-9e6b-6c33be9816e1\") "
Aug 04 18:22:37.936791 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.936759    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2ece725-2d43-4eaf-a080-03e69eee3a68-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a2ece725-2d43-4eaf-a080-03e69eee3a68" (UID: "a2ece725-2d43-4eaf-a080-03e69eee3a68"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:22:37.936791 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.936773    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12eaa26c-86f5-4938-b886-5d3ec6f192f2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "12eaa26c-86f5-4938-b886-5d3ec6f192f2" (UID: "12eaa26c-86f5-4938-b886-5d3ec6f192f2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:22:37.937323 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.937177    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e1d568d-dfa0-45e9-9e6b-6c33be9816e1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2e1d568d-dfa0-45e9-9e6b-6c33be9816e1" (UID: "2e1d568d-dfa0-45e9-9e6b-6c33be9816e1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:22:37.938948 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.938923    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2ece725-2d43-4eaf-a080-03e69eee3a68-kube-api-access-f6bqz" (OuterVolumeSpecName: "kube-api-access-f6bqz") pod "a2ece725-2d43-4eaf-a080-03e69eee3a68" (UID: "a2ece725-2d43-4eaf-a080-03e69eee3a68"). InnerVolumeSpecName "kube-api-access-f6bqz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:22:37.938948 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.938929    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12eaa26c-86f5-4938-b886-5d3ec6f192f2-kube-api-access-cl42v" (OuterVolumeSpecName: "kube-api-access-cl42v") pod "12eaa26c-86f5-4938-b886-5d3ec6f192f2" (UID: "12eaa26c-86f5-4938-b886-5d3ec6f192f2"). InnerVolumeSpecName "kube-api-access-cl42v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:22:37.939274 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.939248    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e1d568d-dfa0-45e9-9e6b-6c33be9816e1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2e1d568d-dfa0-45e9-9e6b-6c33be9816e1" (UID: "2e1d568d-dfa0-45e9-9e6b-6c33be9816e1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:22:37.939351 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:37.939249    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e1d568d-dfa0-45e9-9e6b-6c33be9816e1-kube-api-access-fpl7m" (OuterVolumeSpecName: "kube-api-access-fpl7m") pod "2e1d568d-dfa0-45e9-9e6b-6c33be9816e1" (UID: "2e1d568d-dfa0-45e9-9e6b-6c33be9816e1"). InnerVolumeSpecName "kube-api-access-fpl7m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:22:38.037346 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:38.037313    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12eaa26c-86f5-4938-b886-5d3ec6f192f2-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:22:38.037346 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:38.037341    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpl7m\" (UniqueName: \"kubernetes.io/projected/2e1d568d-dfa0-45e9-9e6b-6c33be9816e1-kube-api-access-fpl7m\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:22:38.037346 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:38.037351    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cl42v\" (UniqueName: \"kubernetes.io/projected/12eaa26c-86f5-4938-b886-5d3ec6f192f2-kube-api-access-cl42v\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:22:38.037799 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:38.037361    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e1d568d-dfa0-45e9-9e6b-6c33be9816e1-openshift-service-ca\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:22:38.037799 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:38.037372    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e1d568d-dfa0-45e9-9e6b-6c33be9816e1-maas-api-tls\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:22:38.037799 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:38.037381    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6bqz\" (UniqueName: \"kubernetes.io/projected/a2ece725-2d43-4eaf-a080-03e69eee3a68-kube-api-access-f6bqz\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:22:38.037799 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:38.037388    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2ece725-2d43-4eaf-a080-03e69eee3a68-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:22:38.225556 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:38.225515    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q"]
Aug 04 18:22:38.232271 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:38.232230    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-745b1b91-546854dfd4-vlk5q"]
Aug 04 18:22:38.251281 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:38.251219    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv"]
Aug 04 18:22:38.253929 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:38.253897    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-745b1b91-68c787d7d6-h94lv"]
Aug 04 18:22:38.268715 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:38.268677    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx"]
Aug 04 18:22:38.273402 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:38.273374    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-745b1b91-78fb6d8ccc-g9hzx"]
Aug 04 18:22:38.354219 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:38.354180    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12eaa26c-86f5-4938-b886-5d3ec6f192f2" path="/var/lib/kubelet/pods/12eaa26c-86f5-4938-b886-5d3ec6f192f2/volumes"
Aug 04 18:22:38.354636 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:38.354622    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e1d568d-dfa0-45e9-9e6b-6c33be9816e1" path="/var/lib/kubelet/pods/2e1d568d-dfa0-45e9-9e6b-6c33be9816e1/volumes"
Aug 04 18:22:38.355008 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:38.354979    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2ece725-2d43-4eaf-a080-03e69eee3a68" path="/var/lib/kubelet/pods/a2ece725-2d43-4eaf-a080-03e69eee3a68/volumes"
Aug 04 18:22:41.864470 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:41.864431    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266"]
Aug 04 18:22:41.865001 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:41.864964    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2ece725-2d43-4eaf-a080-03e69eee3a68" containerName="payload-processing"
Aug 04 18:22:41.865089 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:41.865005    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2ece725-2d43-4eaf-a080-03e69eee3a68" containerName="payload-processing"
Aug 04 18:22:41.865089 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:41.865017    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e1d568d-dfa0-45e9-9e6b-6c33be9816e1" containerName="maas-api"
Aug 04 18:22:41.865089 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:41.865027    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e1d568d-dfa0-45e9-9e6b-6c33be9816e1" containerName="maas-api"
Aug 04 18:22:41.865089 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:41.865042    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12eaa26c-86f5-4938-b886-5d3ec6f192f2" containerName="payload-pre-processing"
Aug 04 18:22:41.865089 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:41.865051    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="12eaa26c-86f5-4938-b886-5d3ec6f192f2" containerName="payload-pre-processing"
Aug 04 18:22:41.865372 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:41.865171    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="12eaa26c-86f5-4938-b886-5d3ec6f192f2" containerName="payload-pre-processing"
Aug 04 18:22:41.865372 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:41.865185    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e1d568d-dfa0-45e9-9e6b-6c33be9816e1" containerName="maas-api"
Aug 04 18:22:41.865372 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:41.865195    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2ece725-2d43-4eaf-a080-03e69eee3a68" containerName="payload-processing"
Aug 04 18:22:41.869953 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:41.869931    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266"
Aug 04 18:22:41.872194 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:41.872174    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-16a9c503-serving-cert\""
Aug 04 18:22:41.880772 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:41.880743    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266"]
Aug 04 18:22:41.967641 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:41.967594    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b985105a-50cb-4d88-8759-4afc9da58a4f-maas-api-tls\") pod \"maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266\" (UID: \"b985105a-50cb-4d88-8759-4afc9da58a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266"
Aug 04 18:22:41.967828 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:41.967666    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b985105a-50cb-4d88-8759-4afc9da58a4f-openshift-service-ca\") pod \"maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266\" (UID: \"b985105a-50cb-4d88-8759-4afc9da58a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266"
Aug 04 18:22:41.967828 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:41.967695    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9pkg\" (UniqueName: \"kubernetes.io/projected/b985105a-50cb-4d88-8759-4afc9da58a4f-kube-api-access-w9pkg\") pod \"maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266\" (UID: \"b985105a-50cb-4d88-8759-4afc9da58a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266"
Aug 04 18:22:42.068213 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.068167    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b985105a-50cb-4d88-8759-4afc9da58a4f-openshift-service-ca\") pod \"maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266\" (UID: \"b985105a-50cb-4d88-8759-4afc9da58a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266"
Aug 04 18:22:42.068213 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.068215    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9pkg\" (UniqueName: \"kubernetes.io/projected/b985105a-50cb-4d88-8759-4afc9da58a4f-kube-api-access-w9pkg\") pod \"maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266\" (UID: \"b985105a-50cb-4d88-8759-4afc9da58a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266"
Aug 04 18:22:42.068486 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.068295    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b985105a-50cb-4d88-8759-4afc9da58a4f-maas-api-tls\") pod \"maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266\" (UID: \"b985105a-50cb-4d88-8759-4afc9da58a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266"
Aug 04 18:22:42.068938 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.068916    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b985105a-50cb-4d88-8759-4afc9da58a4f-openshift-service-ca\") pod \"maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266\" (UID: \"b985105a-50cb-4d88-8759-4afc9da58a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266"
Aug 04 18:22:42.070894 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.070871    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b985105a-50cb-4d88-8759-4afc9da58a4f-maas-api-tls\") pod \"maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266\" (UID: \"b985105a-50cb-4d88-8759-4afc9da58a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266"
Aug 04 18:22:42.077968 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.077929    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9pkg\" (UniqueName: \"kubernetes.io/projected/b985105a-50cb-4d88-8759-4afc9da58a4f-kube-api-access-w9pkg\") pod \"maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266\" (UID: \"b985105a-50cb-4d88-8759-4afc9da58a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266"
Aug 04 18:22:42.184323 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.184282    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266"
Aug 04 18:22:42.535081 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.535054    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266"]
Aug 04 18:22:42.536896 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:22:42.536865    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb985105a_50cb_4d88_8759_4afc9da58a4f.slice/crio-027021ef27711cf814dbca27c4b76cda7fb331de7c05483b9c0684135018d9ab WatchSource:0}: Error finding container 027021ef27711cf814dbca27c4b76cda7fb331de7c05483b9c0684135018d9ab: Status 404 returned error can't find the container with id 027021ef27711cf814dbca27c4b76cda7fb331de7c05483b9c0684135018d9ab
Aug 04 18:22:42.647467 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.647432    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4"]
Aug 04 18:22:42.652212 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.652191    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4"
Aug 04 18:22:42.654927 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.654909    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-16a9c503-dockercfg-2ctbx\""
Aug 04 18:22:42.655018 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.654927    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-16a9c503\""
Aug 04 18:22:42.661467 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.661441    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4"]
Aug 04 18:22:42.673244 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.673220    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0775f20-4ea4-4e78-94f7-93d321b2239e-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4\" (UID: \"d0775f20-4ea4-4e78-94f7-93d321b2239e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4"
Aug 04 18:22:42.673345 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.673294    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvthw\" (UniqueName: \"kubernetes.io/projected/d0775f20-4ea4-4e78-94f7-93d321b2239e-kube-api-access-pvthw\") pod \"payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4\" (UID: \"d0775f20-4ea4-4e78-94f7-93d321b2239e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4"
Aug 04 18:22:42.723112 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.723090    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m"]
Aug 04 18:22:42.726518 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.726503    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m"
Aug 04 18:22:42.743924 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.743903    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m"]
Aug 04 18:22:42.774043 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.774013    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0775f20-4ea4-4e78-94f7-93d321b2239e-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4\" (UID: \"d0775f20-4ea4-4e78-94f7-93d321b2239e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4"
Aug 04 18:22:42.774195 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.774091    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/faa0ea61-8fa3-496e-8946-e2b03294907a-plugins-config-volume\") pod \"payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m\" (UID: \"faa0ea61-8fa3-496e-8946-e2b03294907a\") " pod="openshift-ingress/payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m"
Aug 04 18:22:42.774195 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.774130    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvthw\" (UniqueName: \"kubernetes.io/projected/d0775f20-4ea4-4e78-94f7-93d321b2239e-kube-api-access-pvthw\") pod \"payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4\" (UID: \"d0775f20-4ea4-4e78-94f7-93d321b2239e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4"
Aug 04 18:22:42.774195 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.774154    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdkqp\" (UniqueName: \"kubernetes.io/projected/faa0ea61-8fa3-496e-8946-e2b03294907a-kube-api-access-fdkqp\") pod \"payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m\" (UID: \"faa0ea61-8fa3-496e-8946-e2b03294907a\") " pod="openshift-ingress/payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m"
Aug 04 18:22:42.774710 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.774690    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0775f20-4ea4-4e78-94f7-93d321b2239e-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4\" (UID: \"d0775f20-4ea4-4e78-94f7-93d321b2239e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4"
Aug 04 18:22:42.783539 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.783517    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvthw\" (UniqueName: \"kubernetes.io/projected/d0775f20-4ea4-4e78-94f7-93d321b2239e-kube-api-access-pvthw\") pod \"payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4\" (UID: \"d0775f20-4ea4-4e78-94f7-93d321b2239e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4"
Aug 04 18:22:42.874742 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.874720    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/faa0ea61-8fa3-496e-8946-e2b03294907a-plugins-config-volume\") pod \"payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m\" (UID: \"faa0ea61-8fa3-496e-8946-e2b03294907a\") " pod="openshift-ingress/payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m"
Aug 04 18:22:42.875071 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.874753    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdkqp\" (UniqueName: \"kubernetes.io/projected/faa0ea61-8fa3-496e-8946-e2b03294907a-kube-api-access-fdkqp\") pod \"payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m\" (UID: \"faa0ea61-8fa3-496e-8946-e2b03294907a\") " pod="openshift-ingress/payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m"
Aug 04 18:22:42.875500 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.875481    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/faa0ea61-8fa3-496e-8946-e2b03294907a-plugins-config-volume\") pod \"payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m\" (UID: \"faa0ea61-8fa3-496e-8946-e2b03294907a\") " pod="openshift-ingress/payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m"
Aug 04 18:22:42.885605 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.885583    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdkqp\" (UniqueName: \"kubernetes.io/projected/faa0ea61-8fa3-496e-8946-e2b03294907a-kube-api-access-fdkqp\") pod \"payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m\" (UID: \"faa0ea61-8fa3-496e-8946-e2b03294907a\") " pod="openshift-ingress/payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m"
Aug 04 18:22:42.922077 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.922033    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266" event={"ID":"b985105a-50cb-4d88-8759-4afc9da58a4f","Type":"ContainerStarted","Data":"027021ef27711cf814dbca27c4b76cda7fb331de7c05483b9c0684135018d9ab"}
Aug 04 18:22:42.964132 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:42.964097    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4"
Aug 04 18:22:43.036777 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:43.036742    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m"
Aug 04 18:22:43.121600 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:43.121577    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4"]
Aug 04 18:22:43.124882 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:22:43.124842    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0775f20_4ea4_4e78_94f7_93d321b2239e.slice/crio-25df52e6656d438fc1d1d4a17234989416e2d4af771f617714cb7c8daf5e5b51 WatchSource:0}: Error finding container 25df52e6656d438fc1d1d4a17234989416e2d4af771f617714cb7c8daf5e5b51: Status 404 returned error can't find the container with id 25df52e6656d438fc1d1d4a17234989416e2d4af771f617714cb7c8daf5e5b51
Aug 04 18:22:43.203622 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:43.203501    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m"]
Aug 04 18:22:43.206032 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:22:43.206005    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaa0ea61_8fa3_496e_8946_e2b03294907a.slice/crio-b4ab3ea4edaf2d0f2facb1a1c82d662fa090bca4d1f4b516ea402c7a5183d396 WatchSource:0}: Error finding container b4ab3ea4edaf2d0f2facb1a1c82d662fa090bca4d1f4b516ea402c7a5183d396: Status 404 returned error can't find the container with id b4ab3ea4edaf2d0f2facb1a1c82d662fa090bca4d1f4b516ea402c7a5183d396
Aug 04 18:22:43.930103 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:43.930068    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266" event={"ID":"b985105a-50cb-4d88-8759-4afc9da58a4f","Type":"ContainerStarted","Data":"19aab5777dcb27ecd9be014622ff3be141f85a328d0fcc36e2cbcdebb79eb3aa"}
Aug 04 18:22:43.930585 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:43.930274    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266"
Aug 04 18:22:43.932190 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:43.932158    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m" event={"ID":"faa0ea61-8fa3-496e-8946-e2b03294907a","Type":"ContainerStarted","Data":"5e1ea6963c6bf3448859a99115d70b07c9fa54f17fc1618667f9c06de591384c"}
Aug 04 18:22:43.932330 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:43.932217    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m" event={"ID":"faa0ea61-8fa3-496e-8946-e2b03294907a","Type":"ContainerStarted","Data":"b4ab3ea4edaf2d0f2facb1a1c82d662fa090bca4d1f4b516ea402c7a5183d396"}
Aug 04 18:22:43.932486 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:43.932473    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m"
Aug 04 18:22:43.933801 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:43.933778    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4" event={"ID":"d0775f20-4ea4-4e78-94f7-93d321b2239e","Type":"ContainerStarted","Data":"c9f5dc0cddab9098270ca97a5c4baade5044a52e75276579af90011760d2bc55"}
Aug 04 18:22:43.933918 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:43.933819    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4" event={"ID":"d0775f20-4ea4-4e78-94f7-93d321b2239e","Type":"ContainerStarted","Data":"25df52e6656d438fc1d1d4a17234989416e2d4af771f617714cb7c8daf5e5b51"}
Aug 04 18:22:43.934160 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:43.933962    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4"
Aug 04 18:22:43.948437 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:43.948384    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266" podStartSLOduration=2.615229004 podStartE2EDuration="2.94836699s" totalImagesPullingTime="333.137986ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:22:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:22:43.947281707 +0000 UTC m=+1846.130441884" watchObservedRunningTime="2026-08-04 18:22:43.94836699 +0000 UTC m=+1846.131527173"
Aug 04 18:22:43.965609 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:43.965563    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4" podStartSLOduration=1.965544905 podStartE2EDuration="1.965544905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:22:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:22:43.963159319 +0000 UTC m=+1846.146319498" watchObservedRunningTime="2026-08-04 18:22:43.965544905 +0000 UTC m=+1846.148705084"
Aug 04 18:22:43.979269 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:43.979208    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m" podStartSLOduration=1.9791943600000002 podStartE2EDuration="1.97919436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:22:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:22:43.97849091 +0000 UTC m=+1846.161651086" watchObservedRunningTime="2026-08-04 18:22:43.97919436 +0000 UTC m=+1846.162354595"
Aug 04 18:22:49.945362 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:49.945332    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266"
Aug 04 18:22:54.942170 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:54.942140    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4"
Aug 04 18:22:55.946639 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:22:55.946609    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m"
Aug 04 18:23:12.732606 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:12.732573    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266"]
Aug 04 18:23:12.733120 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:12.732770    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266" podUID="b985105a-50cb-4d88-8759-4afc9da58a4f" containerName="maas-api" containerID="cri-o://19aab5777dcb27ecd9be014622ff3be141f85a328d0fcc36e2cbcdebb79eb3aa" gracePeriod=30
Aug 04 18:23:12.807422 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:12.807378    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m"]
Aug 04 18:23:12.807721 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:12.807670    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m" podUID="faa0ea61-8fa3-496e-8946-e2b03294907a" containerName="payload-processing" containerID="cri-o://5e1ea6963c6bf3448859a99115d70b07c9fa54f17fc1618667f9c06de591384c" gracePeriod=30
Aug 04 18:23:12.813278 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:12.813250    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4"]
Aug 04 18:23:12.813632 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:12.813601    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4" podUID="d0775f20-4ea4-4e78-94f7-93d321b2239e" containerName="payload-pre-processing" containerID="cri-o://c9f5dc0cddab9098270ca97a5c4baade5044a52e75276579af90011760d2bc55" gracePeriod=30
Aug 04 18:23:13.077239 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.077209    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266"
Aug 04 18:23:13.081444 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.081419    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m"
Aug 04 18:23:13.083554 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.083531    2560 generic.go:356] "Generic (PLEG): container finished" podID="d0775f20-4ea4-4e78-94f7-93d321b2239e" containerID="c9f5dc0cddab9098270ca97a5c4baade5044a52e75276579af90011760d2bc55" exitCode=0
Aug 04 18:23:13.083681 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.083600    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4" event={"ID":"d0775f20-4ea4-4e78-94f7-93d321b2239e","Type":"ContainerDied","Data":"c9f5dc0cddab9098270ca97a5c4baade5044a52e75276579af90011760d2bc55"}
Aug 04 18:23:13.085008 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.084964    2560 generic.go:356] "Generic (PLEG): container finished" podID="b985105a-50cb-4d88-8759-4afc9da58a4f" containerID="19aab5777dcb27ecd9be014622ff3be141f85a328d0fcc36e2cbcdebb79eb3aa" exitCode=0
Aug 04 18:23:13.085109 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.085023    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266"
Aug 04 18:23:13.085109 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.085060    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266" event={"ID":"b985105a-50cb-4d88-8759-4afc9da58a4f","Type":"ContainerDied","Data":"19aab5777dcb27ecd9be014622ff3be141f85a328d0fcc36e2cbcdebb79eb3aa"}
Aug 04 18:23:13.085215 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.085118    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266" event={"ID":"b985105a-50cb-4d88-8759-4afc9da58a4f","Type":"ContainerDied","Data":"027021ef27711cf814dbca27c4b76cda7fb331de7c05483b9c0684135018d9ab"}
Aug 04 18:23:13.085215 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.085153    2560 scope.go:117] "RemoveContainer" containerID="19aab5777dcb27ecd9be014622ff3be141f85a328d0fcc36e2cbcdebb79eb3aa"
Aug 04 18:23:13.086383 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.086359    2560 generic.go:356] "Generic (PLEG): container finished" podID="faa0ea61-8fa3-496e-8946-e2b03294907a" containerID="5e1ea6963c6bf3448859a99115d70b07c9fa54f17fc1618667f9c06de591384c" exitCode=0
Aug 04 18:23:13.086455 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.086389    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m" event={"ID":"faa0ea61-8fa3-496e-8946-e2b03294907a","Type":"ContainerDied","Data":"5e1ea6963c6bf3448859a99115d70b07c9fa54f17fc1618667f9c06de591384c"}
Aug 04 18:23:13.086455 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.086418    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m" event={"ID":"faa0ea61-8fa3-496e-8946-e2b03294907a","Type":"ContainerDied","Data":"b4ab3ea4edaf2d0f2facb1a1c82d662fa090bca4d1f4b516ea402c7a5183d396"}
Aug 04 18:23:13.086525 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.086466    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m"
Aug 04 18:23:13.093913 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.093896    2560 scope.go:117] "RemoveContainer" containerID="19aab5777dcb27ecd9be014622ff3be141f85a328d0fcc36e2cbcdebb79eb3aa"
Aug 04 18:23:13.094236 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:23:13.094219    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19aab5777dcb27ecd9be014622ff3be141f85a328d0fcc36e2cbcdebb79eb3aa\": container with ID starting with 19aab5777dcb27ecd9be014622ff3be141f85a328d0fcc36e2cbcdebb79eb3aa not found: ID does not exist" containerID="19aab5777dcb27ecd9be014622ff3be141f85a328d0fcc36e2cbcdebb79eb3aa"
Aug 04 18:23:13.094303 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.094246    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19aab5777dcb27ecd9be014622ff3be141f85a328d0fcc36e2cbcdebb79eb3aa"} err="failed to get container status \"19aab5777dcb27ecd9be014622ff3be141f85a328d0fcc36e2cbcdebb79eb3aa\": rpc error: code = NotFound desc = could not find container \"19aab5777dcb27ecd9be014622ff3be141f85a328d0fcc36e2cbcdebb79eb3aa\": container with ID starting with 19aab5777dcb27ecd9be014622ff3be141f85a328d0fcc36e2cbcdebb79eb3aa not found: ID does not exist"
Aug 04 18:23:13.094303 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.094264    2560 scope.go:117] "RemoveContainer" containerID="5e1ea6963c6bf3448859a99115d70b07c9fa54f17fc1618667f9c06de591384c"
Aug 04 18:23:13.108224 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.108205    2560 scope.go:117] "RemoveContainer" containerID="5e1ea6963c6bf3448859a99115d70b07c9fa54f17fc1618667f9c06de591384c"
Aug 04 18:23:13.108618 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:23:13.108585    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e1ea6963c6bf3448859a99115d70b07c9fa54f17fc1618667f9c06de591384c\": container with ID starting with 5e1ea6963c6bf3448859a99115d70b07c9fa54f17fc1618667f9c06de591384c not found: ID does not exist" containerID="5e1ea6963c6bf3448859a99115d70b07c9fa54f17fc1618667f9c06de591384c"
Aug 04 18:23:13.108723 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.108617    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e1ea6963c6bf3448859a99115d70b07c9fa54f17fc1618667f9c06de591384c"} err="failed to get container status \"5e1ea6963c6bf3448859a99115d70b07c9fa54f17fc1618667f9c06de591384c\": rpc error: code = NotFound desc = could not find container \"5e1ea6963c6bf3448859a99115d70b07c9fa54f17fc1618667f9c06de591384c\": container with ID starting with 5e1ea6963c6bf3448859a99115d70b07c9fa54f17fc1618667f9c06de591384c not found: ID does not exist"
Aug 04 18:23:13.123134 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.123102    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4"
Aug 04 18:23:13.148544 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.148512    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9pkg\" (UniqueName: \"kubernetes.io/projected/b985105a-50cb-4d88-8759-4afc9da58a4f-kube-api-access-w9pkg\") pod \"b985105a-50cb-4d88-8759-4afc9da58a4f\" (UID: \"b985105a-50cb-4d88-8759-4afc9da58a4f\") "
Aug 04 18:23:13.148756 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.148593    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/faa0ea61-8fa3-496e-8946-e2b03294907a-plugins-config-volume\") pod \"faa0ea61-8fa3-496e-8946-e2b03294907a\" (UID: \"faa0ea61-8fa3-496e-8946-e2b03294907a\") "
Aug 04 18:23:13.148756 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.148626    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdkqp\" (UniqueName: \"kubernetes.io/projected/faa0ea61-8fa3-496e-8946-e2b03294907a-kube-api-access-fdkqp\") pod \"faa0ea61-8fa3-496e-8946-e2b03294907a\" (UID: \"faa0ea61-8fa3-496e-8946-e2b03294907a\") "
Aug 04 18:23:13.148756 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.148675    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b985105a-50cb-4d88-8759-4afc9da58a4f-openshift-service-ca\") pod \"b985105a-50cb-4d88-8759-4afc9da58a4f\" (UID: \"b985105a-50cb-4d88-8759-4afc9da58a4f\") "
Aug 04 18:23:13.148756 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.148735    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b985105a-50cb-4d88-8759-4afc9da58a4f-maas-api-tls\") pod \"b985105a-50cb-4d88-8759-4afc9da58a4f\" (UID: \"b985105a-50cb-4d88-8759-4afc9da58a4f\") "
Aug 04 18:23:13.149187 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.149155    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b985105a-50cb-4d88-8759-4afc9da58a4f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b985105a-50cb-4d88-8759-4afc9da58a4f" (UID: "b985105a-50cb-4d88-8759-4afc9da58a4f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:23:13.149339 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.149215    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/faa0ea61-8fa3-496e-8946-e2b03294907a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "faa0ea61-8fa3-496e-8946-e2b03294907a" (UID: "faa0ea61-8fa3-496e-8946-e2b03294907a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:23:13.150961 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.150925    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b985105a-50cb-4d88-8759-4afc9da58a4f-kube-api-access-w9pkg" (OuterVolumeSpecName: "kube-api-access-w9pkg") pod "b985105a-50cb-4d88-8759-4afc9da58a4f" (UID: "b985105a-50cb-4d88-8759-4afc9da58a4f"). InnerVolumeSpecName "kube-api-access-w9pkg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:23:13.151269 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.151247    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b985105a-50cb-4d88-8759-4afc9da58a4f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b985105a-50cb-4d88-8759-4afc9da58a4f" (UID: "b985105a-50cb-4d88-8759-4afc9da58a4f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:23:13.151269 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.151245    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/faa0ea61-8fa3-496e-8946-e2b03294907a-kube-api-access-fdkqp" (OuterVolumeSpecName: "kube-api-access-fdkqp") pod "faa0ea61-8fa3-496e-8946-e2b03294907a" (UID: "faa0ea61-8fa3-496e-8946-e2b03294907a"). InnerVolumeSpecName "kube-api-access-fdkqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:23:13.249749 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.249661    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvthw\" (UniqueName: \"kubernetes.io/projected/d0775f20-4ea4-4e78-94f7-93d321b2239e-kube-api-access-pvthw\") pod \"d0775f20-4ea4-4e78-94f7-93d321b2239e\" (UID: \"d0775f20-4ea4-4e78-94f7-93d321b2239e\") "
Aug 04 18:23:13.249749 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.249730    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0775f20-4ea4-4e78-94f7-93d321b2239e-plugins-config-volume\") pod \"d0775f20-4ea4-4e78-94f7-93d321b2239e\" (UID: \"d0775f20-4ea4-4e78-94f7-93d321b2239e\") "
Aug 04 18:23:13.249936 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.249903    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/faa0ea61-8fa3-496e-8946-e2b03294907a-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:23:13.249936 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.249914    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdkqp\" (UniqueName: \"kubernetes.io/projected/faa0ea61-8fa3-496e-8946-e2b03294907a-kube-api-access-fdkqp\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:23:13.249936 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.249924    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b985105a-50cb-4d88-8759-4afc9da58a4f-openshift-service-ca\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:23:13.249936 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.249933    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b985105a-50cb-4d88-8759-4afc9da58a4f-maas-api-tls\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:23:13.250102 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.249941    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9pkg\" (UniqueName: \"kubernetes.io/projected/b985105a-50cb-4d88-8759-4afc9da58a4f-kube-api-access-w9pkg\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:23:13.250137 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.250111    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0775f20-4ea4-4e78-94f7-93d321b2239e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d0775f20-4ea4-4e78-94f7-93d321b2239e" (UID: "d0775f20-4ea4-4e78-94f7-93d321b2239e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:23:13.251867 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.251845    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0775f20-4ea4-4e78-94f7-93d321b2239e-kube-api-access-pvthw" (OuterVolumeSpecName: "kube-api-access-pvthw") pod "d0775f20-4ea4-4e78-94f7-93d321b2239e" (UID: "d0775f20-4ea4-4e78-94f7-93d321b2239e"). InnerVolumeSpecName "kube-api-access-pvthw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:23:13.350898 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.350871    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvthw\" (UniqueName: \"kubernetes.io/projected/d0775f20-4ea4-4e78-94f7-93d321b2239e-kube-api-access-pvthw\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:23:13.350898 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.350896    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0775f20-4ea4-4e78-94f7-93d321b2239e-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:23:13.412982 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.412947    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266"]
Aug 04 18:23:13.415313 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.415289    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-16a9c503-76b96bf6f9-pv266"]
Aug 04 18:23:13.426970 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.426945    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m"]
Aug 04 18:23:13.431082 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:13.431061    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-16a9c503-5754bc75c7-lcw8m"]
Aug 04 18:23:14.094357 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:14.094265    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4"
Aug 04 18:23:14.094357 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:14.094284    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4" event={"ID":"d0775f20-4ea4-4e78-94f7-93d321b2239e","Type":"ContainerDied","Data":"25df52e6656d438fc1d1d4a17234989416e2d4af771f617714cb7c8daf5e5b51"}
Aug 04 18:23:14.094872 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:14.094379    2560 scope.go:117] "RemoveContainer" containerID="c9f5dc0cddab9098270ca97a5c4baade5044a52e75276579af90011760d2bc55"
Aug 04 18:23:14.116216 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:14.116191    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4"]
Aug 04 18:23:14.123260 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:14.123239    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-16a9c503-f9b85f8b4-8fwd4"]
Aug 04 18:23:14.354197 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:14.354115    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b985105a-50cb-4d88-8759-4afc9da58a4f" path="/var/lib/kubelet/pods/b985105a-50cb-4d88-8759-4afc9da58a4f/volumes"
Aug 04 18:23:14.354531 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:14.354516    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0775f20-4ea4-4e78-94f7-93d321b2239e" path="/var/lib/kubelet/pods/d0775f20-4ea4-4e78-94f7-93d321b2239e/volumes"
Aug 04 18:23:14.354879 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:14.354865    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="faa0ea61-8fa3-496e-8946-e2b03294907a" path="/var/lib/kubelet/pods/faa0ea61-8fa3-496e-8946-e2b03294907a/volumes"
Aug 04 18:23:16.971181 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:16.971146    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq"]
Aug 04 18:23:16.971533 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:16.971502    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faa0ea61-8fa3-496e-8946-e2b03294907a" containerName="payload-processing"
Aug 04 18:23:16.971533 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:16.971513    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="faa0ea61-8fa3-496e-8946-e2b03294907a" containerName="payload-processing"
Aug 04 18:23:16.971613 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:16.971535    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b985105a-50cb-4d88-8759-4afc9da58a4f" containerName="maas-api"
Aug 04 18:23:16.971613 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:16.971541    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b985105a-50cb-4d88-8759-4afc9da58a4f" containerName="maas-api"
Aug 04 18:23:16.971613 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:16.971554    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0775f20-4ea4-4e78-94f7-93d321b2239e" containerName="payload-pre-processing"
Aug 04 18:23:16.971613 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:16.971560    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0775f20-4ea4-4e78-94f7-93d321b2239e" containerName="payload-pre-processing"
Aug 04 18:23:16.971613 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:16.971614    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="faa0ea61-8fa3-496e-8946-e2b03294907a" containerName="payload-processing"
Aug 04 18:23:16.971759 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:16.971623    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0775f20-4ea4-4e78-94f7-93d321b2239e" containerName="payload-pre-processing"
Aug 04 18:23:16.971759 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:16.971629    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b985105a-50cb-4d88-8759-4afc9da58a4f" containerName="maas-api"
Aug 04 18:23:16.976247 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:16.976229    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq"
Aug 04 18:23:16.980227 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:16.980207    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-bcb508fd-serving-cert\""
Aug 04 18:23:17.042060 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:17.042028    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq"]
Aug 04 18:23:17.085458 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:17.085431    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9296a3b-0183-4743-a3e1-35ac79d724fa-openshift-service-ca\") pod \"maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq\" (UID: \"a9296a3b-0183-4743-a3e1-35ac79d724fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq"
Aug 04 18:23:17.085621 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:17.085468    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9296a3b-0183-4743-a3e1-35ac79d724fa-maas-api-tls\") pod \"maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq\" (UID: \"a9296a3b-0183-4743-a3e1-35ac79d724fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq"
Aug 04 18:23:17.085621 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:17.085512    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrt95\" (UniqueName: \"kubernetes.io/projected/a9296a3b-0183-4743-a3e1-35ac79d724fa-kube-api-access-nrt95\") pod \"maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq\" (UID: \"a9296a3b-0183-4743-a3e1-35ac79d724fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq"
Aug 04 18:23:17.186573 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:17.186530    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrt95\" (UniqueName: \"kubernetes.io/projected/a9296a3b-0183-4743-a3e1-35ac79d724fa-kube-api-access-nrt95\") pod \"maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq\" (UID: \"a9296a3b-0183-4743-a3e1-35ac79d724fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq"
Aug 04 18:23:17.186948 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:17.186642    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9296a3b-0183-4743-a3e1-35ac79d724fa-openshift-service-ca\") pod \"maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq\" (UID: \"a9296a3b-0183-4743-a3e1-35ac79d724fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq"
Aug 04 18:23:17.186948 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:17.186682    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9296a3b-0183-4743-a3e1-35ac79d724fa-maas-api-tls\") pod \"maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq\" (UID: \"a9296a3b-0183-4743-a3e1-35ac79d724fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq"
Aug 04 18:23:17.186948 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:23:17.186869    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-serving-cert: secret "maas-api-e2e-ait-bcb508fd-serving-cert" not found
Aug 04 18:23:17.187133 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:23:17.187029    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9296a3b-0183-4743-a3e1-35ac79d724fa-maas-api-tls podName:a9296a3b-0183-4743-a3e1-35ac79d724fa nodeName:}" failed. No retries permitted until 2026-08-04 18:23:17.686955577 +0000 UTC m=+1879.870115745 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a9296a3b-0183-4743-a3e1-35ac79d724fa-maas-api-tls") pod "maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq" (UID: "a9296a3b-0183-4743-a3e1-35ac79d724fa") : secret "maas-api-e2e-ait-bcb508fd-serving-cert" not found
Aug 04 18:23:17.187392 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:17.187371    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9296a3b-0183-4743-a3e1-35ac79d724fa-openshift-service-ca\") pod \"maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq\" (UID: \"a9296a3b-0183-4743-a3e1-35ac79d724fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq"
Aug 04 18:23:17.200471 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:17.200445    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrt95\" (UniqueName: \"kubernetes.io/projected/a9296a3b-0183-4743-a3e1-35ac79d724fa-kube-api-access-nrt95\") pod \"maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq\" (UID: \"a9296a3b-0183-4743-a3e1-35ac79d724fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq"
Aug 04 18:23:17.692125 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:17.692097    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9296a3b-0183-4743-a3e1-35ac79d724fa-maas-api-tls\") pod \"maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq\" (UID: \"a9296a3b-0183-4743-a3e1-35ac79d724fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq"
Aug 04 18:23:17.694569 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:17.694541    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9296a3b-0183-4743-a3e1-35ac79d724fa-maas-api-tls\") pod \"maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq\" (UID: \"a9296a3b-0183-4743-a3e1-35ac79d724fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq"
Aug 04 18:23:17.789400 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:17.789372    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz"]
Aug 04 18:23:17.793027 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:17.793001    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz"
Aug 04 18:23:17.795359 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:17.795340    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-bcb508fd-dockercfg-44lgp\""
Aug 04 18:23:17.795476 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:17.795340    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-bcb508fd\""
Aug 04 18:23:17.802274 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:17.802252    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz"]
Aug 04 18:23:17.829230 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:17.829205    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66"]
Aug 04 18:23:17.832713 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:17.832697    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66"
Aug 04 18:23:17.842876 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:17.842851    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66"]
Aug 04 18:23:17.886585 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:17.886554    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq"
Aug 04 18:23:17.893948 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:17.893921    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlqj9\" (UniqueName: \"kubernetes.io/projected/82975171-73eb-4a57-8e4a-effeb82bb97b-kube-api-access-dlqj9\") pod \"payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz\" (UID: \"82975171-73eb-4a57-8e4a-effeb82bb97b\") " pod="openshift-ingress/payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz"
Aug 04 18:23:17.894057 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:17.894033    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82975171-73eb-4a57-8e4a-effeb82bb97b-plugins-config-volume\") pod \"payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz\" (UID: \"82975171-73eb-4a57-8e4a-effeb82bb97b\") " pod="openshift-ingress/payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz"
Aug 04 18:23:17.995611 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:17.995575    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlqj9\" (UniqueName: \"kubernetes.io/projected/82975171-73eb-4a57-8e4a-effeb82bb97b-kube-api-access-dlqj9\") pod \"payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz\" (UID: \"82975171-73eb-4a57-8e4a-effeb82bb97b\") " pod="openshift-ingress/payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz"
Aug 04 18:23:17.995611 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:17.995630    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82975171-73eb-4a57-8e4a-effeb82bb97b-plugins-config-volume\") pod \"payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz\" (UID: \"82975171-73eb-4a57-8e4a-effeb82bb97b\") " pod="openshift-ingress/payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz"
Aug 04 18:23:17.996194 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:17.995676    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phws8\" (UniqueName: \"kubernetes.io/projected/b5419bfa-7f03-45fe-a808-d83c5ca6a895-kube-api-access-phws8\") pod \"payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66\" (UID: \"b5419bfa-7f03-45fe-a808-d83c5ca6a895\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66"
Aug 04 18:23:17.996194 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:17.995711    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5419bfa-7f03-45fe-a808-d83c5ca6a895-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66\" (UID: \"b5419bfa-7f03-45fe-a808-d83c5ca6a895\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66"
Aug 04 18:23:17.996480 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:17.996460    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82975171-73eb-4a57-8e4a-effeb82bb97b-plugins-config-volume\") pod \"payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz\" (UID: \"82975171-73eb-4a57-8e4a-effeb82bb97b\") " pod="openshift-ingress/payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz"
Aug 04 18:23:18.005520 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:18.005493    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlqj9\" (UniqueName: \"kubernetes.io/projected/82975171-73eb-4a57-8e4a-effeb82bb97b-kube-api-access-dlqj9\") pod \"payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz\" (UID: \"82975171-73eb-4a57-8e4a-effeb82bb97b\") " pod="openshift-ingress/payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz"
Aug 04 18:23:18.030341 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:18.030320    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq"]
Aug 04 18:23:18.032673 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:23:18.032640    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9296a3b_0183_4743_a3e1_35ac79d724fa.slice/crio-db85b1114c960e4da04734eccc7dbaf790faea4990bec6cd1c411271013c3bed WatchSource:0}: Error finding container db85b1114c960e4da04734eccc7dbaf790faea4990bec6cd1c411271013c3bed: Status 404 returned error can't find the container with id db85b1114c960e4da04734eccc7dbaf790faea4990bec6cd1c411271013c3bed
Aug 04 18:23:18.097293 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:18.097261    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phws8\" (UniqueName: \"kubernetes.io/projected/b5419bfa-7f03-45fe-a808-d83c5ca6a895-kube-api-access-phws8\") pod \"payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66\" (UID: \"b5419bfa-7f03-45fe-a808-d83c5ca6a895\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66"
Aug 04 18:23:18.097454 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:18.097312    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5419bfa-7f03-45fe-a808-d83c5ca6a895-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66\" (UID: \"b5419bfa-7f03-45fe-a808-d83c5ca6a895\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66"
Aug 04 18:23:18.098014 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:18.097966    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5419bfa-7f03-45fe-a808-d83c5ca6a895-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66\" (UID: \"b5419bfa-7f03-45fe-a808-d83c5ca6a895\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66"
Aug 04 18:23:18.104063 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:18.104043    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz"
Aug 04 18:23:18.106043 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:18.106024    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phws8\" (UniqueName: \"kubernetes.io/projected/b5419bfa-7f03-45fe-a808-d83c5ca6a895-kube-api-access-phws8\") pod \"payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66\" (UID: \"b5419bfa-7f03-45fe-a808-d83c5ca6a895\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66"
Aug 04 18:23:18.120254 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:18.120220    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq" event={"ID":"a9296a3b-0183-4743-a3e1-35ac79d724fa","Type":"ContainerStarted","Data":"db85b1114c960e4da04734eccc7dbaf790faea4990bec6cd1c411271013c3bed"}
Aug 04 18:23:18.142828 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:18.142799    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66"
Aug 04 18:23:18.272311 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:18.272279    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz"]
Aug 04 18:23:18.274188 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:23:18.274156    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82975171_73eb_4a57_8e4a_effeb82bb97b.slice/crio-75bd1887513dbaad35b6106e8541568cffb6474d982488fdc53ea5b724c70a3f WatchSource:0}: Error finding container 75bd1887513dbaad35b6106e8541568cffb6474d982488fdc53ea5b724c70a3f: Status 404 returned error can't find the container with id 75bd1887513dbaad35b6106e8541568cffb6474d982488fdc53ea5b724c70a3f
Aug 04 18:23:18.300702 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:18.300675    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66"]
Aug 04 18:23:18.302399 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:23:18.302372    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5419bfa_7f03_45fe_a808_d83c5ca6a895.slice/crio-54ea4139eca8b45ff13ba1998b4295334c94f494d584180bcafc81fdcfc5553b WatchSource:0}: Error finding container 54ea4139eca8b45ff13ba1998b4295334c94f494d584180bcafc81fdcfc5553b: Status 404 returned error can't find the container with id 54ea4139eca8b45ff13ba1998b4295334c94f494d584180bcafc81fdcfc5553b
Aug 04 18:23:19.128393 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:19.128354    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66" event={"ID":"b5419bfa-7f03-45fe-a808-d83c5ca6a895","Type":"ContainerStarted","Data":"10bd1ab96bd16af1067ad1f0789a0c252ced3ee5e8f7147dcc47f869c62d2904"}
Aug 04 18:23:19.128877 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:19.128452    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66" event={"ID":"b5419bfa-7f03-45fe-a808-d83c5ca6a895","Type":"ContainerStarted","Data":"54ea4139eca8b45ff13ba1998b4295334c94f494d584180bcafc81fdcfc5553b"}
Aug 04 18:23:19.128877 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:19.128514    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66"
Aug 04 18:23:19.130165 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:19.130138    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz" event={"ID":"82975171-73eb-4a57-8e4a-effeb82bb97b","Type":"ContainerStarted","Data":"1e64ac3e8d01da8a4803791effa509c39886bc61e6624dca02b9e7ed22b6258e"}
Aug 04 18:23:19.130302 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:19.130181    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz" event={"ID":"82975171-73eb-4a57-8e4a-effeb82bb97b","Type":"ContainerStarted","Data":"75bd1887513dbaad35b6106e8541568cffb6474d982488fdc53ea5b724c70a3f"}
Aug 04 18:23:19.130302 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:19.130206    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz"
Aug 04 18:23:19.131712 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:19.131687    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq" event={"ID":"a9296a3b-0183-4743-a3e1-35ac79d724fa","Type":"ContainerStarted","Data":"4da2c38507e4d5feb239a3e65d08546f93a4418fc639489cf584ef4779dd0725"}
Aug 04 18:23:19.131835 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:19.131821    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq"
Aug 04 18:23:19.151147 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:19.151105    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66" podStartSLOduration=2.151091797 podStartE2EDuration="2.151091797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:23:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:23:19.149325995 +0000 UTC m=+1881.332486185" watchObservedRunningTime="2026-08-04 18:23:19.151091797 +0000 UTC m=+1881.334251973"
Aug 04 18:23:19.187333 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:19.187288    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz" podStartSLOduration=2.1872748140000002 podStartE2EDuration="2.187274814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:23:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:23:19.186856568 +0000 UTC m=+1881.370016743" watchObservedRunningTime="2026-08-04 18:23:19.187274814 +0000 UTC m=+1881.370434990"
Aug 04 18:23:19.187941 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:19.187912    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq" podStartSLOduration=2.608468921 podStartE2EDuration="3.187904457s" totalImagesPullingTime="579.435536ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:23:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:23:19.168542685 +0000 UTC m=+1881.351702863" watchObservedRunningTime="2026-08-04 18:23:19.187904457 +0000 UTC m=+1881.371064633"
Aug 04 18:23:25.148035 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:25.148002    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq"
Aug 04 18:23:30.143730 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:30.143685    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66" podUID="b5419bfa-7f03-45fe-a808-d83c5ca6a895" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.55:9004: connect: connection refused"
Aug 04 18:23:30.144227 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:30.144146    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz"
Aug 04 18:23:40.142498 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:40.142452    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66"
Aug 04 18:23:47.814412 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:47.814373    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq"]
Aug 04 18:23:47.814947 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:47.814615    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq" podUID="a9296a3b-0183-4743-a3e1-35ac79d724fa" containerName="maas-api" containerID="cri-o://4da2c38507e4d5feb239a3e65d08546f93a4418fc639489cf584ef4779dd0725" gracePeriod=30
Aug 04 18:23:47.886928 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:47.886898    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz"]
Aug 04 18:23:47.887212 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:47.887164    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz" podUID="82975171-73eb-4a57-8e4a-effeb82bb97b" containerName="payload-processing" containerID="cri-o://1e64ac3e8d01da8a4803791effa509c39886bc61e6624dca02b9e7ed22b6258e" gracePeriod=30
Aug 04 18:23:47.892307 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:47.892280    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66"]
Aug 04 18:23:47.892507 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:47.892485    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66" podUID="b5419bfa-7f03-45fe-a808-d83c5ca6a895" containerName="payload-pre-processing" containerID="cri-o://10bd1ab96bd16af1067ad1f0789a0c252ced3ee5e8f7147dcc47f869c62d2904" gracePeriod=30
Aug 04 18:23:48.157043 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.157014    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq"
Aug 04 18:23:48.176849 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.176824    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz"
Aug 04 18:23:48.184716 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.184694    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66"
Aug 04 18:23:48.246014 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.245960    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9296a3b-0183-4743-a3e1-35ac79d724fa-openshift-service-ca\") pod \"a9296a3b-0183-4743-a3e1-35ac79d724fa\" (UID: \"a9296a3b-0183-4743-a3e1-35ac79d724fa\") "
Aug 04 18:23:48.246149 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.246034    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5419bfa-7f03-45fe-a808-d83c5ca6a895-plugins-config-volume\") pod \"b5419bfa-7f03-45fe-a808-d83c5ca6a895\" (UID: \"b5419bfa-7f03-45fe-a808-d83c5ca6a895\") "
Aug 04 18:23:48.246149 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.246051    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phws8\" (UniqueName: \"kubernetes.io/projected/b5419bfa-7f03-45fe-a808-d83c5ca6a895-kube-api-access-phws8\") pod \"b5419bfa-7f03-45fe-a808-d83c5ca6a895\" (UID: \"b5419bfa-7f03-45fe-a808-d83c5ca6a895\") "
Aug 04 18:23:48.246149 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.246084    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrt95\" (UniqueName: \"kubernetes.io/projected/a9296a3b-0183-4743-a3e1-35ac79d724fa-kube-api-access-nrt95\") pod \"a9296a3b-0183-4743-a3e1-35ac79d724fa\" (UID: \"a9296a3b-0183-4743-a3e1-35ac79d724fa\") "
Aug 04 18:23:48.246149 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.246102    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlqj9\" (UniqueName: \"kubernetes.io/projected/82975171-73eb-4a57-8e4a-effeb82bb97b-kube-api-access-dlqj9\") pod \"82975171-73eb-4a57-8e4a-effeb82bb97b\" (UID: \"82975171-73eb-4a57-8e4a-effeb82bb97b\") "
Aug 04 18:23:48.246149 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.246121    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82975171-73eb-4a57-8e4a-effeb82bb97b-plugins-config-volume\") pod \"82975171-73eb-4a57-8e4a-effeb82bb97b\" (UID: \"82975171-73eb-4a57-8e4a-effeb82bb97b\") "
Aug 04 18:23:48.246421 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.246157    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9296a3b-0183-4743-a3e1-35ac79d724fa-maas-api-tls\") pod \"a9296a3b-0183-4743-a3e1-35ac79d724fa\" (UID: \"a9296a3b-0183-4743-a3e1-35ac79d724fa\") "
Aug 04 18:23:48.246481 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.246428    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5419bfa-7f03-45fe-a808-d83c5ca6a895-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b5419bfa-7f03-45fe-a808-d83c5ca6a895" (UID: "b5419bfa-7f03-45fe-a808-d83c5ca6a895"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:23:48.246533 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.246506    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82975171-73eb-4a57-8e4a-effeb82bb97b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "82975171-73eb-4a57-8e4a-effeb82bb97b" (UID: "82975171-73eb-4a57-8e4a-effeb82bb97b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:23:48.246645 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.246608    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9296a3b-0183-4743-a3e1-35ac79d724fa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a9296a3b-0183-4743-a3e1-35ac79d724fa" (UID: "a9296a3b-0183-4743-a3e1-35ac79d724fa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:23:48.248390 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.248365    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82975171-73eb-4a57-8e4a-effeb82bb97b-kube-api-access-dlqj9" (OuterVolumeSpecName: "kube-api-access-dlqj9") pod "82975171-73eb-4a57-8e4a-effeb82bb97b" (UID: "82975171-73eb-4a57-8e4a-effeb82bb97b"). InnerVolumeSpecName "kube-api-access-dlqj9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:23:48.248472 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.248387    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9296a3b-0183-4743-a3e1-35ac79d724fa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a9296a3b-0183-4743-a3e1-35ac79d724fa" (UID: "a9296a3b-0183-4743-a3e1-35ac79d724fa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:23:48.248815 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.248797    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5419bfa-7f03-45fe-a808-d83c5ca6a895-kube-api-access-phws8" (OuterVolumeSpecName: "kube-api-access-phws8") pod "b5419bfa-7f03-45fe-a808-d83c5ca6a895" (UID: "b5419bfa-7f03-45fe-a808-d83c5ca6a895"). InnerVolumeSpecName "kube-api-access-phws8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:23:48.248895 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.248809    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9296a3b-0183-4743-a3e1-35ac79d724fa-kube-api-access-nrt95" (OuterVolumeSpecName: "kube-api-access-nrt95") pod "a9296a3b-0183-4743-a3e1-35ac79d724fa" (UID: "a9296a3b-0183-4743-a3e1-35ac79d724fa"). InnerVolumeSpecName "kube-api-access-nrt95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:23:48.284399 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.284371    2560 generic.go:356] "Generic (PLEG): container finished" podID="a9296a3b-0183-4743-a3e1-35ac79d724fa" containerID="4da2c38507e4d5feb239a3e65d08546f93a4418fc639489cf584ef4779dd0725" exitCode=0
Aug 04 18:23:48.284499 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.284413    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq"
Aug 04 18:23:48.284499 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.284455    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq" event={"ID":"a9296a3b-0183-4743-a3e1-35ac79d724fa","Type":"ContainerDied","Data":"4da2c38507e4d5feb239a3e65d08546f93a4418fc639489cf584ef4779dd0725"}
Aug 04 18:23:48.284578 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.284506    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq" event={"ID":"a9296a3b-0183-4743-a3e1-35ac79d724fa","Type":"ContainerDied","Data":"db85b1114c960e4da04734eccc7dbaf790faea4990bec6cd1c411271013c3bed"}
Aug 04 18:23:48.284578 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.284536    2560 scope.go:117] "RemoveContainer" containerID="4da2c38507e4d5feb239a3e65d08546f93a4418fc639489cf584ef4779dd0725"
Aug 04 18:23:48.285844 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.285822    2560 generic.go:356] "Generic (PLEG): container finished" podID="b5419bfa-7f03-45fe-a808-d83c5ca6a895" containerID="10bd1ab96bd16af1067ad1f0789a0c252ced3ee5e8f7147dcc47f869c62d2904" exitCode=0
Aug 04 18:23:48.285946 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.285868    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66"
Aug 04 18:23:48.285946 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.285893    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66" event={"ID":"b5419bfa-7f03-45fe-a808-d83c5ca6a895","Type":"ContainerDied","Data":"10bd1ab96bd16af1067ad1f0789a0c252ced3ee5e8f7147dcc47f869c62d2904"}
Aug 04 18:23:48.285946 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.285937    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66" event={"ID":"b5419bfa-7f03-45fe-a808-d83c5ca6a895","Type":"ContainerDied","Data":"54ea4139eca8b45ff13ba1998b4295334c94f494d584180bcafc81fdcfc5553b"}
Aug 04 18:23:48.287260 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.287239    2560 generic.go:356] "Generic (PLEG): container finished" podID="82975171-73eb-4a57-8e4a-effeb82bb97b" containerID="1e64ac3e8d01da8a4803791effa509c39886bc61e6624dca02b9e7ed22b6258e" exitCode=0
Aug 04 18:23:48.287374 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.287267    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz" event={"ID":"82975171-73eb-4a57-8e4a-effeb82bb97b","Type":"ContainerDied","Data":"1e64ac3e8d01da8a4803791effa509c39886bc61e6624dca02b9e7ed22b6258e"}
Aug 04 18:23:48.287374 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.287293    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz"
Aug 04 18:23:48.287374 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.287296    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz" event={"ID":"82975171-73eb-4a57-8e4a-effeb82bb97b","Type":"ContainerDied","Data":"75bd1887513dbaad35b6106e8541568cffb6474d982488fdc53ea5b724c70a3f"}
Aug 04 18:23:48.298616 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.297611    2560 scope.go:117] "RemoveContainer" containerID="4da2c38507e4d5feb239a3e65d08546f93a4418fc639489cf584ef4779dd0725"
Aug 04 18:23:48.298616 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:23:48.297941    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4da2c38507e4d5feb239a3e65d08546f93a4418fc639489cf584ef4779dd0725\": container with ID starting with 4da2c38507e4d5feb239a3e65d08546f93a4418fc639489cf584ef4779dd0725 not found: ID does not exist" containerID="4da2c38507e4d5feb239a3e65d08546f93a4418fc639489cf584ef4779dd0725"
Aug 04 18:23:48.298616 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.297978    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4da2c38507e4d5feb239a3e65d08546f93a4418fc639489cf584ef4779dd0725"} err="failed to get container status \"4da2c38507e4d5feb239a3e65d08546f93a4418fc639489cf584ef4779dd0725\": rpc error: code = NotFound desc = could not find container \"4da2c38507e4d5feb239a3e65d08546f93a4418fc639489cf584ef4779dd0725\": container with ID starting with 4da2c38507e4d5feb239a3e65d08546f93a4418fc639489cf584ef4779dd0725 not found: ID does not exist"
Aug 04 18:23:48.298616 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.298021    2560 scope.go:117] "RemoveContainer" containerID="10bd1ab96bd16af1067ad1f0789a0c252ced3ee5e8f7147dcc47f869c62d2904"
Aug 04 18:23:48.309866 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.309846    2560 scope.go:117] "RemoveContainer" containerID="10bd1ab96bd16af1067ad1f0789a0c252ced3ee5e8f7147dcc47f869c62d2904"
Aug 04 18:23:48.310160 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:23:48.310138    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10bd1ab96bd16af1067ad1f0789a0c252ced3ee5e8f7147dcc47f869c62d2904\": container with ID starting with 10bd1ab96bd16af1067ad1f0789a0c252ced3ee5e8f7147dcc47f869c62d2904 not found: ID does not exist" containerID="10bd1ab96bd16af1067ad1f0789a0c252ced3ee5e8f7147dcc47f869c62d2904"
Aug 04 18:23:48.310230 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.310167    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10bd1ab96bd16af1067ad1f0789a0c252ced3ee5e8f7147dcc47f869c62d2904"} err="failed to get container status \"10bd1ab96bd16af1067ad1f0789a0c252ced3ee5e8f7147dcc47f869c62d2904\": rpc error: code = NotFound desc = could not find container \"10bd1ab96bd16af1067ad1f0789a0c252ced3ee5e8f7147dcc47f869c62d2904\": container with ID starting with 10bd1ab96bd16af1067ad1f0789a0c252ced3ee5e8f7147dcc47f869c62d2904 not found: ID does not exist"
Aug 04 18:23:48.310230 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.310183    2560 scope.go:117] "RemoveContainer" containerID="1e64ac3e8d01da8a4803791effa509c39886bc61e6624dca02b9e7ed22b6258e"
Aug 04 18:23:48.318813 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.318795    2560 scope.go:117] "RemoveContainer" containerID="1e64ac3e8d01da8a4803791effa509c39886bc61e6624dca02b9e7ed22b6258e"
Aug 04 18:23:48.319077 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:23:48.319047    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e64ac3e8d01da8a4803791effa509c39886bc61e6624dca02b9e7ed22b6258e\": container with ID starting with 1e64ac3e8d01da8a4803791effa509c39886bc61e6624dca02b9e7ed22b6258e not found: ID does not exist" containerID="1e64ac3e8d01da8a4803791effa509c39886bc61e6624dca02b9e7ed22b6258e"
Aug 04 18:23:48.319138 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.319087    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e64ac3e8d01da8a4803791effa509c39886bc61e6624dca02b9e7ed22b6258e"} err="failed to get container status \"1e64ac3e8d01da8a4803791effa509c39886bc61e6624dca02b9e7ed22b6258e\": rpc error: code = NotFound desc = could not find container \"1e64ac3e8d01da8a4803791effa509c39886bc61e6624dca02b9e7ed22b6258e\": container with ID starting with 1e64ac3e8d01da8a4803791effa509c39886bc61e6624dca02b9e7ed22b6258e not found: ID does not exist"
Aug 04 18:23:48.319614 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.319597    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq"]
Aug 04 18:23:48.322660 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.322640    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bcb508fd-7df8f7fb5b-q9grq"]
Aug 04 18:23:48.334142 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.334122    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66"]
Aug 04 18:23:48.338421 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.338366    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bcb508fd-55fc8d578d-mmd66"]
Aug 04 18:23:48.347911 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.347890    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5419bfa-7f03-45fe-a808-d83c5ca6a895-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:23:48.348045 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.348031    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phws8\" (UniqueName: \"kubernetes.io/projected/b5419bfa-7f03-45fe-a808-d83c5ca6a895-kube-api-access-phws8\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:23:48.348107 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.348053    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrt95\" (UniqueName: \"kubernetes.io/projected/a9296a3b-0183-4743-a3e1-35ac79d724fa-kube-api-access-nrt95\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:23:48.348198 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.348147    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlqj9\" (UniqueName: \"kubernetes.io/projected/82975171-73eb-4a57-8e4a-effeb82bb97b-kube-api-access-dlqj9\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:23:48.348198 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.348168    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82975171-73eb-4a57-8e4a-effeb82bb97b-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:23:48.348198 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.348181    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9296a3b-0183-4743-a3e1-35ac79d724fa-maas-api-tls\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:23:48.348198 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.348195    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9296a3b-0183-4743-a3e1-35ac79d724fa-openshift-service-ca\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:23:48.353884 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.353864    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9296a3b-0183-4743-a3e1-35ac79d724fa" path="/var/lib/kubelet/pods/a9296a3b-0183-4743-a3e1-35ac79d724fa/volumes"
Aug 04 18:23:48.354224 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.354211    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5419bfa-7f03-45fe-a808-d83c5ca6a895" path="/var/lib/kubelet/pods/b5419bfa-7f03-45fe-a808-d83c5ca6a895/volumes"
Aug 04 18:23:48.354442 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.354431    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz"]
Aug 04 18:23:48.354480 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:48.354454    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bcb508fd-6cf7bfc5f6-vhljz"]
Aug 04 18:23:50.353106 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:50.353062    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82975171-73eb-4a57-8e4a-effeb82bb97b" path="/var/lib/kubelet/pods/82975171-73eb-4a57-8e4a-effeb82bb97b/volumes"
Aug 04 18:23:53.199116 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:53.199088    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-5768665896-7mcnm"]
Aug 04 18:23:53.199473 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:53.199457    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82975171-73eb-4a57-8e4a-effeb82bb97b" containerName="payload-processing"
Aug 04 18:23:53.199473 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:53.199470    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="82975171-73eb-4a57-8e4a-effeb82bb97b" containerName="payload-processing"
Aug 04 18:23:53.199551 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:53.199483    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5419bfa-7f03-45fe-a808-d83c5ca6a895" containerName="payload-pre-processing"
Aug 04 18:23:53.199551 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:53.199488    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5419bfa-7f03-45fe-a808-d83c5ca6a895" containerName="payload-pre-processing"
Aug 04 18:23:53.199551 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:53.199514    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9296a3b-0183-4743-a3e1-35ac79d724fa" containerName="maas-api"
Aug 04 18:23:53.199551 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:53.199520    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9296a3b-0183-4743-a3e1-35ac79d724fa" containerName="maas-api"
Aug 04 18:23:53.199665 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:53.199592    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9296a3b-0183-4743-a3e1-35ac79d724fa" containerName="maas-api"
Aug 04 18:23:53.199665 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:53.199600    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="82975171-73eb-4a57-8e4a-effeb82bb97b" containerName="payload-processing"
Aug 04 18:23:53.199665 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:53.199608    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5419bfa-7f03-45fe-a808-d83c5ca6a895" containerName="payload-pre-processing"
Aug 04 18:23:53.203853 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:53.203833    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-5768665896-7mcnm"
Aug 04 18:23:53.206537 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:53.206516    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-07becd73-serving-cert\""
Aug 04 18:23:53.220623 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:53.220602    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-5768665896-7mcnm"]
Aug 04 18:23:53.291112 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:53.291082    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca391bc6-11eb-4a08-9930-1826935ee5e2-openshift-service-ca\") pod \"maas-api-e2e-derive-07becd73-5768665896-7mcnm\" (UID: \"ca391bc6-11eb-4a08-9930-1826935ee5e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-5768665896-7mcnm"
Aug 04 18:23:53.291314 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:53.291116    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw4sl\" (UniqueName: \"kubernetes.io/projected/ca391bc6-11eb-4a08-9930-1826935ee5e2-kube-api-access-gw4sl\") pod \"maas-api-e2e-derive-07becd73-5768665896-7mcnm\" (UID: \"ca391bc6-11eb-4a08-9930-1826935ee5e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-5768665896-7mcnm"
Aug 04 18:23:53.291314 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:53.291147    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca391bc6-11eb-4a08-9930-1826935ee5e2-maas-api-tls\") pod \"maas-api-e2e-derive-07becd73-5768665896-7mcnm\" (UID: \"ca391bc6-11eb-4a08-9930-1826935ee5e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-5768665896-7mcnm"
Aug 04 18:23:53.392478 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:53.392450    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca391bc6-11eb-4a08-9930-1826935ee5e2-openshift-service-ca\") pod \"maas-api-e2e-derive-07becd73-5768665896-7mcnm\" (UID: \"ca391bc6-11eb-4a08-9930-1826935ee5e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-5768665896-7mcnm"
Aug 04 18:23:53.392621 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:53.392502    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw4sl\" (UniqueName: \"kubernetes.io/projected/ca391bc6-11eb-4a08-9930-1826935ee5e2-kube-api-access-gw4sl\") pod \"maas-api-e2e-derive-07becd73-5768665896-7mcnm\" (UID: \"ca391bc6-11eb-4a08-9930-1826935ee5e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-5768665896-7mcnm"
Aug 04 18:23:53.392621 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:53.392555    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca391bc6-11eb-4a08-9930-1826935ee5e2-maas-api-tls\") pod \"maas-api-e2e-derive-07becd73-5768665896-7mcnm\" (UID: \"ca391bc6-11eb-4a08-9930-1826935ee5e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-5768665896-7mcnm"
Aug 04 18:23:53.392744 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:23:53.392686    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-serving-cert: secret "maas-api-e2e-derive-07becd73-serving-cert" not found
Aug 04 18:23:53.392797 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:23:53.392767    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca391bc6-11eb-4a08-9930-1826935ee5e2-maas-api-tls podName:ca391bc6-11eb-4a08-9930-1826935ee5e2 nodeName:}" failed. No retries permitted until 2026-08-04 18:23:53.892745571 +0000 UTC m=+1916.075905728 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ca391bc6-11eb-4a08-9930-1826935ee5e2-maas-api-tls") pod "maas-api-e2e-derive-07becd73-5768665896-7mcnm" (UID: "ca391bc6-11eb-4a08-9930-1826935ee5e2") : secret "maas-api-e2e-derive-07becd73-serving-cert" not found
Aug 04 18:23:53.393228 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:53.393209    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca391bc6-11eb-4a08-9930-1826935ee5e2-openshift-service-ca\") pod \"maas-api-e2e-derive-07becd73-5768665896-7mcnm\" (UID: \"ca391bc6-11eb-4a08-9930-1826935ee5e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-5768665896-7mcnm"
Aug 04 18:23:53.401681 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:53.401659    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw4sl\" (UniqueName: \"kubernetes.io/projected/ca391bc6-11eb-4a08-9930-1826935ee5e2-kube-api-access-gw4sl\") pod \"maas-api-e2e-derive-07becd73-5768665896-7mcnm\" (UID: \"ca391bc6-11eb-4a08-9930-1826935ee5e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-5768665896-7mcnm"
Aug 04 18:23:53.896982 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:53.896950    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca391bc6-11eb-4a08-9930-1826935ee5e2-maas-api-tls\") pod \"maas-api-e2e-derive-07becd73-5768665896-7mcnm\" (UID: \"ca391bc6-11eb-4a08-9930-1826935ee5e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-5768665896-7mcnm"
Aug 04 18:23:53.899413 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:53.899385    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca391bc6-11eb-4a08-9930-1826935ee5e2-maas-api-tls\") pod \"maas-api-e2e-derive-07becd73-5768665896-7mcnm\" (UID: \"ca391bc6-11eb-4a08-9930-1826935ee5e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-5768665896-7mcnm"
Aug 04 18:23:53.990859 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:53.990829    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-07becd73-b4d888598-jz9br"]
Aug 04 18:23:53.994414 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:53.994399    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-07becd73-b4d888598-jz9br"
Aug 04 18:23:53.997186 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:53.997159    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-07becd73\""
Aug 04 18:23:53.997433 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:53.997409    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-07becd73-dockercfg-nptsf\""
Aug 04 18:23:54.007899 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:54.007870    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-07becd73-b4d888598-jz9br"]
Aug 04 18:23:54.046013 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:54.045963    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s"]
Aug 04 18:23:54.049296 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:54.049282    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s"
Aug 04 18:23:54.059635 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:54.059611    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s"]
Aug 04 18:23:54.098770 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:54.098744    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw6hg\" (UniqueName: \"kubernetes.io/projected/7c575131-5aa9-41c6-908a-20461858ee00-kube-api-access-cw6hg\") pod \"payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s\" (UID: \"7c575131-5aa9-41c6-908a-20461858ee00\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s"
Aug 04 18:23:54.098877 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:54.098812    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ec7ced6-04a1-42fc-9a0d-c8fb42531079-plugins-config-volume\") pod \"payload-processing-e2e-derive-07becd73-b4d888598-jz9br\" (UID: \"4ec7ced6-04a1-42fc-9a0d-c8fb42531079\") " pod="openshift-ingress/payload-processing-e2e-derive-07becd73-b4d888598-jz9br"
Aug 04 18:23:54.098877 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:54.098842    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c575131-5aa9-41c6-908a-20461858ee00-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s\" (UID: \"7c575131-5aa9-41c6-908a-20461858ee00\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s"
Aug 04 18:23:54.098877 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:54.098859    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6vg4\" (UniqueName: \"kubernetes.io/projected/4ec7ced6-04a1-42fc-9a0d-c8fb42531079-kube-api-access-c6vg4\") pod \"payload-processing-e2e-derive-07becd73-b4d888598-jz9br\" (UID: \"4ec7ced6-04a1-42fc-9a0d-c8fb42531079\") " pod="openshift-ingress/payload-processing-e2e-derive-07becd73-b4d888598-jz9br"
Aug 04 18:23:54.114690 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:54.114667    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-5768665896-7mcnm"
Aug 04 18:23:54.200089 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:54.200056    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ec7ced6-04a1-42fc-9a0d-c8fb42531079-plugins-config-volume\") pod \"payload-processing-e2e-derive-07becd73-b4d888598-jz9br\" (UID: \"4ec7ced6-04a1-42fc-9a0d-c8fb42531079\") " pod="openshift-ingress/payload-processing-e2e-derive-07becd73-b4d888598-jz9br"
Aug 04 18:23:54.200526 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:54.200101    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c575131-5aa9-41c6-908a-20461858ee00-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s\" (UID: \"7c575131-5aa9-41c6-908a-20461858ee00\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s"
Aug 04 18:23:54.200526 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:54.200129    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6vg4\" (UniqueName: \"kubernetes.io/projected/4ec7ced6-04a1-42fc-9a0d-c8fb42531079-kube-api-access-c6vg4\") pod \"payload-processing-e2e-derive-07becd73-b4d888598-jz9br\" (UID: \"4ec7ced6-04a1-42fc-9a0d-c8fb42531079\") " pod="openshift-ingress/payload-processing-e2e-derive-07becd73-b4d888598-jz9br"
Aug 04 18:23:54.200526 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:54.200225    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cw6hg\" (UniqueName: \"kubernetes.io/projected/7c575131-5aa9-41c6-908a-20461858ee00-kube-api-access-cw6hg\") pod \"payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s\" (UID: \"7c575131-5aa9-41c6-908a-20461858ee00\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s"
Aug 04 18:23:54.201319 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:54.200922    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ec7ced6-04a1-42fc-9a0d-c8fb42531079-plugins-config-volume\") pod \"payload-processing-e2e-derive-07becd73-b4d888598-jz9br\" (UID: \"4ec7ced6-04a1-42fc-9a0d-c8fb42531079\") " pod="openshift-ingress/payload-processing-e2e-derive-07becd73-b4d888598-jz9br"
Aug 04 18:23:54.201319 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:54.200970    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c575131-5aa9-41c6-908a-20461858ee00-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s\" (UID: \"7c575131-5aa9-41c6-908a-20461858ee00\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s"
Aug 04 18:23:54.210153 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:54.210098    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6vg4\" (UniqueName: \"kubernetes.io/projected/4ec7ced6-04a1-42fc-9a0d-c8fb42531079-kube-api-access-c6vg4\") pod \"payload-processing-e2e-derive-07becd73-b4d888598-jz9br\" (UID: \"4ec7ced6-04a1-42fc-9a0d-c8fb42531079\") " pod="openshift-ingress/payload-processing-e2e-derive-07becd73-b4d888598-jz9br"
Aug 04 18:23:54.210681 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:54.210657    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw6hg\" (UniqueName: \"kubernetes.io/projected/7c575131-5aa9-41c6-908a-20461858ee00-kube-api-access-cw6hg\") pod \"payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s\" (UID: \"7c575131-5aa9-41c6-908a-20461858ee00\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s"
Aug 04 18:23:54.254725 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:54.254699    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-5768665896-7mcnm"]
Aug 04 18:23:54.256969 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:23:54.256942    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca391bc6_11eb_4a08_9930_1826935ee5e2.slice/crio-8f5ff140eea850ebaa7c2ecf56ce6f2d407ddb69a20140180f997db7ec061b48 WatchSource:0}: Error finding container 8f5ff140eea850ebaa7c2ecf56ce6f2d407ddb69a20140180f997db7ec061b48: Status 404 returned error can't find the container with id 8f5ff140eea850ebaa7c2ecf56ce6f2d407ddb69a20140180f997db7ec061b48
Aug 04 18:23:54.307478 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:54.307452    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-07becd73-b4d888598-jz9br"
Aug 04 18:23:54.321059 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:54.321026    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-5768665896-7mcnm" event={"ID":"ca391bc6-11eb-4a08-9930-1826935ee5e2","Type":"ContainerStarted","Data":"8f5ff140eea850ebaa7c2ecf56ce6f2d407ddb69a20140180f997db7ec061b48"}
Aug 04 18:23:54.359484 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:54.359366    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s"
Aug 04 18:23:54.456425 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:54.456403    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-07becd73-b4d888598-jz9br"]
Aug 04 18:23:54.458384 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:23:54.458339    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ec7ced6_04a1_42fc_9a0d_c8fb42531079.slice/crio-8b1572bbbdc1e7a2244c37297d168247cd973f4c2978eece4e0fd9fd877579fb WatchSource:0}: Error finding container 8b1572bbbdc1e7a2244c37297d168247cd973f4c2978eece4e0fd9fd877579fb: Status 404 returned error can't find the container with id 8b1572bbbdc1e7a2244c37297d168247cd973f4c2978eece4e0fd9fd877579fb
Aug 04 18:23:54.540044 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:54.540019    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s"]
Aug 04 18:23:54.542310 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:23:54.542284    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c575131_5aa9_41c6_908a_20461858ee00.slice/crio-feed9a1a1e023d30ea47949992174958a2676cfe3ae11a4b667f6a7deaa3b969 WatchSource:0}: Error finding container feed9a1a1e023d30ea47949992174958a2676cfe3ae11a4b667f6a7deaa3b969: Status 404 returned error can't find the container with id feed9a1a1e023d30ea47949992174958a2676cfe3ae11a4b667f6a7deaa3b969
Aug 04 18:23:55.327438 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:55.327386    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-5768665896-7mcnm" event={"ID":"ca391bc6-11eb-4a08-9930-1826935ee5e2","Type":"ContainerStarted","Data":"4f40e067679e267984050b9829155c79bd0f51013598a046b535fd0c93aa26cb"}
Aug 04 18:23:55.327897 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:55.327452    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-5768665896-7mcnm"
Aug 04 18:23:55.329054 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:55.329025    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-07becd73-b4d888598-jz9br" event={"ID":"4ec7ced6-04a1-42fc-9a0d-c8fb42531079","Type":"ContainerStarted","Data":"7493e1e318028b858f9b07556ef057c49eafbbd640ff3244aa3b8bfbf81f5ee1"}
Aug 04 18:23:55.329211 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:55.329071    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-07becd73-b4d888598-jz9br" event={"ID":"4ec7ced6-04a1-42fc-9a0d-c8fb42531079","Type":"ContainerStarted","Data":"8b1572bbbdc1e7a2244c37297d168247cd973f4c2978eece4e0fd9fd877579fb"}
Aug 04 18:23:55.329211 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:55.329102    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-07becd73-b4d888598-jz9br"
Aug 04 18:23:55.330577 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:55.330554    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s" event={"ID":"7c575131-5aa9-41c6-908a-20461858ee00","Type":"ContainerStarted","Data":"3412ad3c6aa8f67dee2f18574a65f3b44ae5339d662ba98f4724b868df357728"}
Aug 04 18:23:55.330728 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:55.330600    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s"
Aug 04 18:23:55.330728 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:55.330624    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s" event={"ID":"7c575131-5aa9-41c6-908a-20461858ee00","Type":"ContainerStarted","Data":"feed9a1a1e023d30ea47949992174958a2676cfe3ae11a4b667f6a7deaa3b969"}
Aug 04 18:23:55.347195 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:55.347152    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-5768665896-7mcnm" podStartSLOduration=2.038937957 podStartE2EDuration="2.347138653s" totalImagesPullingTime="308.200696ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:23:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:23:55.346118847 +0000 UTC m=+1917.529279027" watchObservedRunningTime="2026-08-04 18:23:55.347138653 +0000 UTC m=+1917.530298829"
Aug 04 18:23:55.363317 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:55.363281    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-07becd73-b4d888598-jz9br" podStartSLOduration=2.363267749 podStartE2EDuration="2.363267749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:23:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:23:55.361958959 +0000 UTC m=+1917.545119135" watchObservedRunningTime="2026-08-04 18:23:55.363267749 +0000 UTC m=+1917.546427924"
Aug 04 18:23:55.378492 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:23:55.378449    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s" podStartSLOduration=1.378437073 podStartE2EDuration="1.378437073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:23:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:23:55.377894518 +0000 UTC m=+1917.561054696" watchObservedRunningTime="2026-08-04 18:23:55.378437073 +0000 UTC m=+1917.561597256"
Aug 04 18:24:01.342126 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:01.342087    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-5768665896-7mcnm"
Aug 04 18:24:06.337708 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:06.337676    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s"
Aug 04 18:24:06.338114 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:06.337862    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-07becd73-b4d888598-jz9br"
Aug 04 18:24:23.933607 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:23.933572    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-5768665896-7mcnm"]
Aug 04 18:24:23.934203 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:23.933806    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-5768665896-7mcnm" podUID="ca391bc6-11eb-4a08-9930-1826935ee5e2" containerName="maas-api" containerID="cri-o://4f40e067679e267984050b9829155c79bd0f51013598a046b535fd0c93aa26cb" gracePeriod=30
Aug 04 18:24:24.069018 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.068972    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-07becd73-b4d888598-jz9br"]
Aug 04 18:24:24.069293 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.069262    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-07becd73-b4d888598-jz9br" podUID="4ec7ced6-04a1-42fc-9a0d-c8fb42531079" containerName="payload-processing" containerID="cri-o://7493e1e318028b858f9b07556ef057c49eafbbd640ff3244aa3b8bfbf81f5ee1" gracePeriod=30
Aug 04 18:24:24.094198 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.094061    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s"]
Aug 04 18:24:24.094403 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.094378    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s" podUID="7c575131-5aa9-41c6-908a-20461858ee00" containerName="payload-pre-processing" containerID="cri-o://3412ad3c6aa8f67dee2f18574a65f3b44ae5339d662ba98f4724b868df357728" gracePeriod=30
Aug 04 18:24:24.227648 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.227616    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-5768665896-7mcnm"
Aug 04 18:24:24.368418 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.368386    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca391bc6-11eb-4a08-9930-1826935ee5e2-maas-api-tls\") pod \"ca391bc6-11eb-4a08-9930-1826935ee5e2\" (UID: \"ca391bc6-11eb-4a08-9930-1826935ee5e2\") "
Aug 04 18:24:24.368600 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.368437    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gw4sl\" (UniqueName: \"kubernetes.io/projected/ca391bc6-11eb-4a08-9930-1826935ee5e2-kube-api-access-gw4sl\") pod \"ca391bc6-11eb-4a08-9930-1826935ee5e2\" (UID: \"ca391bc6-11eb-4a08-9930-1826935ee5e2\") "
Aug 04 18:24:24.368600 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.368467    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca391bc6-11eb-4a08-9930-1826935ee5e2-openshift-service-ca\") pod \"ca391bc6-11eb-4a08-9930-1826935ee5e2\" (UID: \"ca391bc6-11eb-4a08-9930-1826935ee5e2\") "
Aug 04 18:24:24.369138 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.369110    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca391bc6-11eb-4a08-9930-1826935ee5e2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ca391bc6-11eb-4a08-9930-1826935ee5e2" (UID: "ca391bc6-11eb-4a08-9930-1826935ee5e2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:24:24.372899 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.372872    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca391bc6-11eb-4a08-9930-1826935ee5e2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ca391bc6-11eb-4a08-9930-1826935ee5e2" (UID: "ca391bc6-11eb-4a08-9930-1826935ee5e2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:24:24.373416 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.373365    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca391bc6-11eb-4a08-9930-1826935ee5e2-kube-api-access-gw4sl" (OuterVolumeSpecName: "kube-api-access-gw4sl") pod "ca391bc6-11eb-4a08-9930-1826935ee5e2" (UID: "ca391bc6-11eb-4a08-9930-1826935ee5e2"). InnerVolumeSpecName "kube-api-access-gw4sl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:24:24.387440 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.387418    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-07becd73-b4d888598-jz9br"
Aug 04 18:24:24.392739 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.392717    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s"
Aug 04 18:24:24.469842 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.469752    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c575131-5aa9-41c6-908a-20461858ee00-plugins-config-volume\") pod \"7c575131-5aa9-41c6-908a-20461858ee00\" (UID: \"7c575131-5aa9-41c6-908a-20461858ee00\") "
Aug 04 18:24:24.469842 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.469794    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cw6hg\" (UniqueName: \"kubernetes.io/projected/7c575131-5aa9-41c6-908a-20461858ee00-kube-api-access-cw6hg\") pod \"7c575131-5aa9-41c6-908a-20461858ee00\" (UID: \"7c575131-5aa9-41c6-908a-20461858ee00\") "
Aug 04 18:24:24.469842 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.469844    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ec7ced6-04a1-42fc-9a0d-c8fb42531079-plugins-config-volume\") pod \"4ec7ced6-04a1-42fc-9a0d-c8fb42531079\" (UID: \"4ec7ced6-04a1-42fc-9a0d-c8fb42531079\") "
Aug 04 18:24:24.470170 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.469880    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6vg4\" (UniqueName: \"kubernetes.io/projected/4ec7ced6-04a1-42fc-9a0d-c8fb42531079-kube-api-access-c6vg4\") pod \"4ec7ced6-04a1-42fc-9a0d-c8fb42531079\" (UID: \"4ec7ced6-04a1-42fc-9a0d-c8fb42531079\") "
Aug 04 18:24:24.470227 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.470183    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c575131-5aa9-41c6-908a-20461858ee00-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7c575131-5aa9-41c6-908a-20461858ee00" (UID: "7c575131-5aa9-41c6-908a-20461858ee00"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:24:24.470338 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.470255    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ec7ced6-04a1-42fc-9a0d-c8fb42531079-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4ec7ced6-04a1-42fc-9a0d-c8fb42531079" (UID: "4ec7ced6-04a1-42fc-9a0d-c8fb42531079"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:24:24.470338 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.470333    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ec7ced6-04a1-42fc-9a0d-c8fb42531079-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:24:24.470448 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.470346    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca391bc6-11eb-4a08-9930-1826935ee5e2-maas-api-tls\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:24:24.470448 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.470355    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gw4sl\" (UniqueName: \"kubernetes.io/projected/ca391bc6-11eb-4a08-9930-1826935ee5e2-kube-api-access-gw4sl\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:24:24.470448 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.470363    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca391bc6-11eb-4a08-9930-1826935ee5e2-openshift-service-ca\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:24:24.470448 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.470371    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c575131-5aa9-41c6-908a-20461858ee00-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:24:24.472332 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.472311    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ec7ced6-04a1-42fc-9a0d-c8fb42531079-kube-api-access-c6vg4" (OuterVolumeSpecName: "kube-api-access-c6vg4") pod "4ec7ced6-04a1-42fc-9a0d-c8fb42531079" (UID: "4ec7ced6-04a1-42fc-9a0d-c8fb42531079"). InnerVolumeSpecName "kube-api-access-c6vg4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:24:24.472402 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.472350    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c575131-5aa9-41c6-908a-20461858ee00-kube-api-access-cw6hg" (OuterVolumeSpecName: "kube-api-access-cw6hg") pod "7c575131-5aa9-41c6-908a-20461858ee00" (UID: "7c575131-5aa9-41c6-908a-20461858ee00"). InnerVolumeSpecName "kube-api-access-cw6hg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:24:24.489616 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.489587    2560 generic.go:356] "Generic (PLEG): container finished" podID="ca391bc6-11eb-4a08-9930-1826935ee5e2" containerID="4f40e067679e267984050b9829155c79bd0f51013598a046b535fd0c93aa26cb" exitCode=0
Aug 04 18:24:24.489745 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.489628    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-5768665896-7mcnm"
Aug 04 18:24:24.489745 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.489680    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-5768665896-7mcnm" event={"ID":"ca391bc6-11eb-4a08-9930-1826935ee5e2","Type":"ContainerDied","Data":"4f40e067679e267984050b9829155c79bd0f51013598a046b535fd0c93aa26cb"}
Aug 04 18:24:24.489745 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.489740    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-5768665896-7mcnm" event={"ID":"ca391bc6-11eb-4a08-9930-1826935ee5e2","Type":"ContainerDied","Data":"8f5ff140eea850ebaa7c2ecf56ce6f2d407ddb69a20140180f997db7ec061b48"}
Aug 04 18:24:24.489911 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.489772    2560 scope.go:117] "RemoveContainer" containerID="4f40e067679e267984050b9829155c79bd0f51013598a046b535fd0c93aa26cb"
Aug 04 18:24:24.491013 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.490952    2560 generic.go:356] "Generic (PLEG): container finished" podID="4ec7ced6-04a1-42fc-9a0d-c8fb42531079" containerID="7493e1e318028b858f9b07556ef057c49eafbbd640ff3244aa3b8bfbf81f5ee1" exitCode=0
Aug 04 18:24:24.491088 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.491012    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-07becd73-b4d888598-jz9br"
Aug 04 18:24:24.491163 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.491015    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-07becd73-b4d888598-jz9br" event={"ID":"4ec7ced6-04a1-42fc-9a0d-c8fb42531079","Type":"ContainerDied","Data":"7493e1e318028b858f9b07556ef057c49eafbbd640ff3244aa3b8bfbf81f5ee1"}
Aug 04 18:24:24.491221 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.491180    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-07becd73-b4d888598-jz9br" event={"ID":"4ec7ced6-04a1-42fc-9a0d-c8fb42531079","Type":"ContainerDied","Data":"8b1572bbbdc1e7a2244c37297d168247cd973f4c2978eece4e0fd9fd877579fb"}
Aug 04 18:24:24.492546 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.492518    2560 generic.go:356] "Generic (PLEG): container finished" podID="7c575131-5aa9-41c6-908a-20461858ee00" containerID="3412ad3c6aa8f67dee2f18574a65f3b44ae5339d662ba98f4724b868df357728" exitCode=0
Aug 04 18:24:24.492662 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.492550    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s" event={"ID":"7c575131-5aa9-41c6-908a-20461858ee00","Type":"ContainerDied","Data":"3412ad3c6aa8f67dee2f18574a65f3b44ae5339d662ba98f4724b868df357728"}
Aug 04 18:24:24.492662 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.492558    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s"
Aug 04 18:24:24.492662 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.492586    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s" event={"ID":"7c575131-5aa9-41c6-908a-20461858ee00","Type":"ContainerDied","Data":"feed9a1a1e023d30ea47949992174958a2676cfe3ae11a4b667f6a7deaa3b969"}
Aug 04 18:24:24.499273 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.499139    2560 scope.go:117] "RemoveContainer" containerID="4f40e067679e267984050b9829155c79bd0f51013598a046b535fd0c93aa26cb"
Aug 04 18:24:24.499457 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:24:24.499434    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f40e067679e267984050b9829155c79bd0f51013598a046b535fd0c93aa26cb\": container with ID starting with 4f40e067679e267984050b9829155c79bd0f51013598a046b535fd0c93aa26cb not found: ID does not exist" containerID="4f40e067679e267984050b9829155c79bd0f51013598a046b535fd0c93aa26cb"
Aug 04 18:24:24.499547 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.499469    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f40e067679e267984050b9829155c79bd0f51013598a046b535fd0c93aa26cb"} err="failed to get container status \"4f40e067679e267984050b9829155c79bd0f51013598a046b535fd0c93aa26cb\": rpc error: code = NotFound desc = could not find container \"4f40e067679e267984050b9829155c79bd0f51013598a046b535fd0c93aa26cb\": container with ID starting with 4f40e067679e267984050b9829155c79bd0f51013598a046b535fd0c93aa26cb not found: ID does not exist"
Aug 04 18:24:24.499547 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.499494    2560 scope.go:117] "RemoveContainer" containerID="7493e1e318028b858f9b07556ef057c49eafbbd640ff3244aa3b8bfbf81f5ee1"
Aug 04 18:24:24.508056 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.508039    2560 scope.go:117] "RemoveContainer" containerID="7493e1e318028b858f9b07556ef057c49eafbbd640ff3244aa3b8bfbf81f5ee1"
Aug 04 18:24:24.508295 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:24:24.508279    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7493e1e318028b858f9b07556ef057c49eafbbd640ff3244aa3b8bfbf81f5ee1\": container with ID starting with 7493e1e318028b858f9b07556ef057c49eafbbd640ff3244aa3b8bfbf81f5ee1 not found: ID does not exist" containerID="7493e1e318028b858f9b07556ef057c49eafbbd640ff3244aa3b8bfbf81f5ee1"
Aug 04 18:24:24.508339 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.508303    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7493e1e318028b858f9b07556ef057c49eafbbd640ff3244aa3b8bfbf81f5ee1"} err="failed to get container status \"7493e1e318028b858f9b07556ef057c49eafbbd640ff3244aa3b8bfbf81f5ee1\": rpc error: code = NotFound desc = could not find container \"7493e1e318028b858f9b07556ef057c49eafbbd640ff3244aa3b8bfbf81f5ee1\": container with ID starting with 7493e1e318028b858f9b07556ef057c49eafbbd640ff3244aa3b8bfbf81f5ee1 not found: ID does not exist"
Aug 04 18:24:24.508339 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.508320    2560 scope.go:117] "RemoveContainer" containerID="3412ad3c6aa8f67dee2f18574a65f3b44ae5339d662ba98f4724b868df357728"
Aug 04 18:24:24.515100 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.515077    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-5768665896-7mcnm"]
Aug 04 18:24:24.516284 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.516268    2560 scope.go:117] "RemoveContainer" containerID="3412ad3c6aa8f67dee2f18574a65f3b44ae5339d662ba98f4724b868df357728"
Aug 04 18:24:24.516539 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:24:24.516522    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3412ad3c6aa8f67dee2f18574a65f3b44ae5339d662ba98f4724b868df357728\": container with ID starting with 3412ad3c6aa8f67dee2f18574a65f3b44ae5339d662ba98f4724b868df357728 not found: ID does not exist" containerID="3412ad3c6aa8f67dee2f18574a65f3b44ae5339d662ba98f4724b868df357728"
Aug 04 18:24:24.516584 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.516545    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3412ad3c6aa8f67dee2f18574a65f3b44ae5339d662ba98f4724b868df357728"} err="failed to get container status \"3412ad3c6aa8f67dee2f18574a65f3b44ae5339d662ba98f4724b868df357728\": rpc error: code = NotFound desc = could not find container \"3412ad3c6aa8f67dee2f18574a65f3b44ae5339d662ba98f4724b868df357728\": container with ID starting with 3412ad3c6aa8f67dee2f18574a65f3b44ae5339d662ba98f4724b868df357728 not found: ID does not exist"
Aug 04 18:24:24.521920 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.521899    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-07becd73-5768665896-7mcnm"]
Aug 04 18:24:24.538841 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.538812    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-07becd73-b4d888598-jz9br"]
Aug 04 18:24:24.540013 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.539972    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-07becd73-b4d888598-jz9br"]
Aug 04 18:24:24.550501 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.550478    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s"]
Aug 04 18:24:24.554828 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.554804    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-07becd73-5f57476f68-g5q2s"]
Aug 04 18:24:24.571496 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.571473    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6vg4\" (UniqueName: \"kubernetes.io/projected/4ec7ced6-04a1-42fc-9a0d-c8fb42531079-kube-api-access-c6vg4\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:24:24.571496 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:24.571495    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cw6hg\" (UniqueName: \"kubernetes.io/projected/7c575131-5aa9-41c6-908a-20461858ee00-kube-api-access-cw6hg\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:24:26.354000 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:26.353952    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ec7ced6-04a1-42fc-9a0d-c8fb42531079" path="/var/lib/kubelet/pods/4ec7ced6-04a1-42fc-9a0d-c8fb42531079/volumes"
Aug 04 18:24:26.354384 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:26.354355    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c575131-5aa9-41c6-908a-20461858ee00" path="/var/lib/kubelet/pods/7c575131-5aa9-41c6-908a-20461858ee00/volumes"
Aug 04 18:24:26.354685 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:24:26.354672    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca391bc6-11eb-4a08-9930-1826935ee5e2" path="/var/lib/kubelet/pods/ca391bc6-11eb-4a08-9930-1826935ee5e2/volumes"
Aug 04 18:26:35.885609 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:35.885533    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b"]
Aug 04 18:26:35.886184 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:35.886079    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ec7ced6-04a1-42fc-9a0d-c8fb42531079" containerName="payload-processing"
Aug 04 18:26:35.886184 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:35.886098    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ec7ced6-04a1-42fc-9a0d-c8fb42531079" containerName="payload-processing"
Aug 04 18:26:35.886184 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:35.886114    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca391bc6-11eb-4a08-9930-1826935ee5e2" containerName="maas-api"
Aug 04 18:26:35.886184 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:35.886122    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca391bc6-11eb-4a08-9930-1826935ee5e2" containerName="maas-api"
Aug 04 18:26:35.886184 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:35.886145    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c575131-5aa9-41c6-908a-20461858ee00" containerName="payload-pre-processing"
Aug 04 18:26:35.886184 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:35.886155    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c575131-5aa9-41c6-908a-20461858ee00" containerName="payload-pre-processing"
Aug 04 18:26:35.886493 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:35.886252    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ec7ced6-04a1-42fc-9a0d-c8fb42531079" containerName="payload-processing"
Aug 04 18:26:35.886493 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:35.886268    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca391bc6-11eb-4a08-9930-1826935ee5e2" containerName="maas-api"
Aug 04 18:26:35.886493 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:35.886279    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c575131-5aa9-41c6-908a-20461858ee00" containerName="payload-pre-processing"
Aug 04 18:26:35.889769 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:35.889744    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b"
Aug 04 18:26:35.892656 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:35.892639    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-6abb689a-serving-cert\""
Aug 04 18:26:35.903125 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:35.903101    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b"]
Aug 04 18:26:35.955534 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:35.955509    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ts5l\" (UniqueName: \"kubernetes.io/projected/e674674e-c180-454d-a2d7-cce421047e28-kube-api-access-8ts5l\") pod \"maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b\" (UID: \"e674674e-c180-454d-a2d7-cce421047e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b"
Aug 04 18:26:35.955660 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:35.955561    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e674674e-c180-454d-a2d7-cce421047e28-openshift-service-ca\") pod \"maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b\" (UID: \"e674674e-c180-454d-a2d7-cce421047e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b"
Aug 04 18:26:35.955660 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:35.955627    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e674674e-c180-454d-a2d7-cce421047e28-maas-api-tls\") pod \"maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b\" (UID: \"e674674e-c180-454d-a2d7-cce421047e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b"
Aug 04 18:26:36.056597 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:36.056575    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ts5l\" (UniqueName: \"kubernetes.io/projected/e674674e-c180-454d-a2d7-cce421047e28-kube-api-access-8ts5l\") pod \"maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b\" (UID: \"e674674e-c180-454d-a2d7-cce421047e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b"
Aug 04 18:26:36.056716 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:36.056618    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e674674e-c180-454d-a2d7-cce421047e28-openshift-service-ca\") pod \"maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b\" (UID: \"e674674e-c180-454d-a2d7-cce421047e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b"
Aug 04 18:26:36.056716 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:36.056663    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e674674e-c180-454d-a2d7-cce421047e28-maas-api-tls\") pod \"maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b\" (UID: \"e674674e-c180-454d-a2d7-cce421047e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b"
Aug 04 18:26:36.057357 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:36.057332    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e674674e-c180-454d-a2d7-cce421047e28-openshift-service-ca\") pod \"maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b\" (UID: \"e674674e-c180-454d-a2d7-cce421047e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b"
Aug 04 18:26:36.059211 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:36.059190    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e674674e-c180-454d-a2d7-cce421047e28-maas-api-tls\") pod \"maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b\" (UID: \"e674674e-c180-454d-a2d7-cce421047e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b"
Aug 04 18:26:36.069374 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:36.069354    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ts5l\" (UniqueName: \"kubernetes.io/projected/e674674e-c180-454d-a2d7-cce421047e28-kube-api-access-8ts5l\") pod \"maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b\" (UID: \"e674674e-c180-454d-a2d7-cce421047e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b"
Aug 04 18:26:36.203352 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:36.203328    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b"
Aug 04 18:26:36.376052 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:36.376031    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b"]
Aug 04 18:26:36.378871 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:26:36.378841    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode674674e_c180_454d_a2d7_cce421047e28.slice/crio-938ceee8cd29acbd37b347aeca221549b02cfcb58ae994d1bd3a8ae709baae23 WatchSource:0}: Error finding container 938ceee8cd29acbd37b347aeca221549b02cfcb58ae994d1bd3a8ae709baae23: Status 404 returned error can't find the container with id 938ceee8cd29acbd37b347aeca221549b02cfcb58ae994d1bd3a8ae709baae23
Aug 04 18:26:36.963782 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:36.963750    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj"]
Aug 04 18:26:36.967201 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:36.967185    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj"
Aug 04 18:26:36.972853 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:36.972834    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-6abb689a-dockercfg-vn4ph\""
Aug 04 18:26:36.972964 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:36.972834    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-6abb689a\""
Aug 04 18:26:36.996547 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:36.996520    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj"]
Aug 04 18:26:37.018551 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:37.018524    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b"]
Aug 04 18:26:37.022375 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:37.022355    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b"
Aug 04 18:26:37.039194 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:37.039167    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b"]
Aug 04 18:26:37.065303 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:37.065278    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w229w\" (UniqueName: \"kubernetes.io/projected/49aa2b07-81a5-4b56-a5b7-12a56915b8a4-kube-api-access-w229w\") pod \"payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b\" (UID: \"49aa2b07-81a5-4b56-a5b7-12a56915b8a4\") " pod="openshift-ingress/payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b"
Aug 04 18:26:37.065424 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:37.065318    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49aa2b07-81a5-4b56-a5b7-12a56915b8a4-plugins-config-volume\") pod \"payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b\" (UID: \"49aa2b07-81a5-4b56-a5b7-12a56915b8a4\") " pod="openshift-ingress/payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b"
Aug 04 18:26:37.065424 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:37.065345    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwc85\" (UniqueName: \"kubernetes.io/projected/3cfbef58-150a-44c8-87c4-49ee31095e99-kube-api-access-zwc85\") pod \"payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj\" (UID: \"3cfbef58-150a-44c8-87c4-49ee31095e99\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj"
Aug 04 18:26:37.065495 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:37.065450    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cfbef58-150a-44c8-87c4-49ee31095e99-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj\" (UID: \"3cfbef58-150a-44c8-87c4-49ee31095e99\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj"
Aug 04 18:26:37.119476 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:37.119403    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b" event={"ID":"e674674e-c180-454d-a2d7-cce421047e28","Type":"ContainerStarted","Data":"4dec3003b371e099e59fd501497a7706ae17a9eeb4fe27f423cac8fd6fa0c764"}
Aug 04 18:26:37.119476 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:37.119452    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b" event={"ID":"e674674e-c180-454d-a2d7-cce421047e28","Type":"ContainerStarted","Data":"938ceee8cd29acbd37b347aeca221549b02cfcb58ae994d1bd3a8ae709baae23"}
Aug 04 18:26:37.119648 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:37.119510    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b"
Aug 04 18:26:37.156291 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:37.156001    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b" podStartSLOduration=1.694490692 podStartE2EDuration="2.155966682s" totalImagesPullingTime="461.47599ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:26:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:26:37.154924453 +0000 UTC m=+2079.338084630" watchObservedRunningTime="2026-08-04 18:26:37.155966682 +0000 UTC m=+2079.339126860"
Aug 04 18:26:37.166390 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:37.166362    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w229w\" (UniqueName: \"kubernetes.io/projected/49aa2b07-81a5-4b56-a5b7-12a56915b8a4-kube-api-access-w229w\") pod \"payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b\" (UID: \"49aa2b07-81a5-4b56-a5b7-12a56915b8a4\") " pod="openshift-ingress/payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b"
Aug 04 18:26:37.166534 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:37.166415    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49aa2b07-81a5-4b56-a5b7-12a56915b8a4-plugins-config-volume\") pod \"payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b\" (UID: \"49aa2b07-81a5-4b56-a5b7-12a56915b8a4\") " pod="openshift-ingress/payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b"
Aug 04 18:26:37.166534 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:37.166441    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwc85\" (UniqueName: \"kubernetes.io/projected/3cfbef58-150a-44c8-87c4-49ee31095e99-kube-api-access-zwc85\") pod \"payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj\" (UID: \"3cfbef58-150a-44c8-87c4-49ee31095e99\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj"
Aug 04 18:26:37.166738 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:37.166696    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cfbef58-150a-44c8-87c4-49ee31095e99-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj\" (UID: \"3cfbef58-150a-44c8-87c4-49ee31095e99\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj"
Aug 04 18:26:37.167284 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:37.167259    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49aa2b07-81a5-4b56-a5b7-12a56915b8a4-plugins-config-volume\") pod \"payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b\" (UID: \"49aa2b07-81a5-4b56-a5b7-12a56915b8a4\") " pod="openshift-ingress/payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b"
Aug 04 18:26:37.167398 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:37.167357    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cfbef58-150a-44c8-87c4-49ee31095e99-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj\" (UID: \"3cfbef58-150a-44c8-87c4-49ee31095e99\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj"
Aug 04 18:26:37.178632 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:37.178600    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwc85\" (UniqueName: \"kubernetes.io/projected/3cfbef58-150a-44c8-87c4-49ee31095e99-kube-api-access-zwc85\") pod \"payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj\" (UID: \"3cfbef58-150a-44c8-87c4-49ee31095e99\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj"
Aug 04 18:26:37.179040 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:37.179023    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w229w\" (UniqueName: \"kubernetes.io/projected/49aa2b07-81a5-4b56-a5b7-12a56915b8a4-kube-api-access-w229w\") pod \"payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b\" (UID: \"49aa2b07-81a5-4b56-a5b7-12a56915b8a4\") " pod="openshift-ingress/payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b"
Aug 04 18:26:37.276409 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:37.276384    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj"
Aug 04 18:26:37.333392 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:37.333362    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b"
Aug 04 18:26:37.423775 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:37.423700    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj"]
Aug 04 18:26:37.427265 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:26:37.427232    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cfbef58_150a_44c8_87c4_49ee31095e99.slice/crio-ff5a2f066cd2141dc4eabc25b8328c17d81f80b1ecc1d760fc538f4a61e06c05 WatchSource:0}: Error finding container ff5a2f066cd2141dc4eabc25b8328c17d81f80b1ecc1d760fc538f4a61e06c05: Status 404 returned error can't find the container with id ff5a2f066cd2141dc4eabc25b8328c17d81f80b1ecc1d760fc538f4a61e06c05
Aug 04 18:26:37.479896 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:37.479875    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b"]
Aug 04 18:26:37.481478 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:26:37.481457    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49aa2b07_81a5_4b56_a5b7_12a56915b8a4.slice/crio-13fe46e9dfc0287cb15e35fc1072e13f917d75f76d7c01a217bc677c9ede13f8 WatchSource:0}: Error finding container 13fe46e9dfc0287cb15e35fc1072e13f917d75f76d7c01a217bc677c9ede13f8: Status 404 returned error can't find the container with id 13fe46e9dfc0287cb15e35fc1072e13f917d75f76d7c01a217bc677c9ede13f8
Aug 04 18:26:38.136195 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:38.136157    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj" event={"ID":"3cfbef58-150a-44c8-87c4-49ee31095e99","Type":"ContainerStarted","Data":"816d1c42bf356f5f3c6700a0c8b1e67a1f521c9b3a13dc2a4dcf4fc6ff7e44d7"}
Aug 04 18:26:38.136754 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:38.136729    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj" event={"ID":"3cfbef58-150a-44c8-87c4-49ee31095e99","Type":"ContainerStarted","Data":"ff5a2f066cd2141dc4eabc25b8328c17d81f80b1ecc1d760fc538f4a61e06c05"}
Aug 04 18:26:38.136883 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:38.136815    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj"
Aug 04 18:26:38.138062 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:38.138035    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b" event={"ID":"49aa2b07-81a5-4b56-a5b7-12a56915b8a4","Type":"ContainerStarted","Data":"1877b17671e6bec9996fef98d4f8ac574765fa6a91708150099d8c2fd55d7074"}
Aug 04 18:26:38.138154 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:38.138083    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b" event={"ID":"49aa2b07-81a5-4b56-a5b7-12a56915b8a4","Type":"ContainerStarted","Data":"13fe46e9dfc0287cb15e35fc1072e13f917d75f76d7c01a217bc677c9ede13f8"}
Aug 04 18:26:38.138213 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:38.138145    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b"
Aug 04 18:26:38.161142 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:38.161105    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj" podStartSLOduration=2.161092612 podStartE2EDuration="2.161092612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:26:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:26:38.157767653 +0000 UTC m=+2080.340927829" watchObservedRunningTime="2026-08-04 18:26:38.161092612 +0000 UTC m=+2080.344252800"
Aug 04 18:26:38.179368 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:38.179334    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b" podStartSLOduration=2.179321345 podStartE2EDuration="2.179321345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:26:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:26:38.178029349 +0000 UTC m=+2080.361189523" watchObservedRunningTime="2026-08-04 18:26:38.179321345 +0000 UTC m=+2080.362481521"
Aug 04 18:26:43.144149 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:43.144121    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b"
Aug 04 18:26:49.144903 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:49.144875    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b"
Aug 04 18:26:50.149399 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:50.149370    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj"
Aug 04 18:26:58.448469 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:58.448377    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/2.log"
Aug 04 18:26:58.474089 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:58.454324    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 18:26:58.474089 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:58.457335    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/2.log"
Aug 04 18:26:58.474089 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:26:58.464813    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 18:27:18.276438 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.276402    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b"]
Aug 04 18:27:18.276962 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.276591    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b" podUID="e674674e-c180-454d-a2d7-cce421047e28" containerName="maas-api" containerID="cri-o://4dec3003b371e099e59fd501497a7706ae17a9eeb4fe27f423cac8fd6fa0c764" gracePeriod=30
Aug 04 18:27:18.369392 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.369349    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b"]
Aug 04 18:27:18.369607 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.369561    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b" podUID="49aa2b07-81a5-4b56-a5b7-12a56915b8a4" containerName="payload-processing" containerID="cri-o://1877b17671e6bec9996fef98d4f8ac574765fa6a91708150099d8c2fd55d7074" gracePeriod=30
Aug 04 18:27:18.390175 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.390146    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj"]
Aug 04 18:27:18.390432 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.390401    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj" podUID="3cfbef58-150a-44c8-87c4-49ee31095e99" containerName="payload-pre-processing" containerID="cri-o://816d1c42bf356f5f3c6700a0c8b1e67a1f521c9b3a13dc2a4dcf4fc6ff7e44d7" gracePeriod=30
Aug 04 18:27:18.628408 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.628380    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b"
Aug 04 18:27:18.675342 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.675320    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b"
Aug 04 18:27:18.678586 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.678568    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj"
Aug 04 18:27:18.722957 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.722931    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ts5l\" (UniqueName: \"kubernetes.io/projected/e674674e-c180-454d-a2d7-cce421047e28-kube-api-access-8ts5l\") pod \"e674674e-c180-454d-a2d7-cce421047e28\" (UID: \"e674674e-c180-454d-a2d7-cce421047e28\") "
Aug 04 18:27:18.723124 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.723031    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e674674e-c180-454d-a2d7-cce421047e28-openshift-service-ca\") pod \"e674674e-c180-454d-a2d7-cce421047e28\" (UID: \"e674674e-c180-454d-a2d7-cce421047e28\") "
Aug 04 18:27:18.723197 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.723141    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e674674e-c180-454d-a2d7-cce421047e28-maas-api-tls\") pod \"e674674e-c180-454d-a2d7-cce421047e28\" (UID: \"e674674e-c180-454d-a2d7-cce421047e28\") "
Aug 04 18:27:18.723557 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.723511    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e674674e-c180-454d-a2d7-cce421047e28-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e674674e-c180-454d-a2d7-cce421047e28" (UID: "e674674e-c180-454d-a2d7-cce421047e28"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:27:18.723790 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.723766    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e674674e-c180-454d-a2d7-cce421047e28-openshift-service-ca\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:27:18.725319 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.725289    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e674674e-c180-454d-a2d7-cce421047e28-kube-api-access-8ts5l" (OuterVolumeSpecName: "kube-api-access-8ts5l") pod "e674674e-c180-454d-a2d7-cce421047e28" (UID: "e674674e-c180-454d-a2d7-cce421047e28"). InnerVolumeSpecName "kube-api-access-8ts5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:27:18.725426 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.725414    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e674674e-c180-454d-a2d7-cce421047e28-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e674674e-c180-454d-a2d7-cce421047e28" (UID: "e674674e-c180-454d-a2d7-cce421047e28"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:27:18.824244 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.824169    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w229w\" (UniqueName: \"kubernetes.io/projected/49aa2b07-81a5-4b56-a5b7-12a56915b8a4-kube-api-access-w229w\") pod \"49aa2b07-81a5-4b56-a5b7-12a56915b8a4\" (UID: \"49aa2b07-81a5-4b56-a5b7-12a56915b8a4\") "
Aug 04 18:27:18.824383 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.824251    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49aa2b07-81a5-4b56-a5b7-12a56915b8a4-plugins-config-volume\") pod \"49aa2b07-81a5-4b56-a5b7-12a56915b8a4\" (UID: \"49aa2b07-81a5-4b56-a5b7-12a56915b8a4\") "
Aug 04 18:27:18.824383 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.824267    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwc85\" (UniqueName: \"kubernetes.io/projected/3cfbef58-150a-44c8-87c4-49ee31095e99-kube-api-access-zwc85\") pod \"3cfbef58-150a-44c8-87c4-49ee31095e99\" (UID: \"3cfbef58-150a-44c8-87c4-49ee31095e99\") "
Aug 04 18:27:18.824383 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.824302    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cfbef58-150a-44c8-87c4-49ee31095e99-plugins-config-volume\") pod \"3cfbef58-150a-44c8-87c4-49ee31095e99\" (UID: \"3cfbef58-150a-44c8-87c4-49ee31095e99\") "
Aug 04 18:27:18.824506 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.824434    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e674674e-c180-454d-a2d7-cce421047e28-maas-api-tls\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:27:18.824506 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.824445    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ts5l\" (UniqueName: \"kubernetes.io/projected/e674674e-c180-454d-a2d7-cce421047e28-kube-api-access-8ts5l\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:27:18.824657 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.824624    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49aa2b07-81a5-4b56-a5b7-12a56915b8a4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "49aa2b07-81a5-4b56-a5b7-12a56915b8a4" (UID: "49aa2b07-81a5-4b56-a5b7-12a56915b8a4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:27:18.824777 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.824715    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cfbef58-150a-44c8-87c4-49ee31095e99-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3cfbef58-150a-44c8-87c4-49ee31095e99" (UID: "3cfbef58-150a-44c8-87c4-49ee31095e99"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:27:18.826414 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.826392    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cfbef58-150a-44c8-87c4-49ee31095e99-kube-api-access-zwc85" (OuterVolumeSpecName: "kube-api-access-zwc85") pod "3cfbef58-150a-44c8-87c4-49ee31095e99" (UID: "3cfbef58-150a-44c8-87c4-49ee31095e99"). InnerVolumeSpecName "kube-api-access-zwc85". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:27:18.826512 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.826490    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49aa2b07-81a5-4b56-a5b7-12a56915b8a4-kube-api-access-w229w" (OuterVolumeSpecName: "kube-api-access-w229w") pod "49aa2b07-81a5-4b56-a5b7-12a56915b8a4" (UID: "49aa2b07-81a5-4b56-a5b7-12a56915b8a4"). InnerVolumeSpecName "kube-api-access-w229w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:27:18.924936 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.924902    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w229w\" (UniqueName: \"kubernetes.io/projected/49aa2b07-81a5-4b56-a5b7-12a56915b8a4-kube-api-access-w229w\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:27:18.924936 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.924933    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49aa2b07-81a5-4b56-a5b7-12a56915b8a4-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:27:18.925134 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.924948    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwc85\" (UniqueName: \"kubernetes.io/projected/3cfbef58-150a-44c8-87c4-49ee31095e99-kube-api-access-zwc85\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:27:18.925134 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:18.924957    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cfbef58-150a-44c8-87c4-49ee31095e99-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:27:19.339982 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:19.339949    2560 generic.go:356] "Generic (PLEG): container finished" podID="3cfbef58-150a-44c8-87c4-49ee31095e99" containerID="816d1c42bf356f5f3c6700a0c8b1e67a1f521c9b3a13dc2a4dcf4fc6ff7e44d7" exitCode=0
Aug 04 18:27:19.340427 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:19.340018    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj"
Aug 04 18:27:19.340427 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:19.340035    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj" event={"ID":"3cfbef58-150a-44c8-87c4-49ee31095e99","Type":"ContainerDied","Data":"816d1c42bf356f5f3c6700a0c8b1e67a1f521c9b3a13dc2a4dcf4fc6ff7e44d7"}
Aug 04 18:27:19.340427 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:19.340092    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj" event={"ID":"3cfbef58-150a-44c8-87c4-49ee31095e99","Type":"ContainerDied","Data":"ff5a2f066cd2141dc4eabc25b8328c17d81f80b1ecc1d760fc538f4a61e06c05"}
Aug 04 18:27:19.340427 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:19.340122    2560 scope.go:117] "RemoveContainer" containerID="816d1c42bf356f5f3c6700a0c8b1e67a1f521c9b3a13dc2a4dcf4fc6ff7e44d7"
Aug 04 18:27:19.341516 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:19.341498    2560 generic.go:356] "Generic (PLEG): container finished" podID="e674674e-c180-454d-a2d7-cce421047e28" containerID="4dec3003b371e099e59fd501497a7706ae17a9eeb4fe27f423cac8fd6fa0c764" exitCode=0
Aug 04 18:27:19.341772 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:19.341552    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b"
Aug 04 18:27:19.341772 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:19.341525    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b" event={"ID":"e674674e-c180-454d-a2d7-cce421047e28","Type":"ContainerDied","Data":"4dec3003b371e099e59fd501497a7706ae17a9eeb4fe27f423cac8fd6fa0c764"}
Aug 04 18:27:19.341772 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:19.341690    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b" event={"ID":"e674674e-c180-454d-a2d7-cce421047e28","Type":"ContainerDied","Data":"938ceee8cd29acbd37b347aeca221549b02cfcb58ae994d1bd3a8ae709baae23"}
Aug 04 18:27:19.343086 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:19.343062    2560 generic.go:356] "Generic (PLEG): container finished" podID="49aa2b07-81a5-4b56-a5b7-12a56915b8a4" containerID="1877b17671e6bec9996fef98d4f8ac574765fa6a91708150099d8c2fd55d7074" exitCode=0
Aug 04 18:27:19.343201 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:19.343102    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b" event={"ID":"49aa2b07-81a5-4b56-a5b7-12a56915b8a4","Type":"ContainerDied","Data":"1877b17671e6bec9996fef98d4f8ac574765fa6a91708150099d8c2fd55d7074"}
Aug 04 18:27:19.343201 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:19.343135    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b" event={"ID":"49aa2b07-81a5-4b56-a5b7-12a56915b8a4","Type":"ContainerDied","Data":"13fe46e9dfc0287cb15e35fc1072e13f917d75f76d7c01a217bc677c9ede13f8"}
Aug 04 18:27:19.343201 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:19.343101    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b"
Aug 04 18:27:19.349146 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:19.349127    2560 scope.go:117] "RemoveContainer" containerID="816d1c42bf356f5f3c6700a0c8b1e67a1f521c9b3a13dc2a4dcf4fc6ff7e44d7"
Aug 04 18:27:19.349486 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:27:19.349460    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"816d1c42bf356f5f3c6700a0c8b1e67a1f521c9b3a13dc2a4dcf4fc6ff7e44d7\": container with ID starting with 816d1c42bf356f5f3c6700a0c8b1e67a1f521c9b3a13dc2a4dcf4fc6ff7e44d7 not found: ID does not exist" containerID="816d1c42bf356f5f3c6700a0c8b1e67a1f521c9b3a13dc2a4dcf4fc6ff7e44d7"
Aug 04 18:27:19.349559 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:19.349497    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"816d1c42bf356f5f3c6700a0c8b1e67a1f521c9b3a13dc2a4dcf4fc6ff7e44d7"} err="failed to get container status \"816d1c42bf356f5f3c6700a0c8b1e67a1f521c9b3a13dc2a4dcf4fc6ff7e44d7\": rpc error: code = NotFound desc = could not find container \"816d1c42bf356f5f3c6700a0c8b1e67a1f521c9b3a13dc2a4dcf4fc6ff7e44d7\": container with ID starting with 816d1c42bf356f5f3c6700a0c8b1e67a1f521c9b3a13dc2a4dcf4fc6ff7e44d7 not found: ID does not exist"
Aug 04 18:27:19.349559 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:19.349520    2560 scope.go:117] "RemoveContainer" containerID="4dec3003b371e099e59fd501497a7706ae17a9eeb4fe27f423cac8fd6fa0c764"
Aug 04 18:27:19.358508 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:19.358496    2560 scope.go:117] "RemoveContainer" containerID="4dec3003b371e099e59fd501497a7706ae17a9eeb4fe27f423cac8fd6fa0c764"
Aug 04 18:27:19.358735 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:27:19.358717    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dec3003b371e099e59fd501497a7706ae17a9eeb4fe27f423cac8fd6fa0c764\": container with ID starting with 4dec3003b371e099e59fd501497a7706ae17a9eeb4fe27f423cac8fd6fa0c764 not found: ID does not exist" containerID="4dec3003b371e099e59fd501497a7706ae17a9eeb4fe27f423cac8fd6fa0c764"
Aug 04 18:27:19.358775 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:19.358742    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dec3003b371e099e59fd501497a7706ae17a9eeb4fe27f423cac8fd6fa0c764"} err="failed to get container status \"4dec3003b371e099e59fd501497a7706ae17a9eeb4fe27f423cac8fd6fa0c764\": rpc error: code = NotFound desc = could not find container \"4dec3003b371e099e59fd501497a7706ae17a9eeb4fe27f423cac8fd6fa0c764\": container with ID starting with 4dec3003b371e099e59fd501497a7706ae17a9eeb4fe27f423cac8fd6fa0c764 not found: ID does not exist"
Aug 04 18:27:19.358775 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:19.358757    2560 scope.go:117] "RemoveContainer" containerID="1877b17671e6bec9996fef98d4f8ac574765fa6a91708150099d8c2fd55d7074"
Aug 04 18:27:19.366148 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:19.366134    2560 scope.go:117] "RemoveContainer" containerID="1877b17671e6bec9996fef98d4f8ac574765fa6a91708150099d8c2fd55d7074"
Aug 04 18:27:19.366381 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:27:19.366364    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1877b17671e6bec9996fef98d4f8ac574765fa6a91708150099d8c2fd55d7074\": container with ID starting with 1877b17671e6bec9996fef98d4f8ac574765fa6a91708150099d8c2fd55d7074 not found: ID does not exist" containerID="1877b17671e6bec9996fef98d4f8ac574765fa6a91708150099d8c2fd55d7074"
Aug 04 18:27:19.366425 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:19.366388    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1877b17671e6bec9996fef98d4f8ac574765fa6a91708150099d8c2fd55d7074"} err="failed to get container status \"1877b17671e6bec9996fef98d4f8ac574765fa6a91708150099d8c2fd55d7074\": rpc error: code = NotFound desc = could not find container \"1877b17671e6bec9996fef98d4f8ac574765fa6a91708150099d8c2fd55d7074\": container with ID starting with 1877b17671e6bec9996fef98d4f8ac574765fa6a91708150099d8c2fd55d7074 not found: ID does not exist"
Aug 04 18:27:19.370758 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:19.370738    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj"]
Aug 04 18:27:19.375720 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:19.375699    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6abb689a-697598b88d-v7dsj"]
Aug 04 18:27:19.387321 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:19.387304    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b"]
Aug 04 18:27:19.391846 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:19.391827    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6abb689a-55649d9498-7kh5b"]
Aug 04 18:27:19.404868 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:19.404847    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b"]
Aug 04 18:27:19.410760 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:19.410739    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6abb689a-5b8bf74566-rr55b"]
Aug 04 18:27:20.352588 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:20.352557    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cfbef58-150a-44c8-87c4-49ee31095e99" path="/var/lib/kubelet/pods/3cfbef58-150a-44c8-87c4-49ee31095e99/volumes"
Aug 04 18:27:20.352934 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:20.352911    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49aa2b07-81a5-4b56-a5b7-12a56915b8a4" path="/var/lib/kubelet/pods/49aa2b07-81a5-4b56-a5b7-12a56915b8a4/volumes"
Aug 04 18:27:20.353268 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:27:20.353255    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e674674e-c180-454d-a2d7-cce421047e28" path="/var/lib/kubelet/pods/e674674e-c180-454d-a2d7-cce421047e28/volumes"
Aug 04 18:29:46.945697 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:46.945657    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d"]
Aug 04 18:29:46.946257 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:46.946239    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49aa2b07-81a5-4b56-a5b7-12a56915b8a4" containerName="payload-processing"
Aug 04 18:29:46.946322 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:46.946259    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="49aa2b07-81a5-4b56-a5b7-12a56915b8a4" containerName="payload-processing"
Aug 04 18:29:46.946322 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:46.946287    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cfbef58-150a-44c8-87c4-49ee31095e99" containerName="payload-pre-processing"
Aug 04 18:29:46.946322 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:46.946296    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cfbef58-150a-44c8-87c4-49ee31095e99" containerName="payload-pre-processing"
Aug 04 18:29:46.946322 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:46.946311    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e674674e-c180-454d-a2d7-cce421047e28" containerName="maas-api"
Aug 04 18:29:46.946322 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:46.946319    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e674674e-c180-454d-a2d7-cce421047e28" containerName="maas-api"
Aug 04 18:29:46.946577 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:46.946408    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="49aa2b07-81a5-4b56-a5b7-12a56915b8a4" containerName="payload-processing"
Aug 04 18:29:46.946577 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:46.946419    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cfbef58-150a-44c8-87c4-49ee31095e99" containerName="payload-pre-processing"
Aug 04 18:29:46.946577 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:46.946429    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e674674e-c180-454d-a2d7-cce421047e28" containerName="maas-api"
Aug 04 18:29:46.949964 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:46.949940    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d"
Aug 04 18:29:46.952255 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:46.952231    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-ab5c72f0-serving-cert\""
Aug 04 18:29:46.960975 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:46.960944    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d"]
Aug 04 18:29:47.021209 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:47.021167    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08c6f92a-31c7-4a59-bfd5-c9258f0be05c-openshift-service-ca\") pod \"maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d\" (UID: \"08c6f92a-31c7-4a59-bfd5-c9258f0be05c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d"
Aug 04 18:29:47.021418 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:47.021307    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08c6f92a-31c7-4a59-bfd5-c9258f0be05c-maas-api-tls\") pod \"maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d\" (UID: \"08c6f92a-31c7-4a59-bfd5-c9258f0be05c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d"
Aug 04 18:29:47.021418 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:47.021339    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-569qp\" (UniqueName: \"kubernetes.io/projected/08c6f92a-31c7-4a59-bfd5-c9258f0be05c-kube-api-access-569qp\") pod \"maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d\" (UID: \"08c6f92a-31c7-4a59-bfd5-c9258f0be05c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d"
Aug 04 18:29:47.122802 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:47.122758    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08c6f92a-31c7-4a59-bfd5-c9258f0be05c-maas-api-tls\") pod \"maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d\" (UID: \"08c6f92a-31c7-4a59-bfd5-c9258f0be05c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d"
Aug 04 18:29:47.122802 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:47.122798    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-569qp\" (UniqueName: \"kubernetes.io/projected/08c6f92a-31c7-4a59-bfd5-c9258f0be05c-kube-api-access-569qp\") pod \"maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d\" (UID: \"08c6f92a-31c7-4a59-bfd5-c9258f0be05c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d"
Aug 04 18:29:47.123123 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:47.122838    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08c6f92a-31c7-4a59-bfd5-c9258f0be05c-openshift-service-ca\") pod \"maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d\" (UID: \"08c6f92a-31c7-4a59-bfd5-c9258f0be05c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d"
Aug 04 18:29:47.123123 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:29:47.122916    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-serving-cert: secret "maas-api-e2e-mt-ab5c72f0-serving-cert" not found
Aug 04 18:29:47.123123 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:29:47.123039    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c6f92a-31c7-4a59-bfd5-c9258f0be05c-maas-api-tls podName:08c6f92a-31c7-4a59-bfd5-c9258f0be05c nodeName:}" failed. No retries permitted until 2026-08-04 18:29:47.623020021 +0000 UTC m=+2269.806180190 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/08c6f92a-31c7-4a59-bfd5-c9258f0be05c-maas-api-tls") pod "maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d" (UID: "08c6f92a-31c7-4a59-bfd5-c9258f0be05c") : secret "maas-api-e2e-mt-ab5c72f0-serving-cert" not found
Aug 04 18:29:47.123538 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:47.123519    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08c6f92a-31c7-4a59-bfd5-c9258f0be05c-openshift-service-ca\") pod \"maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d\" (UID: \"08c6f92a-31c7-4a59-bfd5-c9258f0be05c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d"
Aug 04 18:29:47.142827 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:47.142788    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-569qp\" (UniqueName: \"kubernetes.io/projected/08c6f92a-31c7-4a59-bfd5-c9258f0be05c-kube-api-access-569qp\") pod \"maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d\" (UID: \"08c6f92a-31c7-4a59-bfd5-c9258f0be05c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d"
Aug 04 18:29:47.627141 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:47.627102    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08c6f92a-31c7-4a59-bfd5-c9258f0be05c-maas-api-tls\") pod \"maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d\" (UID: \"08c6f92a-31c7-4a59-bfd5-c9258f0be05c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d"
Aug 04 18:29:47.629794 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:47.629765    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08c6f92a-31c7-4a59-bfd5-c9258f0be05c-maas-api-tls\") pod \"maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d\" (UID: \"08c6f92a-31c7-4a59-bfd5-c9258f0be05c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d"
Aug 04 18:29:47.862385 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:47.862349    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d"
Aug 04 18:29:48.024654 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:48.024621    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d"]
Aug 04 18:29:48.026923 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:29:48.026891    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08c6f92a_31c7_4a59_bfd5_c9258f0be05c.slice/crio-ab44c53407326de958967f72703297d77a19eb0a53a30f1ef1e5ccb0e2329d12 WatchSource:0}: Error finding container ab44c53407326de958967f72703297d77a19eb0a53a30f1ef1e5ccb0e2329d12: Status 404 returned error can't find the container with id ab44c53407326de958967f72703297d77a19eb0a53a30f1ef1e5ccb0e2329d12
Aug 04 18:29:48.028433 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:48.028413    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 18:29:48.047833 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:48.047799    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d" event={"ID":"08c6f92a-31c7-4a59-bfd5-c9258f0be05c","Type":"ContainerStarted","Data":"ab44c53407326de958967f72703297d77a19eb0a53a30f1ef1e5ccb0e2329d12"}
Aug 04 18:29:48.163181 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:48.163147    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z"]
Aug 04 18:29:48.167610 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:48.167558    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z"
Aug 04 18:29:48.170158 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:48.170130    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-ab5c72f0\""
Aug 04 18:29:48.170278 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:48.170142    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-ab5c72f0-dockercfg-d7hkl\""
Aug 04 18:29:48.178289 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:48.178256    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z"]
Aug 04 18:29:48.231918 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:48.231887    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c63d3101-884b-4229-98a3-a4bcb4a9ec2f-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z\" (UID: \"c63d3101-884b-4229-98a3-a4bcb4a9ec2f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z"
Aug 04 18:29:48.231918 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:48.231920    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6x7bv\" (UniqueName: \"kubernetes.io/projected/c63d3101-884b-4229-98a3-a4bcb4a9ec2f-kube-api-access-6x7bv\") pod \"payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z\" (UID: \"c63d3101-884b-4229-98a3-a4bcb4a9ec2f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z"
Aug 04 18:29:48.282673 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:48.282635    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp"]
Aug 04 18:29:48.286277 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:48.286251    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp"
Aug 04 18:29:48.299176 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:48.299148    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp"]
Aug 04 18:29:48.332595 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:48.332549    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c63d3101-884b-4229-98a3-a4bcb4a9ec2f-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z\" (UID: \"c63d3101-884b-4229-98a3-a4bcb4a9ec2f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z"
Aug 04 18:29:48.332799 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:48.332600    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6x7bv\" (UniqueName: \"kubernetes.io/projected/c63d3101-884b-4229-98a3-a4bcb4a9ec2f-kube-api-access-6x7bv\") pod \"payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z\" (UID: \"c63d3101-884b-4229-98a3-a4bcb4a9ec2f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z"
Aug 04 18:29:48.333378 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:48.333351    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c63d3101-884b-4229-98a3-a4bcb4a9ec2f-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z\" (UID: \"c63d3101-884b-4229-98a3-a4bcb4a9ec2f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z"
Aug 04 18:29:48.345793 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:48.345765    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6x7bv\" (UniqueName: \"kubernetes.io/projected/c63d3101-884b-4229-98a3-a4bcb4a9ec2f-kube-api-access-6x7bv\") pod \"payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z\" (UID: \"c63d3101-884b-4229-98a3-a4bcb4a9ec2f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z"
Aug 04 18:29:48.433508 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:48.433407    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmqtt\" (UniqueName: \"kubernetes.io/projected/287f64cf-64ac-49fd-9643-7566074259ba-kube-api-access-kmqtt\") pod \"payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp\" (UID: \"287f64cf-64ac-49fd-9643-7566074259ba\") " pod="openshift-ingress/payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp"
Aug 04 18:29:48.433695 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:48.433564    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/287f64cf-64ac-49fd-9643-7566074259ba-plugins-config-volume\") pod \"payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp\" (UID: \"287f64cf-64ac-49fd-9643-7566074259ba\") " pod="openshift-ingress/payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp"
Aug 04 18:29:48.480246 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:48.480208    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z"
Aug 04 18:29:48.534388 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:48.534330    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmqtt\" (UniqueName: \"kubernetes.io/projected/287f64cf-64ac-49fd-9643-7566074259ba-kube-api-access-kmqtt\") pod \"payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp\" (UID: \"287f64cf-64ac-49fd-9643-7566074259ba\") " pod="openshift-ingress/payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp"
Aug 04 18:29:48.534802 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:48.534482    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/287f64cf-64ac-49fd-9643-7566074259ba-plugins-config-volume\") pod \"payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp\" (UID: \"287f64cf-64ac-49fd-9643-7566074259ba\") " pod="openshift-ingress/payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp"
Aug 04 18:29:48.535919 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:48.535890    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/287f64cf-64ac-49fd-9643-7566074259ba-plugins-config-volume\") pod \"payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp\" (UID: \"287f64cf-64ac-49fd-9643-7566074259ba\") " pod="openshift-ingress/payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp"
Aug 04 18:29:48.547486 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:48.547425    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmqtt\" (UniqueName: \"kubernetes.io/projected/287f64cf-64ac-49fd-9643-7566074259ba-kube-api-access-kmqtt\") pod \"payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp\" (UID: \"287f64cf-64ac-49fd-9643-7566074259ba\") " pod="openshift-ingress/payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp"
Aug 04 18:29:48.600593 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:48.600552    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp"
Aug 04 18:29:48.632965 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:48.632937    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z"]
Aug 04 18:29:48.635250 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:29:48.635215    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc63d3101_884b_4229_98a3_a4bcb4a9ec2f.slice/crio-d3d3c0225fa1db48add81f0ff7e6c137fa37f98efc421a38f45353e28dc7f583 WatchSource:0}: Error finding container d3d3c0225fa1db48add81f0ff7e6c137fa37f98efc421a38f45353e28dc7f583: Status 404 returned error can't find the container with id d3d3c0225fa1db48add81f0ff7e6c137fa37f98efc421a38f45353e28dc7f583
Aug 04 18:29:48.754362 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:48.754340    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp"]
Aug 04 18:29:48.756149 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:29:48.756122    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod287f64cf_64ac_49fd_9643_7566074259ba.slice/crio-3a32ba7dfd0aac698dfbde4a70ecbb00d6b7f2fd8ea967902490820255094705 WatchSource:0}: Error finding container 3a32ba7dfd0aac698dfbde4a70ecbb00d6b7f2fd8ea967902490820255094705: Status 404 returned error can't find the container with id 3a32ba7dfd0aac698dfbde4a70ecbb00d6b7f2fd8ea967902490820255094705
Aug 04 18:29:49.055614 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:49.055514    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d" event={"ID":"08c6f92a-31c7-4a59-bfd5-c9258f0be05c","Type":"ContainerStarted","Data":"e9e9719fb7f532d0cf810bf2bfa6080fdbf9ca640b9750ed254cac76018a4845"}
Aug 04 18:29:49.056074 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:49.055681    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d"
Aug 04 18:29:49.057152 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:49.057128    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z" event={"ID":"c63d3101-884b-4229-98a3-a4bcb4a9ec2f","Type":"ContainerStarted","Data":"925df239f9e1598a31006554cc7e12b8e60bc9f8afeff4e76c44837e2cf91aad"}
Aug 04 18:29:49.057274 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:49.057167    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z" event={"ID":"c63d3101-884b-4229-98a3-a4bcb4a9ec2f","Type":"ContainerStarted","Data":"d3d3c0225fa1db48add81f0ff7e6c137fa37f98efc421a38f45353e28dc7f583"}
Aug 04 18:29:49.057274 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:49.057224    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z"
Aug 04 18:29:49.058759 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:49.058740    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp" event={"ID":"287f64cf-64ac-49fd-9643-7566074259ba","Type":"ContainerStarted","Data":"6bfc3e226a18457295f5cc72e0a79e9f76fb59e371101e8c35eaf56785c074bb"}
Aug 04 18:29:49.058885 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:49.058781    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp" event={"ID":"287f64cf-64ac-49fd-9643-7566074259ba","Type":"ContainerStarted","Data":"3a32ba7dfd0aac698dfbde4a70ecbb00d6b7f2fd8ea967902490820255094705"}
Aug 04 18:29:49.058885 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:49.058809    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp"
Aug 04 18:29:49.073607 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:49.073570    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d" podStartSLOduration=2.593740613 podStartE2EDuration="3.07355652s" totalImagesPullingTime="479.815907ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:29:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:29:49.072240781 +0000 UTC m=+2271.255400996" watchObservedRunningTime="2026-08-04 18:29:49.07355652 +0000 UTC m=+2271.256716696"
Aug 04 18:29:49.088385 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:49.088331    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z" podStartSLOduration=1.088314671 podStartE2EDuration="1.088314671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:29:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:29:49.087808071 +0000 UTC m=+2271.270968250" watchObservedRunningTime="2026-08-04 18:29:49.088314671 +0000 UTC m=+2271.271474847"
Aug 04 18:29:49.107524 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:49.107480    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp" podStartSLOduration=1.107468598 podStartE2EDuration="1.107468598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:29:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:29:49.105707658 +0000 UTC m=+2271.288867865" watchObservedRunningTime="2026-08-04 18:29:49.107468598 +0000 UTC m=+2271.290628773"
Aug 04 18:29:55.070128 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:29:55.070091    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d"
Aug 04 18:30:00.065545 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:30:00.065515    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z"
Aug 04 18:30:00.065947 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:30:00.065621    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp"
Aug 04 18:31:16.261350 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.261321    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d"]
Aug 04 18:31:16.261908 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.261544    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d" podUID="08c6f92a-31c7-4a59-bfd5-c9258f0be05c" containerName="maas-api" containerID="cri-o://e9e9719fb7f532d0cf810bf2bfa6080fdbf9ca640b9750ed254cac76018a4845" gracePeriod=30
Aug 04 18:31:16.338598 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.338560    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp"]
Aug 04 18:31:16.339727 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.339675    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp" podUID="287f64cf-64ac-49fd-9643-7566074259ba" containerName="payload-processing" containerID="cri-o://6bfc3e226a18457295f5cc72e0a79e9f76fb59e371101e8c35eaf56785c074bb" gracePeriod=30
Aug 04 18:31:16.342261 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.342234    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z"]
Aug 04 18:31:16.342483 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.342442    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z" podUID="c63d3101-884b-4229-98a3-a4bcb4a9ec2f" containerName="payload-pre-processing" containerID="cri-o://925df239f9e1598a31006554cc7e12b8e60bc9f8afeff4e76c44837e2cf91aad" gracePeriod=30
Aug 04 18:31:16.491965 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.491925    2560 generic.go:356] "Generic (PLEG): container finished" podID="08c6f92a-31c7-4a59-bfd5-c9258f0be05c" containerID="e9e9719fb7f532d0cf810bf2bfa6080fdbf9ca640b9750ed254cac76018a4845" exitCode=0
Aug 04 18:31:16.492179 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.492026    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d" event={"ID":"08c6f92a-31c7-4a59-bfd5-c9258f0be05c","Type":"ContainerDied","Data":"e9e9719fb7f532d0cf810bf2bfa6080fdbf9ca640b9750ed254cac76018a4845"}
Aug 04 18:31:16.493859 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.493834    2560 generic.go:356] "Generic (PLEG): container finished" podID="c63d3101-884b-4229-98a3-a4bcb4a9ec2f" containerID="925df239f9e1598a31006554cc7e12b8e60bc9f8afeff4e76c44837e2cf91aad" exitCode=0
Aug 04 18:31:16.493968 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.493929    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z" event={"ID":"c63d3101-884b-4229-98a3-a4bcb4a9ec2f","Type":"ContainerDied","Data":"925df239f9e1598a31006554cc7e12b8e60bc9f8afeff4e76c44837e2cf91aad"}
Aug 04 18:31:16.495769 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.495745    2560 generic.go:356] "Generic (PLEG): container finished" podID="287f64cf-64ac-49fd-9643-7566074259ba" containerID="6bfc3e226a18457295f5cc72e0a79e9f76fb59e371101e8c35eaf56785c074bb" exitCode=0
Aug 04 18:31:16.496101 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.495828    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp" event={"ID":"287f64cf-64ac-49fd-9643-7566074259ba","Type":"ContainerDied","Data":"6bfc3e226a18457295f5cc72e0a79e9f76fb59e371101e8c35eaf56785c074bb"}
Aug 04 18:31:16.629502 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.629479    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp"
Aug 04 18:31:16.635350 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.635316    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d"
Aug 04 18:31:16.638266 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.638248    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z"
Aug 04 18:31:16.686437 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.686411    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-569qp\" (UniqueName: \"kubernetes.io/projected/08c6f92a-31c7-4a59-bfd5-c9258f0be05c-kube-api-access-569qp\") pod \"08c6f92a-31c7-4a59-bfd5-c9258f0be05c\" (UID: \"08c6f92a-31c7-4a59-bfd5-c9258f0be05c\") "
Aug 04 18:31:16.686621 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.686453    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/287f64cf-64ac-49fd-9643-7566074259ba-plugins-config-volume\") pod \"287f64cf-64ac-49fd-9643-7566074259ba\" (UID: \"287f64cf-64ac-49fd-9643-7566074259ba\") "
Aug 04 18:31:16.686621 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.686500    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08c6f92a-31c7-4a59-bfd5-c9258f0be05c-maas-api-tls\") pod \"08c6f92a-31c7-4a59-bfd5-c9258f0be05c\" (UID: \"08c6f92a-31c7-4a59-bfd5-c9258f0be05c\") "
Aug 04 18:31:16.686621 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.686603    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08c6f92a-31c7-4a59-bfd5-c9258f0be05c-openshift-service-ca\") pod \"08c6f92a-31c7-4a59-bfd5-c9258f0be05c\" (UID: \"08c6f92a-31c7-4a59-bfd5-c9258f0be05c\") "
Aug 04 18:31:16.686763 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.686649    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6x7bv\" (UniqueName: \"kubernetes.io/projected/c63d3101-884b-4229-98a3-a4bcb4a9ec2f-kube-api-access-6x7bv\") pod \"c63d3101-884b-4229-98a3-a4bcb4a9ec2f\" (UID: \"c63d3101-884b-4229-98a3-a4bcb4a9ec2f\") "
Aug 04 18:31:16.686763 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.686698    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c63d3101-884b-4229-98a3-a4bcb4a9ec2f-plugins-config-volume\") pod \"c63d3101-884b-4229-98a3-a4bcb4a9ec2f\" (UID: \"c63d3101-884b-4229-98a3-a4bcb4a9ec2f\") "
Aug 04 18:31:16.686763 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.686731    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmqtt\" (UniqueName: \"kubernetes.io/projected/287f64cf-64ac-49fd-9643-7566074259ba-kube-api-access-kmqtt\") pod \"287f64cf-64ac-49fd-9643-7566074259ba\" (UID: \"287f64cf-64ac-49fd-9643-7566074259ba\") "
Aug 04 18:31:16.686916 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.686864    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/287f64cf-64ac-49fd-9643-7566074259ba-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "287f64cf-64ac-49fd-9643-7566074259ba" (UID: "287f64cf-64ac-49fd-9643-7566074259ba"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:31:16.687459 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.687432    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/287f64cf-64ac-49fd-9643-7566074259ba-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:31:16.687571 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.687537    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08c6f92a-31c7-4a59-bfd5-c9258f0be05c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "08c6f92a-31c7-4a59-bfd5-c9258f0be05c" (UID: "08c6f92a-31c7-4a59-bfd5-c9258f0be05c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:31:16.687571 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.687540    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c63d3101-884b-4229-98a3-a4bcb4a9ec2f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c63d3101-884b-4229-98a3-a4bcb4a9ec2f" (UID: "c63d3101-884b-4229-98a3-a4bcb4a9ec2f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:31:16.690688 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.690435    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08c6f92a-31c7-4a59-bfd5-c9258f0be05c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "08c6f92a-31c7-4a59-bfd5-c9258f0be05c" (UID: "08c6f92a-31c7-4a59-bfd5-c9258f0be05c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:31:16.690688 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.690495    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08c6f92a-31c7-4a59-bfd5-c9258f0be05c-kube-api-access-569qp" (OuterVolumeSpecName: "kube-api-access-569qp") pod "08c6f92a-31c7-4a59-bfd5-c9258f0be05c" (UID: "08c6f92a-31c7-4a59-bfd5-c9258f0be05c"). InnerVolumeSpecName "kube-api-access-569qp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:31:16.690688 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.690510    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c63d3101-884b-4229-98a3-a4bcb4a9ec2f-kube-api-access-6x7bv" (OuterVolumeSpecName: "kube-api-access-6x7bv") pod "c63d3101-884b-4229-98a3-a4bcb4a9ec2f" (UID: "c63d3101-884b-4229-98a3-a4bcb4a9ec2f"). InnerVolumeSpecName "kube-api-access-6x7bv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:31:16.690688 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.690644    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/287f64cf-64ac-49fd-9643-7566074259ba-kube-api-access-kmqtt" (OuterVolumeSpecName: "kube-api-access-kmqtt") pod "287f64cf-64ac-49fd-9643-7566074259ba" (UID: "287f64cf-64ac-49fd-9643-7566074259ba"). InnerVolumeSpecName "kube-api-access-kmqtt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:31:16.788308 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.788233    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08c6f92a-31c7-4a59-bfd5-c9258f0be05c-maas-api-tls\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:31:16.788308 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.788262    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08c6f92a-31c7-4a59-bfd5-c9258f0be05c-openshift-service-ca\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:31:16.788308 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.788272    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6x7bv\" (UniqueName: \"kubernetes.io/projected/c63d3101-884b-4229-98a3-a4bcb4a9ec2f-kube-api-access-6x7bv\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:31:16.788308 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.788281    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c63d3101-884b-4229-98a3-a4bcb4a9ec2f-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:31:16.788308 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.788291    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmqtt\" (UniqueName: \"kubernetes.io/projected/287f64cf-64ac-49fd-9643-7566074259ba-kube-api-access-kmqtt\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:31:16.788308 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:16.788305    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-569qp\" (UniqueName: \"kubernetes.io/projected/08c6f92a-31c7-4a59-bfd5-c9258f0be05c-kube-api-access-569qp\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:31:17.501379 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:17.501346    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d"
Aug 04 18:31:17.501887 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:17.501383    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d" event={"ID":"08c6f92a-31c7-4a59-bfd5-c9258f0be05c","Type":"ContainerDied","Data":"ab44c53407326de958967f72703297d77a19eb0a53a30f1ef1e5ccb0e2329d12"}
Aug 04 18:31:17.501887 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:17.501454    2560 scope.go:117] "RemoveContainer" containerID="e9e9719fb7f532d0cf810bf2bfa6080fdbf9ca640b9750ed254cac76018a4845"
Aug 04 18:31:17.502813 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:17.502793    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z"
Aug 04 18:31:17.502888 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:17.502811    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z" event={"ID":"c63d3101-884b-4229-98a3-a4bcb4a9ec2f","Type":"ContainerDied","Data":"d3d3c0225fa1db48add81f0ff7e6c137fa37f98efc421a38f45353e28dc7f583"}
Aug 04 18:31:17.504316 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:17.504290    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp" event={"ID":"287f64cf-64ac-49fd-9643-7566074259ba","Type":"ContainerDied","Data":"3a32ba7dfd0aac698dfbde4a70ecbb00d6b7f2fd8ea967902490820255094705"}
Aug 04 18:31:17.504433 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:17.504323    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp"
Aug 04 18:31:17.514683 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:17.514663    2560 scope.go:117] "RemoveContainer" containerID="925df239f9e1598a31006554cc7e12b8e60bc9f8afeff4e76c44837e2cf91aad"
Aug 04 18:31:17.523366 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:17.523349    2560 scope.go:117] "RemoveContainer" containerID="6bfc3e226a18457295f5cc72e0a79e9f76fb59e371101e8c35eaf56785c074bb"
Aug 04 18:31:17.540563 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:17.540532    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d"]
Aug 04 18:31:17.544001 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:17.543967    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ab5c72f0-77594c9c5f-5v62d"]
Aug 04 18:31:17.555647 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:17.555627    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp"]
Aug 04 18:31:17.561114 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:17.561089    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ab5c72f0-85d7bdbc8b-dlzwp"]
Aug 04 18:31:17.576968 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:17.576945    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z"]
Aug 04 18:31:17.581801 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:17.581782    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ab5c72f0-869d6bd66d-fvq8z"]
Aug 04 18:31:18.354804 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:18.354771    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08c6f92a-31c7-4a59-bfd5-c9258f0be05c" path="/var/lib/kubelet/pods/08c6f92a-31c7-4a59-bfd5-c9258f0be05c/volumes"
Aug 04 18:31:18.355175 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:18.355162    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="287f64cf-64ac-49fd-9643-7566074259ba" path="/var/lib/kubelet/pods/287f64cf-64ac-49fd-9643-7566074259ba/volumes"
Aug 04 18:31:18.355487 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:18.355475    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c63d3101-884b-4229-98a3-a4bcb4a9ec2f" path="/var/lib/kubelet/pods/c63d3101-884b-4229-98a3-a4bcb4a9ec2f/volumes"
Aug 04 18:31:58.482894 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:58.482780    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/2.log"
Aug 04 18:31:58.489031 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:58.489005    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 18:31:58.490275 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:58.490253    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/2.log"
Aug 04 18:31:58.495662 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:31:58.495642    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 18:33:22.192586 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.192552    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6"]
Aug 04 18:33:22.193197 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.193138    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c63d3101-884b-4229-98a3-a4bcb4a9ec2f" containerName="payload-pre-processing"
Aug 04 18:33:22.193197 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.193155    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c63d3101-884b-4229-98a3-a4bcb4a9ec2f" containerName="payload-pre-processing"
Aug 04 18:33:22.193197 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.193184    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="287f64cf-64ac-49fd-9643-7566074259ba" containerName="payload-processing"
Aug 04 18:33:22.193197 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.193193    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="287f64cf-64ac-49fd-9643-7566074259ba" containerName="payload-processing"
Aug 04 18:33:22.193425 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.193224    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08c6f92a-31c7-4a59-bfd5-c9258f0be05c" containerName="maas-api"
Aug 04 18:33:22.193425 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.193232    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="08c6f92a-31c7-4a59-bfd5-c9258f0be05c" containerName="maas-api"
Aug 04 18:33:22.193425 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.193341    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="08c6f92a-31c7-4a59-bfd5-c9258f0be05c" containerName="maas-api"
Aug 04 18:33:22.193425 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.193353    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c63d3101-884b-4229-98a3-a4bcb4a9ec2f" containerName="payload-pre-processing"
Aug 04 18:33:22.193425 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.193365    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="287f64cf-64ac-49fd-9643-7566074259ba" containerName="payload-processing"
Aug 04 18:33:22.196956 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.196933    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6"
Aug 04 18:33:22.199924 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.199899    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-c70eb3-serving-cert\""
Aug 04 18:33:22.207801 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.207779    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6"]
Aug 04 18:33:22.251831 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.251807    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f38973a9-9b98-492d-bbd7-5630dd7c92c5-maas-api-tls\") pod \"maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6\" (UID: \"f38973a9-9b98-492d-bbd7-5630dd7c92c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6"
Aug 04 18:33:22.251942 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.251869    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f38973a9-9b98-492d-bbd7-5630dd7c92c5-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6\" (UID: \"f38973a9-9b98-492d-bbd7-5630dd7c92c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6"
Aug 04 18:33:22.251942 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.251928    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrhs9\" (UniqueName: \"kubernetes.io/projected/f38973a9-9b98-492d-bbd7-5630dd7c92c5-kube-api-access-qrhs9\") pod \"maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6\" (UID: \"f38973a9-9b98-492d-bbd7-5630dd7c92c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6"
Aug 04 18:33:22.352319 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.352293    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f38973a9-9b98-492d-bbd7-5630dd7c92c5-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6\" (UID: \"f38973a9-9b98-492d-bbd7-5630dd7c92c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6"
Aug 04 18:33:22.352459 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.352330    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrhs9\" (UniqueName: \"kubernetes.io/projected/f38973a9-9b98-492d-bbd7-5630dd7c92c5-kube-api-access-qrhs9\") pod \"maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6\" (UID: \"f38973a9-9b98-492d-bbd7-5630dd7c92c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6"
Aug 04 18:33:22.352459 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.352390    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f38973a9-9b98-492d-bbd7-5630dd7c92c5-maas-api-tls\") pod \"maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6\" (UID: \"f38973a9-9b98-492d-bbd7-5630dd7c92c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6"
Aug 04 18:33:22.352569 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:33:22.352482    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-serving-cert: secret "maas-api-e2e-inf-a-c70eb3-serving-cert" not found
Aug 04 18:33:22.352569 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:33:22.352541    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f38973a9-9b98-492d-bbd7-5630dd7c92c5-maas-api-tls podName:f38973a9-9b98-492d-bbd7-5630dd7c92c5 nodeName:}" failed. No retries permitted until 2026-08-04 18:33:22.852524454 +0000 UTC m=+2485.035684609 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f38973a9-9b98-492d-bbd7-5630dd7c92c5-maas-api-tls") pod "maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6" (UID: "f38973a9-9b98-492d-bbd7-5630dd7c92c5") : secret "maas-api-e2e-inf-a-c70eb3-serving-cert" not found
Aug 04 18:33:22.352957 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.352936    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f38973a9-9b98-492d-bbd7-5630dd7c92c5-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6\" (UID: \"f38973a9-9b98-492d-bbd7-5630dd7c92c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6"
Aug 04 18:33:22.364121 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.364100    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrhs9\" (UniqueName: \"kubernetes.io/projected/f38973a9-9b98-492d-bbd7-5630dd7c92c5-kube-api-access-qrhs9\") pod \"maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6\" (UID: \"f38973a9-9b98-492d-bbd7-5630dd7c92c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6"
Aug 04 18:33:22.856795 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.856761    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f38973a9-9b98-492d-bbd7-5630dd7c92c5-maas-api-tls\") pod \"maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6\" (UID: \"f38973a9-9b98-492d-bbd7-5630dd7c92c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6"
Aug 04 18:33:22.859325 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.859300    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f38973a9-9b98-492d-bbd7-5630dd7c92c5-maas-api-tls\") pod \"maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6\" (UID: \"f38973a9-9b98-492d-bbd7-5630dd7c92c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6"
Aug 04 18:33:22.903416 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.903390    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75"]
Aug 04 18:33:22.906885 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.906871    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75"
Aug 04 18:33:22.909134 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.909116    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-c70eb3\""
Aug 04 18:33:22.914851 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.914833    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75"]
Aug 04 18:33:22.931384 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.931359    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh"]
Aug 04 18:33:22.936949 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.936932    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh"
Aug 04 18:33:22.939150 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.939131    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-c70eb3-dockercfg-5h878\""
Aug 04 18:33:22.946047 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.946026    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh"]
Aug 04 18:33:22.957393 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.957365    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqqfn\" (UniqueName: \"kubernetes.io/projected/f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576-kube-api-access-nqqfn\") pod \"payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75\" (UID: \"f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75"
Aug 04 18:33:22.957490 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:22.957418    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75\" (UID: \"f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75"
Aug 04 18:33:23.058803 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:23.058773    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqqfn\" (UniqueName: \"kubernetes.io/projected/f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576-kube-api-access-nqqfn\") pod \"payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75\" (UID: \"f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75"
Aug 04 18:33:23.058963 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:23.058811    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75\" (UID: \"f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75"
Aug 04 18:33:23.058963 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:23.058834    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/023952d7-d30a-4894-b6dd-33f269a5aca3-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh\" (UID: \"023952d7-d30a-4894-b6dd-33f269a5aca3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh"
Aug 04 18:33:23.058963 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:23.058861    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sk96\" (UniqueName: \"kubernetes.io/projected/023952d7-d30a-4894-b6dd-33f269a5aca3-kube-api-access-6sk96\") pod \"payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh\" (UID: \"023952d7-d30a-4894-b6dd-33f269a5aca3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh"
Aug 04 18:33:23.059537 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:23.059515    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75\" (UID: \"f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75"
Aug 04 18:33:23.067982 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:23.067959    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqqfn\" (UniqueName: \"kubernetes.io/projected/f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576-kube-api-access-nqqfn\") pod \"payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75\" (UID: \"f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75"
Aug 04 18:33:23.109310 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:23.109256    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6"
Aug 04 18:33:23.159883 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:23.159848    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/023952d7-d30a-4894-b6dd-33f269a5aca3-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh\" (UID: \"023952d7-d30a-4894-b6dd-33f269a5aca3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh"
Aug 04 18:33:23.160103 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:23.159913    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sk96\" (UniqueName: \"kubernetes.io/projected/023952d7-d30a-4894-b6dd-33f269a5aca3-kube-api-access-6sk96\") pod \"payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh\" (UID: \"023952d7-d30a-4894-b6dd-33f269a5aca3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh"
Aug 04 18:33:23.161184 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:23.161150    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/023952d7-d30a-4894-b6dd-33f269a5aca3-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh\" (UID: \"023952d7-d30a-4894-b6dd-33f269a5aca3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh"
Aug 04 18:33:23.170318 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:23.170287    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sk96\" (UniqueName: \"kubernetes.io/projected/023952d7-d30a-4894-b6dd-33f269a5aca3-kube-api-access-6sk96\") pod \"payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh\" (UID: \"023952d7-d30a-4894-b6dd-33f269a5aca3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh"
Aug 04 18:33:23.218670 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:23.218319    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75"
Aug 04 18:33:23.248003 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:23.247962    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh"
Aug 04 18:33:23.248498 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:23.248471    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6"]
Aug 04 18:33:23.250808 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:33:23.250786    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf38973a9_9b98_492d_bbd7_5630dd7c92c5.slice/crio-4ab9c95300117758c7cd61535d74b7e42b6faec08365d777099ecf849d2185a1 WatchSource:0}: Error finding container 4ab9c95300117758c7cd61535d74b7e42b6faec08365d777099ecf849d2185a1: Status 404 returned error can't find the container with id 4ab9c95300117758c7cd61535d74b7e42b6faec08365d777099ecf849d2185a1
Aug 04 18:33:23.367420 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:23.367392    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75"]
Aug 04 18:33:23.368923 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:33:23.368895    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1c1f0c6_0de9_454f_aadb_d1ba8f0eb576.slice/crio-8b375e61210b06b1a3352bb369e583b6430cb142516e0803de09cbce1a754711 WatchSource:0}: Error finding container 8b375e61210b06b1a3352bb369e583b6430cb142516e0803de09cbce1a754711: Status 404 returned error can't find the container with id 8b375e61210b06b1a3352bb369e583b6430cb142516e0803de09cbce1a754711
Aug 04 18:33:23.607162 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:23.606966    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh"]
Aug 04 18:33:23.609343 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:33:23.609315    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod023952d7_d30a_4894_b6dd_33f269a5aca3.slice/crio-9cbcffc6d5f4e02b2e860427e07af12cb0afcb58cab0a205d99d0ec3b180537f WatchSource:0}: Error finding container 9cbcffc6d5f4e02b2e860427e07af12cb0afcb58cab0a205d99d0ec3b180537f: Status 404 returned error can't find the container with id 9cbcffc6d5f4e02b2e860427e07af12cb0afcb58cab0a205d99d0ec3b180537f
Aug 04 18:33:24.102125 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:24.102094    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75" event={"ID":"f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576","Type":"ContainerStarted","Data":"732d5c5d215b670218f8ad2adc34a4c418bd73ceba0d7faabe22643ab17943ce"}
Aug 04 18:33:24.102316 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:24.102144    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75" event={"ID":"f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576","Type":"ContainerStarted","Data":"8b375e61210b06b1a3352bb369e583b6430cb142516e0803de09cbce1a754711"}
Aug 04 18:33:24.102316 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:24.102208    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75"
Aug 04 18:33:24.103785 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:24.103762    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6" event={"ID":"f38973a9-9b98-492d-bbd7-5630dd7c92c5","Type":"ContainerStarted","Data":"a2bf97d42d02137cdd62a6a08eb35f63debd74a0d2969d7b32203d490360b1a7"}
Aug 04 18:33:24.103926 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:24.103813    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6"
Aug 04 18:33:24.103926 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:24.103839    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6" event={"ID":"f38973a9-9b98-492d-bbd7-5630dd7c92c5","Type":"ContainerStarted","Data":"4ab9c95300117758c7cd61535d74b7e42b6faec08365d777099ecf849d2185a1"}
Aug 04 18:33:24.105427 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:24.105401    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh" event={"ID":"023952d7-d30a-4894-b6dd-33f269a5aca3","Type":"ContainerStarted","Data":"b977eb371158c7ab7340938349033b5c923597e9cf5612c2cb8e6b0e0fbcc39a"}
Aug 04 18:33:24.105556 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:24.105444    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh"
Aug 04 18:33:24.105556 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:24.105459    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh" event={"ID":"023952d7-d30a-4894-b6dd-33f269a5aca3","Type":"ContainerStarted","Data":"9cbcffc6d5f4e02b2e860427e07af12cb0afcb58cab0a205d99d0ec3b180537f"}
Aug 04 18:33:24.123983 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:24.123930    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75" podStartSLOduration=2.123914888 podStartE2EDuration="2.123914888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:33:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:33:24.123017696 +0000 UTC m=+2486.306177871" watchObservedRunningTime="2026-08-04 18:33:24.123914888 +0000 UTC m=+2486.307075069"
Aug 04 18:33:24.146753 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:24.146711    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6" podStartSLOduration=1.7385001020000002 podStartE2EDuration="2.146699098s" totalImagesPullingTime="408.198996ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:33:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:33:24.144885525 +0000 UTC m=+2486.328045702" watchObservedRunningTime="2026-08-04 18:33:24.146699098 +0000 UTC m=+2486.329859275"
Aug 04 18:33:24.165367 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:24.165327    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh" podStartSLOduration=2.16531391 podStartE2EDuration="2.16531391s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:33:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:33:24.163600797 +0000 UTC m=+2486.346760973" watchObservedRunningTime="2026-08-04 18:33:24.16531391 +0000 UTC m=+2486.348474086"
Aug 04 18:33:30.115965 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:30.115937    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6"
Aug 04 18:33:35.112390 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:35.112359    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh"
Aug 04 18:33:35.112813 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:35.112468    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75"
Aug 04 18:33:53.783170 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:53.783092    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw"]
Aug 04 18:33:53.786830 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:53.786808    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw"
Aug 04 18:33:53.789426 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:53.789407    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-7f7c0a-serving-cert\""
Aug 04 18:33:53.798075 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:53.798056    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw"]
Aug 04 18:33:53.821529 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:53.821498    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/713b82fb-b09f-4274-9afd-264dfed8d337-maas-api-tls\") pod \"maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw\" (UID: \"713b82fb-b09f-4274-9afd-264dfed8d337\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw"
Aug 04 18:33:53.821629 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:53.821554    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24rx9\" (UniqueName: \"kubernetes.io/projected/713b82fb-b09f-4274-9afd-264dfed8d337-kube-api-access-24rx9\") pod \"maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw\" (UID: \"713b82fb-b09f-4274-9afd-264dfed8d337\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw"
Aug 04 18:33:53.821629 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:53.821613    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/713b82fb-b09f-4274-9afd-264dfed8d337-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw\" (UID: \"713b82fb-b09f-4274-9afd-264dfed8d337\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw"
Aug 04 18:33:53.922603 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:53.922571    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/713b82fb-b09f-4274-9afd-264dfed8d337-maas-api-tls\") pod \"maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw\" (UID: \"713b82fb-b09f-4274-9afd-264dfed8d337\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw"
Aug 04 18:33:53.922756 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:53.922640    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24rx9\" (UniqueName: \"kubernetes.io/projected/713b82fb-b09f-4274-9afd-264dfed8d337-kube-api-access-24rx9\") pod \"maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw\" (UID: \"713b82fb-b09f-4274-9afd-264dfed8d337\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw"
Aug 04 18:33:53.922756 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:53.922673    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/713b82fb-b09f-4274-9afd-264dfed8d337-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw\" (UID: \"713b82fb-b09f-4274-9afd-264dfed8d337\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw"
Aug 04 18:33:53.922756 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:33:53.922736    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-serving-cert: secret "maas-api-e2e-inf-b-7f7c0a-serving-cert" not found
Aug 04 18:33:53.922899 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:33:53.922823    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/713b82fb-b09f-4274-9afd-264dfed8d337-maas-api-tls podName:713b82fb-b09f-4274-9afd-264dfed8d337 nodeName:}" failed. No retries permitted until 2026-08-04 18:33:54.422802693 +0000 UTC m=+2516.605962864 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/713b82fb-b09f-4274-9afd-264dfed8d337-maas-api-tls") pod "maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw" (UID: "713b82fb-b09f-4274-9afd-264dfed8d337") : secret "maas-api-e2e-inf-b-7f7c0a-serving-cert" not found
Aug 04 18:33:53.923538 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:53.923503    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/713b82fb-b09f-4274-9afd-264dfed8d337-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw\" (UID: \"713b82fb-b09f-4274-9afd-264dfed8d337\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw"
Aug 04 18:33:53.933548 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:53.933525    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24rx9\" (UniqueName: \"kubernetes.io/projected/713b82fb-b09f-4274-9afd-264dfed8d337-kube-api-access-24rx9\") pod \"maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw\" (UID: \"713b82fb-b09f-4274-9afd-264dfed8d337\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw"
Aug 04 18:33:54.427100 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:54.427067    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/713b82fb-b09f-4274-9afd-264dfed8d337-maas-api-tls\") pod \"maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw\" (UID: \"713b82fb-b09f-4274-9afd-264dfed8d337\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw"
Aug 04 18:33:54.429634 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:54.429603    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/713b82fb-b09f-4274-9afd-264dfed8d337-maas-api-tls\") pod \"maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw\" (UID: \"713b82fb-b09f-4274-9afd-264dfed8d337\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw"
Aug 04 18:33:54.600616 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:54.600585    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng"]
Aug 04 18:33:54.605203 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:54.605186    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng"
Aug 04 18:33:54.609674 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:54.609654    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-7f7c0a\""
Aug 04 18:33:54.615028 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:54.615005    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng"]
Aug 04 18:33:54.629575 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:54.629557    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f93ea7de-f642-4e45-a201-0a3b618b993e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng\" (UID: \"f93ea7de-f642-4e45-a201-0a3b618b993e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng"
Aug 04 18:33:54.629678 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:54.629661    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9crbl\" (UniqueName: \"kubernetes.io/projected/f93ea7de-f642-4e45-a201-0a3b618b993e-kube-api-access-9crbl\") pod \"payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng\" (UID: \"f93ea7de-f642-4e45-a201-0a3b618b993e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng"
Aug 04 18:33:54.662828 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:54.662801    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x"]
Aug 04 18:33:54.673086 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:54.673066    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x"
Aug 04 18:33:54.676671 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:54.676652    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-7f7c0a-dockercfg-mmwj7\""
Aug 04 18:33:54.687073 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:54.687049    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x"]
Aug 04 18:33:54.701498 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:54.701477    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw"
Aug 04 18:33:54.730699 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:54.730676    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9crbl\" (UniqueName: \"kubernetes.io/projected/f93ea7de-f642-4e45-a201-0a3b618b993e-kube-api-access-9crbl\") pod \"payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng\" (UID: \"f93ea7de-f642-4e45-a201-0a3b618b993e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng"
Aug 04 18:33:54.730810 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:54.730714    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5076398-949a-4bee-abe1-6799071922aa-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x\" (UID: \"d5076398-949a-4bee-abe1-6799071922aa\") " pod="openshift-ingress/payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x"
Aug 04 18:33:54.730810 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:54.730732    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwj5g\" (UniqueName: \"kubernetes.io/projected/d5076398-949a-4bee-abe1-6799071922aa-kube-api-access-gwj5g\") pod \"payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x\" (UID: \"d5076398-949a-4bee-abe1-6799071922aa\") " pod="openshift-ingress/payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x"
Aug 04 18:33:54.730939 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:54.730815    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f93ea7de-f642-4e45-a201-0a3b618b993e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng\" (UID: \"f93ea7de-f642-4e45-a201-0a3b618b993e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng"
Aug 04 18:33:54.731508 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:54.731490    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f93ea7de-f642-4e45-a201-0a3b618b993e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng\" (UID: \"f93ea7de-f642-4e45-a201-0a3b618b993e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng"
Aug 04 18:33:54.740651 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:54.740628    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9crbl\" (UniqueName: \"kubernetes.io/projected/f93ea7de-f642-4e45-a201-0a3b618b993e-kube-api-access-9crbl\") pod \"payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng\" (UID: \"f93ea7de-f642-4e45-a201-0a3b618b993e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng"
Aug 04 18:33:54.831825 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:54.831785    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5076398-949a-4bee-abe1-6799071922aa-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x\" (UID: \"d5076398-949a-4bee-abe1-6799071922aa\") " pod="openshift-ingress/payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x"
Aug 04 18:33:54.832220 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:54.831840    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwj5g\" (UniqueName: \"kubernetes.io/projected/d5076398-949a-4bee-abe1-6799071922aa-kube-api-access-gwj5g\") pod \"payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x\" (UID: \"d5076398-949a-4bee-abe1-6799071922aa\") " pod="openshift-ingress/payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x"
Aug 04 18:33:54.832812 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:54.832786    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5076398-949a-4bee-abe1-6799071922aa-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x\" (UID: \"d5076398-949a-4bee-abe1-6799071922aa\") " pod="openshift-ingress/payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x"
Aug 04 18:33:54.840826 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:54.840795    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwj5g\" (UniqueName: \"kubernetes.io/projected/d5076398-949a-4bee-abe1-6799071922aa-kube-api-access-gwj5g\") pod \"payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x\" (UID: \"d5076398-949a-4bee-abe1-6799071922aa\") " pod="openshift-ingress/payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x"
Aug 04 18:33:54.852329 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:54.852308    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw"]
Aug 04 18:33:54.855410 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:33:54.855389    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod713b82fb_b09f_4274_9afd_264dfed8d337.slice/crio-dbd3c06d6e0b41bf6f144372e4ede9005a399df3bd9a49903878c5078f5a851c WatchSource:0}: Error finding container dbd3c06d6e0b41bf6f144372e4ede9005a399df3bd9a49903878c5078f5a851c: Status 404 returned error can't find the container with id dbd3c06d6e0b41bf6f144372e4ede9005a399df3bd9a49903878c5078f5a851c
Aug 04 18:33:54.916146 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:54.916125    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng"
Aug 04 18:33:54.982787 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:54.982730    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x"
Aug 04 18:33:55.056947 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:55.056708    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng"]
Aug 04 18:33:55.059371 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:33:55.059334    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf93ea7de_f642_4e45_a201_0a3b618b993e.slice/crio-ae8818426badca6defec1f3b9d6ef30bca31bdb0d518f93538bbd9b9ae311467 WatchSource:0}: Error finding container ae8818426badca6defec1f3b9d6ef30bca31bdb0d518f93538bbd9b9ae311467: Status 404 returned error can't find the container with id ae8818426badca6defec1f3b9d6ef30bca31bdb0d518f93538bbd9b9ae311467
Aug 04 18:33:55.138541 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:55.138516    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x"]
Aug 04 18:33:55.140578 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:33:55.140554    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5076398_949a_4bee_abe1_6799071922aa.slice/crio-4094961fd15569f80032c5695a0550af0ee0643cdbed3b12d745a1c59e67d01f WatchSource:0}: Error finding container 4094961fd15569f80032c5695a0550af0ee0643cdbed3b12d745a1c59e67d01f: Status 404 returned error can't find the container with id 4094961fd15569f80032c5695a0550af0ee0643cdbed3b12d745a1c59e67d01f
Aug 04 18:33:55.255688 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:55.255599    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw" event={"ID":"713b82fb-b09f-4274-9afd-264dfed8d337","Type":"ContainerStarted","Data":"dbd3c06d6e0b41bf6f144372e4ede9005a399df3bd9a49903878c5078f5a851c"}
Aug 04 18:33:55.257313 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:55.257287    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x" event={"ID":"d5076398-949a-4bee-abe1-6799071922aa","Type":"ContainerStarted","Data":"8b6cd79ae8d6a1c33f41d497853886c215a0d0e15fd64f73006a9e8666dce80e"}
Aug 04 18:33:55.257469 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:55.257332    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x" event={"ID":"d5076398-949a-4bee-abe1-6799071922aa","Type":"ContainerStarted","Data":"4094961fd15569f80032c5695a0550af0ee0643cdbed3b12d745a1c59e67d01f"}
Aug 04 18:33:55.257469 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:55.257381    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x"
Aug 04 18:33:55.260310 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:55.260289    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng" event={"ID":"f93ea7de-f642-4e45-a201-0a3b618b993e","Type":"ContainerStarted","Data":"e75dce289dfa3b059cef70024197d462f9b89868eb20e2dd5be3e3366a3812ee"}
Aug 04 18:33:55.260418 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:55.260328    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng" event={"ID":"f93ea7de-f642-4e45-a201-0a3b618b993e","Type":"ContainerStarted","Data":"ae8818426badca6defec1f3b9d6ef30bca31bdb0d518f93538bbd9b9ae311467"}
Aug 04 18:33:55.260478 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:55.260453    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng"
Aug 04 18:33:55.275731 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:55.275687    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x" podStartSLOduration=1.2756704380000001 podStartE2EDuration="1.275670438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:33:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:33:55.274616059 +0000 UTC m=+2517.457776235" watchObservedRunningTime="2026-08-04 18:33:55.275670438 +0000 UTC m=+2517.458830618"
Aug 04 18:33:55.292917 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:55.292877    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng" podStartSLOduration=1.29286611 podStartE2EDuration="1.29286611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:33:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:33:55.290947075 +0000 UTC m=+2517.474107251" watchObservedRunningTime="2026-08-04 18:33:55.29286611 +0000 UTC m=+2517.476026306"
Aug 04 18:33:56.267795 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:56.267752    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw" event={"ID":"713b82fb-b09f-4274-9afd-264dfed8d337","Type":"ContainerStarted","Data":"e5c5edf2cbd97bb461f712da8b0a93ac9ea6b8dac97c4067382c91f2ba151a1a"}
Aug 04 18:33:56.268259 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:56.268091    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw"
Aug 04 18:33:56.286304 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:33:56.285782    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw" podStartSLOduration=2.8076685919999997 podStartE2EDuration="3.28576524s" totalImagesPullingTime="478.096648ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:33:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:33:56.285033829 +0000 UTC m=+2518.468194008" watchObservedRunningTime="2026-08-04 18:33:56.28576524 +0000 UTC m=+2518.468925415"
Aug 04 18:34:02.278047 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:34:02.278014    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw"
Aug 04 18:34:06.269906 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:34:06.269881    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x"
Aug 04 18:34:06.270300 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:34:06.270167    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng"
Aug 04 18:36:04.752010 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:04.751961    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6"]
Aug 04 18:36:04.752507 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:04.752212    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6" podUID="f38973a9-9b98-492d-bbd7-5630dd7c92c5" containerName="maas-api" containerID="cri-o://a2bf97d42d02137cdd62a6a08eb35f63debd74a0d2969d7b32203d490360b1a7" gracePeriod=30
Aug 04 18:36:04.835350 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:04.835319    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh"]
Aug 04 18:36:04.835605 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:04.835550    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh" podUID="023952d7-d30a-4894-b6dd-33f269a5aca3" containerName="payload-processing" containerID="cri-o://b977eb371158c7ab7340938349033b5c923597e9cf5612c2cb8e6b0e0fbcc39a" gracePeriod=30
Aug 04 18:36:04.848213 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:04.848186    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75"]
Aug 04 18:36:04.848430 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:04.848402    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75" podUID="f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576" containerName="payload-pre-processing" containerID="cri-o://732d5c5d215b670218f8ad2adc34a4c418bd73ceba0d7faabe22643ab17943ce" gracePeriod=30
Aug 04 18:36:04.905638 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:04.905609    2560 generic.go:356] "Generic (PLEG): container finished" podID="f38973a9-9b98-492d-bbd7-5630dd7c92c5" containerID="a2bf97d42d02137cdd62a6a08eb35f63debd74a0d2969d7b32203d490360b1a7" exitCode=0
Aug 04 18:36:04.905790 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:04.905687    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6" event={"ID":"f38973a9-9b98-492d-bbd7-5630dd7c92c5","Type":"ContainerDied","Data":"a2bf97d42d02137cdd62a6a08eb35f63debd74a0d2969d7b32203d490360b1a7"}
Aug 04 18:36:05.099451 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.099422    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6"
Aug 04 18:36:05.139584 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.139560    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh"
Aug 04 18:36:05.168019 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.167968    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75"
Aug 04 18:36:05.197805 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.197782    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f38973a9-9b98-492d-bbd7-5630dd7c92c5-maas-api-tls\") pod \"f38973a9-9b98-492d-bbd7-5630dd7c92c5\" (UID: \"f38973a9-9b98-492d-bbd7-5630dd7c92c5\") "
Aug 04 18:36:05.197938 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.197825    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f38973a9-9b98-492d-bbd7-5630dd7c92c5-openshift-service-ca\") pod \"f38973a9-9b98-492d-bbd7-5630dd7c92c5\" (UID: \"f38973a9-9b98-492d-bbd7-5630dd7c92c5\") "
Aug 04 18:36:05.197938 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.197851    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6sk96\" (UniqueName: \"kubernetes.io/projected/023952d7-d30a-4894-b6dd-33f269a5aca3-kube-api-access-6sk96\") pod \"023952d7-d30a-4894-b6dd-33f269a5aca3\" (UID: \"023952d7-d30a-4894-b6dd-33f269a5aca3\") "
Aug 04 18:36:05.197938 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.197895    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nqqfn\" (UniqueName: \"kubernetes.io/projected/f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576-kube-api-access-nqqfn\") pod \"f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576\" (UID: \"f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576\") "
Aug 04 18:36:05.198131 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.197944    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/023952d7-d30a-4894-b6dd-33f269a5aca3-plugins-config-volume\") pod \"023952d7-d30a-4894-b6dd-33f269a5aca3\" (UID: \"023952d7-d30a-4894-b6dd-33f269a5aca3\") "
Aug 04 18:36:05.198131 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.198007    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576-plugins-config-volume\") pod \"f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576\" (UID: \"f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576\") "
Aug 04 18:36:05.198131 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.198038    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qrhs9\" (UniqueName: \"kubernetes.io/projected/f38973a9-9b98-492d-bbd7-5630dd7c92c5-kube-api-access-qrhs9\") pod \"f38973a9-9b98-492d-bbd7-5630dd7c92c5\" (UID: \"f38973a9-9b98-492d-bbd7-5630dd7c92c5\") "
Aug 04 18:36:05.198484 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.198457    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f38973a9-9b98-492d-bbd7-5630dd7c92c5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f38973a9-9b98-492d-bbd7-5630dd7c92c5" (UID: "f38973a9-9b98-492d-bbd7-5630dd7c92c5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:36:05.198884 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.198846    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576" (UID: "f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:36:05.198981 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.198952    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/023952d7-d30a-4894-b6dd-33f269a5aca3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "023952d7-d30a-4894-b6dd-33f269a5aca3" (UID: "023952d7-d30a-4894-b6dd-33f269a5aca3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:36:05.200256 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.200227    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f38973a9-9b98-492d-bbd7-5630dd7c92c5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f38973a9-9b98-492d-bbd7-5630dd7c92c5" (UID: "f38973a9-9b98-492d-bbd7-5630dd7c92c5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:36:05.200696 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.200675    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576-kube-api-access-nqqfn" (OuterVolumeSpecName: "kube-api-access-nqqfn") pod "f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576" (UID: "f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576"). InnerVolumeSpecName "kube-api-access-nqqfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:36:05.200813 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.200788    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f38973a9-9b98-492d-bbd7-5630dd7c92c5-kube-api-access-qrhs9" (OuterVolumeSpecName: "kube-api-access-qrhs9") pod "f38973a9-9b98-492d-bbd7-5630dd7c92c5" (UID: "f38973a9-9b98-492d-bbd7-5630dd7c92c5"). InnerVolumeSpecName "kube-api-access-qrhs9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:36:05.200981 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.200963    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/023952d7-d30a-4894-b6dd-33f269a5aca3-kube-api-access-6sk96" (OuterVolumeSpecName: "kube-api-access-6sk96") pod "023952d7-d30a-4894-b6dd-33f269a5aca3" (UID: "023952d7-d30a-4894-b6dd-33f269a5aca3"). InnerVolumeSpecName "kube-api-access-6sk96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:36:05.299616 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.299562    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/023952d7-d30a-4894-b6dd-33f269a5aca3-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:36:05.299616 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.299584    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:36:05.299616 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.299594    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrhs9\" (UniqueName: \"kubernetes.io/projected/f38973a9-9b98-492d-bbd7-5630dd7c92c5-kube-api-access-qrhs9\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:36:05.299616 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.299602    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f38973a9-9b98-492d-bbd7-5630dd7c92c5-maas-api-tls\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:36:05.299616 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.299611    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f38973a9-9b98-492d-bbd7-5630dd7c92c5-openshift-service-ca\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:36:05.299616 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.299619    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6sk96\" (UniqueName: \"kubernetes.io/projected/023952d7-d30a-4894-b6dd-33f269a5aca3-kube-api-access-6sk96\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:36:05.299886 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.299628    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqqfn\" (UniqueName: \"kubernetes.io/projected/f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576-kube-api-access-nqqfn\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:36:05.912165 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.912126    2560 generic.go:356] "Generic (PLEG): container finished" podID="f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576" containerID="732d5c5d215b670218f8ad2adc34a4c418bd73ceba0d7faabe22643ab17943ce" exitCode=0
Aug 04 18:36:05.912595 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.912175    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75"
Aug 04 18:36:05.912595 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.912206    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75" event={"ID":"f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576","Type":"ContainerDied","Data":"732d5c5d215b670218f8ad2adc34a4c418bd73ceba0d7faabe22643ab17943ce"}
Aug 04 18:36:05.912595 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.912261    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75" event={"ID":"f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576","Type":"ContainerDied","Data":"8b375e61210b06b1a3352bb369e583b6430cb142516e0803de09cbce1a754711"}
Aug 04 18:36:05.912595 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.912299    2560 scope.go:117] "RemoveContainer" containerID="732d5c5d215b670218f8ad2adc34a4c418bd73ceba0d7faabe22643ab17943ce"
Aug 04 18:36:05.913762 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.913602    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6"
Aug 04 18:36:05.913762 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.913675    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6" event={"ID":"f38973a9-9b98-492d-bbd7-5630dd7c92c5","Type":"ContainerDied","Data":"4ab9c95300117758c7cd61535d74b7e42b6faec08365d777099ecf849d2185a1"}
Aug 04 18:36:05.915313 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.915285    2560 generic.go:356] "Generic (PLEG): container finished" podID="023952d7-d30a-4894-b6dd-33f269a5aca3" containerID="b977eb371158c7ab7340938349033b5c923597e9cf5612c2cb8e6b0e0fbcc39a" exitCode=0
Aug 04 18:36:05.915436 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.915328    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh"
Aug 04 18:36:05.915436 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.915335    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh" event={"ID":"023952d7-d30a-4894-b6dd-33f269a5aca3","Type":"ContainerDied","Data":"b977eb371158c7ab7340938349033b5c923597e9cf5612c2cb8e6b0e0fbcc39a"}
Aug 04 18:36:05.915436 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.915375    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh" event={"ID":"023952d7-d30a-4894-b6dd-33f269a5aca3","Type":"ContainerDied","Data":"9cbcffc6d5f4e02b2e860427e07af12cb0afcb58cab0a205d99d0ec3b180537f"}
Aug 04 18:36:05.922368 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.922350    2560 scope.go:117] "RemoveContainer" containerID="732d5c5d215b670218f8ad2adc34a4c418bd73ceba0d7faabe22643ab17943ce"
Aug 04 18:36:05.922582 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:36:05.922566    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"732d5c5d215b670218f8ad2adc34a4c418bd73ceba0d7faabe22643ab17943ce\": container with ID starting with 732d5c5d215b670218f8ad2adc34a4c418bd73ceba0d7faabe22643ab17943ce not found: ID does not exist" containerID="732d5c5d215b670218f8ad2adc34a4c418bd73ceba0d7faabe22643ab17943ce"
Aug 04 18:36:05.922650 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.922589    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"732d5c5d215b670218f8ad2adc34a4c418bd73ceba0d7faabe22643ab17943ce"} err="failed to get container status \"732d5c5d215b670218f8ad2adc34a4c418bd73ceba0d7faabe22643ab17943ce\": rpc error: code = NotFound desc = could not find container \"732d5c5d215b670218f8ad2adc34a4c418bd73ceba0d7faabe22643ab17943ce\": container with ID starting with 732d5c5d215b670218f8ad2adc34a4c418bd73ceba0d7faabe22643ab17943ce not found: ID does not exist"
Aug 04 18:36:05.922650 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.922608    2560 scope.go:117] "RemoveContainer" containerID="a2bf97d42d02137cdd62a6a08eb35f63debd74a0d2969d7b32203d490360b1a7"
Aug 04 18:36:05.931147 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.931128    2560 scope.go:117] "RemoveContainer" containerID="b977eb371158c7ab7340938349033b5c923597e9cf5612c2cb8e6b0e0fbcc39a"
Aug 04 18:36:05.939158 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.939146    2560 scope.go:117] "RemoveContainer" containerID="b977eb371158c7ab7340938349033b5c923597e9cf5612c2cb8e6b0e0fbcc39a"
Aug 04 18:36:05.939415 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:36:05.939397    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b977eb371158c7ab7340938349033b5c923597e9cf5612c2cb8e6b0e0fbcc39a\": container with ID starting with b977eb371158c7ab7340938349033b5c923597e9cf5612c2cb8e6b0e0fbcc39a not found: ID does not exist" containerID="b977eb371158c7ab7340938349033b5c923597e9cf5612c2cb8e6b0e0fbcc39a"
Aug 04 18:36:05.939482 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.939419    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b977eb371158c7ab7340938349033b5c923597e9cf5612c2cb8e6b0e0fbcc39a"} err="failed to get container status \"b977eb371158c7ab7340938349033b5c923597e9cf5612c2cb8e6b0e0fbcc39a\": rpc error: code = NotFound desc = could not find container \"b977eb371158c7ab7340938349033b5c923597e9cf5612c2cb8e6b0e0fbcc39a\": container with ID starting with b977eb371158c7ab7340938349033b5c923597e9cf5612c2cb8e6b0e0fbcc39a not found: ID does not exist"
Aug 04 18:36:05.939735 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.939719    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75"]
Aug 04 18:36:05.945698 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.945680    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75"]
Aug 04 18:36:05.957138 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.957118    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6"]
Aug 04 18:36:05.961526 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.961508    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c70eb3-f99b94bf5-mbsw6"]
Aug 04 18:36:05.972392 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.972374    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh"]
Aug 04 18:36:05.979741 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:05.979722    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh"]
Aug 04 18:36:06.112158 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:06.112061    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c70eb3-ddf689bfb-5gx75" podUID="f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.66:9004: i/o timeout"
Aug 04 18:36:06.112158 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:06.112145    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-a-c70eb3-76d9848fcd-fcwjh" podUID="023952d7-d30a-4894-b6dd-33f269a5aca3" containerName="payload-processing" probeResult="failure" output="dial tcp 10.132.0.67:9004: i/o timeout"
Aug 04 18:36:06.353609 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:06.353579    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="023952d7-d30a-4894-b6dd-33f269a5aca3" path="/var/lib/kubelet/pods/023952d7-d30a-4894-b6dd-33f269a5aca3/volumes"
Aug 04 18:36:06.353933 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:06.353920    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576" path="/var/lib/kubelet/pods/f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576/volumes"
Aug 04 18:36:06.354304 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:06.354290    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f38973a9-9b98-492d-bbd7-5630dd7c92c5" path="/var/lib/kubelet/pods/f38973a9-9b98-492d-bbd7-5630dd7c92c5/volumes"
Aug 04 18:36:18.812578 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:18.812537    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw"]
Aug 04 18:36:18.813589 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:18.812806    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw" podUID="713b82fb-b09f-4274-9afd-264dfed8d337" containerName="maas-api" containerID="cri-o://e5c5edf2cbd97bb461f712da8b0a93ac9ea6b8dac97c4067382c91f2ba151a1a" gracePeriod=30
Aug 04 18:36:18.895129 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:18.895096    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x"]
Aug 04 18:36:18.895344 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:18.895321    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x" podUID="d5076398-949a-4bee-abe1-6799071922aa" containerName="payload-processing" containerID="cri-o://8b6cd79ae8d6a1c33f41d497853886c215a0d0e15fd64f73006a9e8666dce80e" gracePeriod=30
Aug 04 18:36:18.908865 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:18.908840    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng"]
Aug 04 18:36:18.909049 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:18.909026    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng" podUID="f93ea7de-f642-4e45-a201-0a3b618b993e" containerName="payload-pre-processing" containerID="cri-o://e75dce289dfa3b059cef70024197d462f9b89868eb20e2dd5be3e3366a3812ee" gracePeriod=30
Aug 04 18:36:18.981134 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:18.981103    2560 generic.go:356] "Generic (PLEG): container finished" podID="713b82fb-b09f-4274-9afd-264dfed8d337" containerID="e5c5edf2cbd97bb461f712da8b0a93ac9ea6b8dac97c4067382c91f2ba151a1a" exitCode=0
Aug 04 18:36:18.981242 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:18.981181    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw" event={"ID":"713b82fb-b09f-4274-9afd-264dfed8d337","Type":"ContainerDied","Data":"e5c5edf2cbd97bb461f712da8b0a93ac9ea6b8dac97c4067382c91f2ba151a1a"}
Aug 04 18:36:19.168774 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.168749    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw"
Aug 04 18:36:19.179228 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.179205    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x"
Aug 04 18:36:19.187899 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.187878    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng"
Aug 04 18:36:19.211791 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.211771    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/713b82fb-b09f-4274-9afd-264dfed8d337-maas-api-tls\") pod \"713b82fb-b09f-4274-9afd-264dfed8d337\" (UID: \"713b82fb-b09f-4274-9afd-264dfed8d337\") "
Aug 04 18:36:19.211915 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.211861    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24rx9\" (UniqueName: \"kubernetes.io/projected/713b82fb-b09f-4274-9afd-264dfed8d337-kube-api-access-24rx9\") pod \"713b82fb-b09f-4274-9afd-264dfed8d337\" (UID: \"713b82fb-b09f-4274-9afd-264dfed8d337\") "
Aug 04 18:36:19.211915 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.211912    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/713b82fb-b09f-4274-9afd-264dfed8d337-openshift-service-ca\") pod \"713b82fb-b09f-4274-9afd-264dfed8d337\" (UID: \"713b82fb-b09f-4274-9afd-264dfed8d337\") "
Aug 04 18:36:19.212346 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.212317    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/713b82fb-b09f-4274-9afd-264dfed8d337-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "713b82fb-b09f-4274-9afd-264dfed8d337" (UID: "713b82fb-b09f-4274-9afd-264dfed8d337"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:36:19.213954 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.213930    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/713b82fb-b09f-4274-9afd-264dfed8d337-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "713b82fb-b09f-4274-9afd-264dfed8d337" (UID: "713b82fb-b09f-4274-9afd-264dfed8d337"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:36:19.214074 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.213961    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/713b82fb-b09f-4274-9afd-264dfed8d337-kube-api-access-24rx9" (OuterVolumeSpecName: "kube-api-access-24rx9") pod "713b82fb-b09f-4274-9afd-264dfed8d337" (UID: "713b82fb-b09f-4274-9afd-264dfed8d337"). InnerVolumeSpecName "kube-api-access-24rx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:36:19.313450 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.313412    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5076398-949a-4bee-abe1-6799071922aa-plugins-config-volume\") pod \"d5076398-949a-4bee-abe1-6799071922aa\" (UID: \"d5076398-949a-4bee-abe1-6799071922aa\") "
Aug 04 18:36:19.313450 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.313457    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwj5g\" (UniqueName: \"kubernetes.io/projected/d5076398-949a-4bee-abe1-6799071922aa-kube-api-access-gwj5g\") pod \"d5076398-949a-4bee-abe1-6799071922aa\" (UID: \"d5076398-949a-4bee-abe1-6799071922aa\") "
Aug 04 18:36:19.313663 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.313494    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9crbl\" (UniqueName: \"kubernetes.io/projected/f93ea7de-f642-4e45-a201-0a3b618b993e-kube-api-access-9crbl\") pod \"f93ea7de-f642-4e45-a201-0a3b618b993e\" (UID: \"f93ea7de-f642-4e45-a201-0a3b618b993e\") "
Aug 04 18:36:19.313663 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.313534    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f93ea7de-f642-4e45-a201-0a3b618b993e-plugins-config-volume\") pod \"f93ea7de-f642-4e45-a201-0a3b618b993e\" (UID: \"f93ea7de-f642-4e45-a201-0a3b618b993e\") "
Aug 04 18:36:19.313780 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.313727    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24rx9\" (UniqueName: \"kubernetes.io/projected/713b82fb-b09f-4274-9afd-264dfed8d337-kube-api-access-24rx9\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:36:19.313780 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.313738    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/713b82fb-b09f-4274-9afd-264dfed8d337-openshift-service-ca\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:36:19.313780 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.313747    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/713b82fb-b09f-4274-9afd-264dfed8d337-maas-api-tls\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:36:19.313964 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.313799    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5076398-949a-4bee-abe1-6799071922aa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d5076398-949a-4bee-abe1-6799071922aa" (UID: "d5076398-949a-4bee-abe1-6799071922aa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:36:19.314065 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.314009    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f93ea7de-f642-4e45-a201-0a3b618b993e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f93ea7de-f642-4e45-a201-0a3b618b993e" (UID: "f93ea7de-f642-4e45-a201-0a3b618b993e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:36:19.315646 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.315622    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f93ea7de-f642-4e45-a201-0a3b618b993e-kube-api-access-9crbl" (OuterVolumeSpecName: "kube-api-access-9crbl") pod "f93ea7de-f642-4e45-a201-0a3b618b993e" (UID: "f93ea7de-f642-4e45-a201-0a3b618b993e"). InnerVolumeSpecName "kube-api-access-9crbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:36:19.315704 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.315667    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5076398-949a-4bee-abe1-6799071922aa-kube-api-access-gwj5g" (OuterVolumeSpecName: "kube-api-access-gwj5g") pod "d5076398-949a-4bee-abe1-6799071922aa" (UID: "d5076398-949a-4bee-abe1-6799071922aa"). InnerVolumeSpecName "kube-api-access-gwj5g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:36:19.414506 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.414466    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5076398-949a-4bee-abe1-6799071922aa-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:36:19.414506 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.414501    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwj5g\" (UniqueName: \"kubernetes.io/projected/d5076398-949a-4bee-abe1-6799071922aa-kube-api-access-gwj5g\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:36:19.414706 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.414516    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9crbl\" (UniqueName: \"kubernetes.io/projected/f93ea7de-f642-4e45-a201-0a3b618b993e-kube-api-access-9crbl\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:36:19.414706 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.414531    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f93ea7de-f642-4e45-a201-0a3b618b993e-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:36:19.987473 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.987439    2560 generic.go:356] "Generic (PLEG): container finished" podID="d5076398-949a-4bee-abe1-6799071922aa" containerID="8b6cd79ae8d6a1c33f41d497853886c215a0d0e15fd64f73006a9e8666dce80e" exitCode=0
Aug 04 18:36:19.987917 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.987486    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x"
Aug 04 18:36:19.987917 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.987522    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x" event={"ID":"d5076398-949a-4bee-abe1-6799071922aa","Type":"ContainerDied","Data":"8b6cd79ae8d6a1c33f41d497853886c215a0d0e15fd64f73006a9e8666dce80e"}
Aug 04 18:36:19.987917 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.987581    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x" event={"ID":"d5076398-949a-4bee-abe1-6799071922aa","Type":"ContainerDied","Data":"4094961fd15569f80032c5695a0550af0ee0643cdbed3b12d745a1c59e67d01f"}
Aug 04 18:36:19.987917 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.987613    2560 scope.go:117] "RemoveContainer" containerID="8b6cd79ae8d6a1c33f41d497853886c215a0d0e15fd64f73006a9e8666dce80e"
Aug 04 18:36:19.988956 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.988932    2560 generic.go:356] "Generic (PLEG): container finished" podID="f93ea7de-f642-4e45-a201-0a3b618b993e" containerID="e75dce289dfa3b059cef70024197d462f9b89868eb20e2dd5be3e3366a3812ee" exitCode=0
Aug 04 18:36:19.989070 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.989023    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng" event={"ID":"f93ea7de-f642-4e45-a201-0a3b618b993e","Type":"ContainerDied","Data":"e75dce289dfa3b059cef70024197d462f9b89868eb20e2dd5be3e3366a3812ee"}
Aug 04 18:36:19.989070 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.989036    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng"
Aug 04 18:36:19.989070 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.989068    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng" event={"ID":"f93ea7de-f642-4e45-a201-0a3b618b993e","Type":"ContainerDied","Data":"ae8818426badca6defec1f3b9d6ef30bca31bdb0d518f93538bbd9b9ae311467"}
Aug 04 18:36:19.990582 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.990548    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw" event={"ID":"713b82fb-b09f-4274-9afd-264dfed8d337","Type":"ContainerDied","Data":"dbd3c06d6e0b41bf6f144372e4ede9005a399df3bd9a49903878c5078f5a851c"}
Aug 04 18:36:19.990582 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.990573    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw"
Aug 04 18:36:19.998332 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.998316    2560 scope.go:117] "RemoveContainer" containerID="8b6cd79ae8d6a1c33f41d497853886c215a0d0e15fd64f73006a9e8666dce80e"
Aug 04 18:36:19.998674 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:36:19.998553    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b6cd79ae8d6a1c33f41d497853886c215a0d0e15fd64f73006a9e8666dce80e\": container with ID starting with 8b6cd79ae8d6a1c33f41d497853886c215a0d0e15fd64f73006a9e8666dce80e not found: ID does not exist" containerID="8b6cd79ae8d6a1c33f41d497853886c215a0d0e15fd64f73006a9e8666dce80e"
Aug 04 18:36:19.998674 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.998576    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b6cd79ae8d6a1c33f41d497853886c215a0d0e15fd64f73006a9e8666dce80e"} err="failed to get container status \"8b6cd79ae8d6a1c33f41d497853886c215a0d0e15fd64f73006a9e8666dce80e\": rpc error: code = NotFound desc = could not find container \"8b6cd79ae8d6a1c33f41d497853886c215a0d0e15fd64f73006a9e8666dce80e\": container with ID starting with 8b6cd79ae8d6a1c33f41d497853886c215a0d0e15fd64f73006a9e8666dce80e not found: ID does not exist"
Aug 04 18:36:19.998674 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:19.998593    2560 scope.go:117] "RemoveContainer" containerID="e75dce289dfa3b059cef70024197d462f9b89868eb20e2dd5be3e3366a3812ee"
Aug 04 18:36:20.013851 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:20.013831    2560 scope.go:117] "RemoveContainer" containerID="e75dce289dfa3b059cef70024197d462f9b89868eb20e2dd5be3e3366a3812ee"
Aug 04 18:36:20.014188 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:36:20.014163    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e75dce289dfa3b059cef70024197d462f9b89868eb20e2dd5be3e3366a3812ee\": container with ID starting with e75dce289dfa3b059cef70024197d462f9b89868eb20e2dd5be3e3366a3812ee not found: ID does not exist" containerID="e75dce289dfa3b059cef70024197d462f9b89868eb20e2dd5be3e3366a3812ee"
Aug 04 18:36:20.014284 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:20.014190    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e75dce289dfa3b059cef70024197d462f9b89868eb20e2dd5be3e3366a3812ee"} err="failed to get container status \"e75dce289dfa3b059cef70024197d462f9b89868eb20e2dd5be3e3366a3812ee\": rpc error: code = NotFound desc = could not find container \"e75dce289dfa3b059cef70024197d462f9b89868eb20e2dd5be3e3366a3812ee\": container with ID starting with e75dce289dfa3b059cef70024197d462f9b89868eb20e2dd5be3e3366a3812ee not found: ID does not exist"
Aug 04 18:36:20.014284 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:20.014215    2560 scope.go:117] "RemoveContainer" containerID="e5c5edf2cbd97bb461f712da8b0a93ac9ea6b8dac97c4067382c91f2ba151a1a"
Aug 04 18:36:20.017227 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:20.017211    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x"]
Aug 04 18:36:20.023573 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:20.023554    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-7f7c0a-ffbf6bf68-w7l5x"]
Aug 04 18:36:20.037277 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:20.037258    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng"]
Aug 04 18:36:20.039964 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:20.039940    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-7f7c0a-5b85d8868d-bzgng"]
Aug 04 18:36:20.051692 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:20.051673    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw"]
Aug 04 18:36:20.057538 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:20.057519    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7f7c0a-7d97fc5494-dznjw"]
Aug 04 18:36:20.354142 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:20.354107    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="713b82fb-b09f-4274-9afd-264dfed8d337" path="/var/lib/kubelet/pods/713b82fb-b09f-4274-9afd-264dfed8d337/volumes"
Aug 04 18:36:20.354469 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:20.354456    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5076398-949a-4bee-abe1-6799071922aa" path="/var/lib/kubelet/pods/d5076398-949a-4bee-abe1-6799071922aa/volumes"
Aug 04 18:36:20.354769 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:20.354757    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f93ea7de-f642-4e45-a201-0a3b618b993e" path="/var/lib/kubelet/pods/f93ea7de-f642-4e45-a201-0a3b618b993e/volumes"
Aug 04 18:36:31.382732 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.382701    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw"]
Aug 04 18:36:31.383163 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.383149    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="023952d7-d30a-4894-b6dd-33f269a5aca3" containerName="payload-processing"
Aug 04 18:36:31.383208 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.383165    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="023952d7-d30a-4894-b6dd-33f269a5aca3" containerName="payload-processing"
Aug 04 18:36:31.383208 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.383180    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5076398-949a-4bee-abe1-6799071922aa" containerName="payload-processing"
Aug 04 18:36:31.383208 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.383187    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5076398-949a-4bee-abe1-6799071922aa" containerName="payload-processing"
Aug 04 18:36:31.383208 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.383204    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576" containerName="payload-pre-processing"
Aug 04 18:36:31.383330 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.383214    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576" containerName="payload-pre-processing"
Aug 04 18:36:31.383330 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.383223    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f38973a9-9b98-492d-bbd7-5630dd7c92c5" containerName="maas-api"
Aug 04 18:36:31.383330 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.383229    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f38973a9-9b98-492d-bbd7-5630dd7c92c5" containerName="maas-api"
Aug 04 18:36:31.383330 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.383237    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f93ea7de-f642-4e45-a201-0a3b618b993e" containerName="payload-pre-processing"
Aug 04 18:36:31.383330 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.383243    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f93ea7de-f642-4e45-a201-0a3b618b993e" containerName="payload-pre-processing"
Aug 04 18:36:31.383330 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.383250    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="713b82fb-b09f-4274-9afd-264dfed8d337" containerName="maas-api"
Aug 04 18:36:31.383330 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.383255    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="713b82fb-b09f-4274-9afd-264dfed8d337" containerName="maas-api"
Aug 04 18:36:31.383330 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.383315    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f38973a9-9b98-492d-bbd7-5630dd7c92c5" containerName="maas-api"
Aug 04 18:36:31.383330 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.383324    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5076398-949a-4bee-abe1-6799071922aa" containerName="payload-processing"
Aug 04 18:36:31.383330 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.383329    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="023952d7-d30a-4894-b6dd-33f269a5aca3" containerName="payload-processing"
Aug 04 18:36:31.383611 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.383337    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f93ea7de-f642-4e45-a201-0a3b618b993e" containerName="payload-pre-processing"
Aug 04 18:36:31.383611 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.383343    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="713b82fb-b09f-4274-9afd-264dfed8d337" containerName="maas-api"
Aug 04 18:36:31.383611 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.383348    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1c1f0c6-0de9-454f-aadb-d1ba8f0eb576" containerName="payload-pre-processing"
Aug 04 18:36:31.387870 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.387844    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw"
Aug 04 18:36:31.390497 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.390474    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-a08d36-serving-cert\""
Aug 04 18:36:31.400054 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.400029    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw"]
Aug 04 18:36:31.512683 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.512650    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8ba4898-2212-4585-a7f4-58173a9a5fee-maas-api-tls\") pod \"maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw\" (UID: \"b8ba4898-2212-4585-a7f4-58173a9a5fee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw"
Aug 04 18:36:31.512860 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.512699    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xfwb\" (UniqueName: \"kubernetes.io/projected/b8ba4898-2212-4585-a7f4-58173a9a5fee-kube-api-access-4xfwb\") pod \"maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw\" (UID: \"b8ba4898-2212-4585-a7f4-58173a9a5fee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw"
Aug 04 18:36:31.512860 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.512736    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8ba4898-2212-4585-a7f4-58173a9a5fee-openshift-service-ca\") pod \"maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw\" (UID: \"b8ba4898-2212-4585-a7f4-58173a9a5fee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw"
Aug 04 18:36:31.614187 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.614151    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8ba4898-2212-4585-a7f4-58173a9a5fee-maas-api-tls\") pod \"maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw\" (UID: \"b8ba4898-2212-4585-a7f4-58173a9a5fee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw"
Aug 04 18:36:31.614378 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.614217    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xfwb\" (UniqueName: \"kubernetes.io/projected/b8ba4898-2212-4585-a7f4-58173a9a5fee-kube-api-access-4xfwb\") pod \"maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw\" (UID: \"b8ba4898-2212-4585-a7f4-58173a9a5fee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw"
Aug 04 18:36:31.614378 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.614266    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8ba4898-2212-4585-a7f4-58173a9a5fee-openshift-service-ca\") pod \"maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw\" (UID: \"b8ba4898-2212-4585-a7f4-58173a9a5fee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw"
Aug 04 18:36:31.614378 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:36:31.614329    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-serving-cert: secret "maas-api-e2e-api-a-a08d36-serving-cert" not found
Aug 04 18:36:31.614549 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:36:31.614417    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8ba4898-2212-4585-a7f4-58173a9a5fee-maas-api-tls podName:b8ba4898-2212-4585-a7f4-58173a9a5fee nodeName:}" failed. No retries permitted until 2026-08-04 18:36:32.114395775 +0000 UTC m=+2674.297555945 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b8ba4898-2212-4585-a7f4-58173a9a5fee-maas-api-tls") pod "maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw" (UID: "b8ba4898-2212-4585-a7f4-58173a9a5fee") : secret "maas-api-e2e-api-a-a08d36-serving-cert" not found
Aug 04 18:36:31.614954 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.614931    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8ba4898-2212-4585-a7f4-58173a9a5fee-openshift-service-ca\") pod \"maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw\" (UID: \"b8ba4898-2212-4585-a7f4-58173a9a5fee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw"
Aug 04 18:36:31.624671 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:31.624646    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xfwb\" (UniqueName: \"kubernetes.io/projected/b8ba4898-2212-4585-a7f4-58173a9a5fee-kube-api-access-4xfwb\") pod \"maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw\" (UID: \"b8ba4898-2212-4585-a7f4-58173a9a5fee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw"
Aug 04 18:36:32.119322 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.119292    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8ba4898-2212-4585-a7f4-58173a9a5fee-maas-api-tls\") pod \"maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw\" (UID: \"b8ba4898-2212-4585-a7f4-58173a9a5fee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw"
Aug 04 18:36:32.121975 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.121948    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8ba4898-2212-4585-a7f4-58173a9a5fee-maas-api-tls\") pod \"maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw\" (UID: \"b8ba4898-2212-4585-a7f4-58173a9a5fee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw"
Aug 04 18:36:32.234787 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.234760    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc"]
Aug 04 18:36:32.238399 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.238385    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc"
Aug 04 18:36:32.241238 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.241219    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-a08d36-dockercfg-wx8f2\""
Aug 04 18:36:32.241764 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.241747    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-a08d36\""
Aug 04 18:36:32.249900 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.249878    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc"]
Aug 04 18:36:32.299538 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.299510    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw"
Aug 04 18:36:32.317838 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.317814    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq"]
Aug 04 18:36:32.321328 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.321304    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fff060f-820e-4f9f-83af-da10bd526d4f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc\" (UID: \"3fff060f-820e-4f9f-83af-da10bd526d4f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc"
Aug 04 18:36:32.321436 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.321354    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fft67\" (UniqueName: \"kubernetes.io/projected/3fff060f-820e-4f9f-83af-da10bd526d4f-kube-api-access-fft67\") pod \"payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc\" (UID: \"3fff060f-820e-4f9f-83af-da10bd526d4f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc"
Aug 04 18:36:32.322060 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.322044    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq"
Aug 04 18:36:32.336095 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.335852    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq"]
Aug 04 18:36:32.422600 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.422563    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tf6v\" (UniqueName: \"kubernetes.io/projected/0f00cebc-ae97-4766-80b8-a38d2c281f53-kube-api-access-4tf6v\") pod \"payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq\" (UID: \"0f00cebc-ae97-4766-80b8-a38d2c281f53\") " pod="openshift-ingress/payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq"
Aug 04 18:36:32.423040 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.422604    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fff060f-820e-4f9f-83af-da10bd526d4f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc\" (UID: \"3fff060f-820e-4f9f-83af-da10bd526d4f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc"
Aug 04 18:36:32.423040 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.422637    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fft67\" (UniqueName: \"kubernetes.io/projected/3fff060f-820e-4f9f-83af-da10bd526d4f-kube-api-access-fft67\") pod \"payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc\" (UID: \"3fff060f-820e-4f9f-83af-da10bd526d4f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc"
Aug 04 18:36:32.423040 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.422661    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f00cebc-ae97-4766-80b8-a38d2c281f53-plugins-config-volume\") pod \"payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq\" (UID: \"0f00cebc-ae97-4766-80b8-a38d2c281f53\") " pod="openshift-ingress/payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq"
Aug 04 18:36:32.423415 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.423393    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fff060f-820e-4f9f-83af-da10bd526d4f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc\" (UID: \"3fff060f-820e-4f9f-83af-da10bd526d4f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc"
Aug 04 18:36:32.434330 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.434301    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fft67\" (UniqueName: \"kubernetes.io/projected/3fff060f-820e-4f9f-83af-da10bd526d4f-kube-api-access-fft67\") pod \"payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc\" (UID: \"3fff060f-820e-4f9f-83af-da10bd526d4f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc"
Aug 04 18:36:32.451190 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.451171    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw"]
Aug 04 18:36:32.453691 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:36:32.453662    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8ba4898_2212_4585_a7f4_58173a9a5fee.slice/crio-abc530d467ffb8e03a80cbdd162aa230204502c9f797b1aefb98b6228ca032d3 WatchSource:0}: Error finding container abc530d467ffb8e03a80cbdd162aa230204502c9f797b1aefb98b6228ca032d3: Status 404 returned error can't find the container with id abc530d467ffb8e03a80cbdd162aa230204502c9f797b1aefb98b6228ca032d3
Aug 04 18:36:32.455078 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.455061    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 18:36:32.523114 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.523090    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f00cebc-ae97-4766-80b8-a38d2c281f53-plugins-config-volume\") pod \"payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq\" (UID: \"0f00cebc-ae97-4766-80b8-a38d2c281f53\") " pod="openshift-ingress/payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq"
Aug 04 18:36:32.523232 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.523168    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tf6v\" (UniqueName: \"kubernetes.io/projected/0f00cebc-ae97-4766-80b8-a38d2c281f53-kube-api-access-4tf6v\") pod \"payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq\" (UID: \"0f00cebc-ae97-4766-80b8-a38d2c281f53\") " pod="openshift-ingress/payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq"
Aug 04 18:36:32.523782 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.523758    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f00cebc-ae97-4766-80b8-a38d2c281f53-plugins-config-volume\") pod \"payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq\" (UID: \"0f00cebc-ae97-4766-80b8-a38d2c281f53\") " pod="openshift-ingress/payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq"
Aug 04 18:36:32.537317 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.537300    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tf6v\" (UniqueName: \"kubernetes.io/projected/0f00cebc-ae97-4766-80b8-a38d2c281f53-kube-api-access-4tf6v\") pod \"payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq\" (UID: \"0f00cebc-ae97-4766-80b8-a38d2c281f53\") " pod="openshift-ingress/payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq"
Aug 04 18:36:32.551117 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.551101    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc"
Aug 04 18:36:32.633798 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.633769    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq"
Aug 04 18:36:32.699919 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.699890    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc"]
Aug 04 18:36:32.703578 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:36:32.703529    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fff060f_820e_4f9f_83af_da10bd526d4f.slice/crio-685e13e080734af90647a6efbcd2387ec674dd328cbb0ebdc577d05697229bf7 WatchSource:0}: Error finding container 685e13e080734af90647a6efbcd2387ec674dd328cbb0ebdc577d05697229bf7: Status 404 returned error can't find the container with id 685e13e080734af90647a6efbcd2387ec674dd328cbb0ebdc577d05697229bf7
Aug 04 18:36:32.795485 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:32.795458    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq"]
Aug 04 18:36:32.796921 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:36:32.796895    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f00cebc_ae97_4766_80b8_a38d2c281f53.slice/crio-4efa94458df7f4c25e614619278edf432a8f1923e5fd33780c44b5b7322e046c WatchSource:0}: Error finding container 4efa94458df7f4c25e614619278edf432a8f1923e5fd33780c44b5b7322e046c: Status 404 returned error can't find the container with id 4efa94458df7f4c25e614619278edf432a8f1923e5fd33780c44b5b7322e046c
Aug 04 18:36:33.059430 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:33.059391    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq" event={"ID":"0f00cebc-ae97-4766-80b8-a38d2c281f53","Type":"ContainerStarted","Data":"ca8533757001887ff6b68dd49fc5eef2019a49e7d110ca9ab3cbc2d59072b550"}
Aug 04 18:36:33.059626 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:33.059463    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq"
Aug 04 18:36:33.059626 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:33.059487    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq" event={"ID":"0f00cebc-ae97-4766-80b8-a38d2c281f53","Type":"ContainerStarted","Data":"4efa94458df7f4c25e614619278edf432a8f1923e5fd33780c44b5b7322e046c"}
Aug 04 18:36:33.061089 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:33.061057    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc" event={"ID":"3fff060f-820e-4f9f-83af-da10bd526d4f","Type":"ContainerStarted","Data":"19c7564f692371e3fccedcc23f425f4db0d4c24fa7da29779c71c974d3346a38"}
Aug 04 18:36:33.061220 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:33.061106    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc" event={"ID":"3fff060f-820e-4f9f-83af-da10bd526d4f","Type":"ContainerStarted","Data":"685e13e080734af90647a6efbcd2387ec674dd328cbb0ebdc577d05697229bf7"}
Aug 04 18:36:33.061220 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:33.061127    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc"
Aug 04 18:36:33.062665 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:33.062643    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw" event={"ID":"b8ba4898-2212-4585-a7f4-58173a9a5fee","Type":"ContainerStarted","Data":"6870c0e07e177df6164ba6710f7e1cd8d6d46813467906b1baf19e7d2eb15f1b"}
Aug 04 18:36:33.062763 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:33.062679    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw" event={"ID":"b8ba4898-2212-4585-a7f4-58173a9a5fee","Type":"ContainerStarted","Data":"abc530d467ffb8e03a80cbdd162aa230204502c9f797b1aefb98b6228ca032d3"}
Aug 04 18:36:33.062763 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:33.062731    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw"
Aug 04 18:36:33.089491 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:33.089445    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq" podStartSLOduration=1.089432291 podStartE2EDuration="1.089432291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:36:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:36:33.087982478 +0000 UTC m=+2675.271142655" watchObservedRunningTime="2026-08-04 18:36:33.089432291 +0000 UTC m=+2675.272592479"
Aug 04 18:36:33.115887 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:33.115843    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw" podStartSLOduration=1.729449682 podStartE2EDuration="2.11583109s" totalImagesPullingTime="386.381408ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:36:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:36:33.115633861 +0000 UTC m=+2675.298794036" watchObservedRunningTime="2026-08-04 18:36:33.11583109 +0000 UTC m=+2675.298991265"
Aug 04 18:36:33.160624 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:33.160573    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc" podStartSLOduration=1.16055855 podStartE2EDuration="1.16055855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:36:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:36:33.158720195 +0000 UTC m=+2675.341880371" watchObservedRunningTime="2026-08-04 18:36:33.16055855 +0000 UTC m=+2675.343718726"
Aug 04 18:36:39.073037 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:39.072983    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw"
Aug 04 18:36:44.069369 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:44.069333    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq"
Aug 04 18:36:44.069808 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:44.069419    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc"
Aug 04 18:36:58.514818 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:58.514625    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/2.log"
Aug 04 18:36:58.520277 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:58.520256    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 18:36:58.520921 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:58.520905    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/2.log"
Aug 04 18:36:58.526422 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:58.526404    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 18:36:59.785687 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:59.785642    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-6f74856975-5l6vm"]
Aug 04 18:36:59.791599 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:59.791576    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-6f74856975-5l6vm"
Aug 04 18:36:59.794309 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:59.794285    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-a77bad-serving-cert\""
Aug 04 18:36:59.800184 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:59.799576    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-6f74856975-5l6vm"]
Aug 04 18:36:59.847121 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:59.847096    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcdtm\" (UniqueName: \"kubernetes.io/projected/3fff781b-238a-4425-9952-198c8fdbecd8-kube-api-access-pcdtm\") pod \"maas-api-e2e-api-b-a77bad-6f74856975-5l6vm\" (UID: \"3fff781b-238a-4425-9952-198c8fdbecd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-6f74856975-5l6vm"
Aug 04 18:36:59.847294 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:59.847158    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fff781b-238a-4425-9952-198c8fdbecd8-openshift-service-ca\") pod \"maas-api-e2e-api-b-a77bad-6f74856975-5l6vm\" (UID: \"3fff781b-238a-4425-9952-198c8fdbecd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-6f74856975-5l6vm"
Aug 04 18:36:59.847294 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:59.847255    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fff781b-238a-4425-9952-198c8fdbecd8-maas-api-tls\") pod \"maas-api-e2e-api-b-a77bad-6f74856975-5l6vm\" (UID: \"3fff781b-238a-4425-9952-198c8fdbecd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-6f74856975-5l6vm"
Aug 04 18:36:59.947672 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:59.947645    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fff781b-238a-4425-9952-198c8fdbecd8-maas-api-tls\") pod \"maas-api-e2e-api-b-a77bad-6f74856975-5l6vm\" (UID: \"3fff781b-238a-4425-9952-198c8fdbecd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-6f74856975-5l6vm"
Aug 04 18:36:59.947817 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:59.947691    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcdtm\" (UniqueName: \"kubernetes.io/projected/3fff781b-238a-4425-9952-198c8fdbecd8-kube-api-access-pcdtm\") pod \"maas-api-e2e-api-b-a77bad-6f74856975-5l6vm\" (UID: \"3fff781b-238a-4425-9952-198c8fdbecd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-6f74856975-5l6vm"
Aug 04 18:36:59.947817 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:36:59.947783    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-serving-cert: secret "maas-api-e2e-api-b-a77bad-serving-cert" not found
Aug 04 18:36:59.947931 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:59.947815    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fff781b-238a-4425-9952-198c8fdbecd8-openshift-service-ca\") pod \"maas-api-e2e-api-b-a77bad-6f74856975-5l6vm\" (UID: \"3fff781b-238a-4425-9952-198c8fdbecd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-6f74856975-5l6vm"
Aug 04 18:36:59.947931 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:36:59.947842    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fff781b-238a-4425-9952-198c8fdbecd8-maas-api-tls podName:3fff781b-238a-4425-9952-198c8fdbecd8 nodeName:}" failed. No retries permitted until 2026-08-04 18:37:00.447826125 +0000 UTC m=+2702.630986280 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3fff781b-238a-4425-9952-198c8fdbecd8-maas-api-tls") pod "maas-api-e2e-api-b-a77bad-6f74856975-5l6vm" (UID: "3fff781b-238a-4425-9952-198c8fdbecd8") : secret "maas-api-e2e-api-b-a77bad-serving-cert" not found
Aug 04 18:36:59.948379 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:59.948361    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fff781b-238a-4425-9952-198c8fdbecd8-openshift-service-ca\") pod \"maas-api-e2e-api-b-a77bad-6f74856975-5l6vm\" (UID: \"3fff781b-238a-4425-9952-198c8fdbecd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-6f74856975-5l6vm"
Aug 04 18:36:59.957266 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:36:59.957246    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcdtm\" (UniqueName: \"kubernetes.io/projected/3fff781b-238a-4425-9952-198c8fdbecd8-kube-api-access-pcdtm\") pod \"maas-api-e2e-api-b-a77bad-6f74856975-5l6vm\" (UID: \"3fff781b-238a-4425-9952-198c8fdbecd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-6f74856975-5l6vm"
Aug 04 18:37:00.450433 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:00.450400    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fff781b-238a-4425-9952-198c8fdbecd8-maas-api-tls\") pod \"maas-api-e2e-api-b-a77bad-6f74856975-5l6vm\" (UID: \"3fff781b-238a-4425-9952-198c8fdbecd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-6f74856975-5l6vm"
Aug 04 18:37:00.452958 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:00.452926    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fff781b-238a-4425-9952-198c8fdbecd8-maas-api-tls\") pod \"maas-api-e2e-api-b-a77bad-6f74856975-5l6vm\" (UID: \"3fff781b-238a-4425-9952-198c8fdbecd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-6f74856975-5l6vm"
Aug 04 18:37:00.572577 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:00.572549    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj"]
Aug 04 18:37:00.576337 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:00.576314    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj"
Aug 04 18:37:00.579493 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:00.579469    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-a77bad\""
Aug 04 18:37:00.589754 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:00.589731    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj"]
Aug 04 18:37:00.591497 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:00.591476    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv"]
Aug 04 18:37:00.595003 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:00.594968    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv"
Aug 04 18:37:00.597636 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:00.597618    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-a77bad-dockercfg-wlrlj\""
Aug 04 18:37:00.608415 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:00.608392    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv"]
Aug 04 18:37:00.652313 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:00.652282    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj\" (UID: \"6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj"
Aug 04 18:37:00.652431 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:00.652320    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fjvj\" (UniqueName: \"kubernetes.io/projected/6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f-kube-api-access-5fjvj\") pod \"payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj\" (UID: \"6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj"
Aug 04 18:37:00.703940 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:00.703888    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-6f74856975-5l6vm"
Aug 04 18:37:00.753208 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:00.753176    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlfkq\" (UniqueName: \"kubernetes.io/projected/5befe5b4-51c0-4117-9d74-cb0e758b5d33-kube-api-access-vlfkq\") pod \"payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv\" (UID: \"5befe5b4-51c0-4117-9d74-cb0e758b5d33\") " pod="openshift-ingress/payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv"
Aug 04 18:37:00.753208 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:00.753215    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj\" (UID: \"6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj"
Aug 04 18:37:00.753457 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:00.753242    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fjvj\" (UniqueName: \"kubernetes.io/projected/6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f-kube-api-access-5fjvj\") pod \"payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj\" (UID: \"6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj"
Aug 04 18:37:00.753457 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:00.753366    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5befe5b4-51c0-4117-9d74-cb0e758b5d33-plugins-config-volume\") pod \"payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv\" (UID: \"5befe5b4-51c0-4117-9d74-cb0e758b5d33\") " pod="openshift-ingress/payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv"
Aug 04 18:37:00.753946 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:00.753923    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj\" (UID: \"6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj"
Aug 04 18:37:00.767559 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:00.767536    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fjvj\" (UniqueName: \"kubernetes.io/projected/6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f-kube-api-access-5fjvj\") pod \"payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj\" (UID: \"6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj"
Aug 04 18:37:00.851764 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:00.851734    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-6f74856975-5l6vm"]
Aug 04 18:37:00.854840 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:00.854814    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlfkq\" (UniqueName: \"kubernetes.io/projected/5befe5b4-51c0-4117-9d74-cb0e758b5d33-kube-api-access-vlfkq\") pod \"payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv\" (UID: \"5befe5b4-51c0-4117-9d74-cb0e758b5d33\") " pod="openshift-ingress/payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv"
Aug 04 18:37:00.855057 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:00.854860    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5befe5b4-51c0-4117-9d74-cb0e758b5d33-plugins-config-volume\") pod \"payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv\" (UID: \"5befe5b4-51c0-4117-9d74-cb0e758b5d33\") " pod="openshift-ingress/payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv"
Aug 04 18:37:00.855057 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:37:00.855019    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fff781b_238a_4425_9952_198c8fdbecd8.slice/crio-be6d649ce0fa56d2716d83784c076267bce86736b2e77cb768aa279c62604868 WatchSource:0}: Error finding container be6d649ce0fa56d2716d83784c076267bce86736b2e77cb768aa279c62604868: Status 404 returned error can't find the container with id be6d649ce0fa56d2716d83784c076267bce86736b2e77cb768aa279c62604868
Aug 04 18:37:00.855535 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:00.855512    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5befe5b4-51c0-4117-9d74-cb0e758b5d33-plugins-config-volume\") pod \"payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv\" (UID: \"5befe5b4-51c0-4117-9d74-cb0e758b5d33\") " pod="openshift-ingress/payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv"
Aug 04 18:37:00.865421 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:00.865402    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlfkq\" (UniqueName: \"kubernetes.io/projected/5befe5b4-51c0-4117-9d74-cb0e758b5d33-kube-api-access-vlfkq\") pod \"payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv\" (UID: \"5befe5b4-51c0-4117-9d74-cb0e758b5d33\") " pod="openshift-ingress/payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv"
Aug 04 18:37:00.889400 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:00.889375    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj"
Aug 04 18:37:00.906171 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:00.906153    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv"
Aug 04 18:37:01.041598 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:01.041419    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj"]
Aug 04 18:37:01.044202 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:37:01.044162    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f9e4bb1_04fd_4d0a_b9e4_61fec8ef342f.slice/crio-97f8481e3e263d2f9f6084f265f700e36a7bfbcd9f38509c18e26409eb16492b WatchSource:0}: Error finding container 97f8481e3e263d2f9f6084f265f700e36a7bfbcd9f38509c18e26409eb16492b: Status 404 returned error can't find the container with id 97f8481e3e263d2f9f6084f265f700e36a7bfbcd9f38509c18e26409eb16492b
Aug 04 18:37:01.062668 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:01.062646    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv"]
Aug 04 18:37:01.065135 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:37:01.065108    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5befe5b4_51c0_4117_9d74_cb0e758b5d33.slice/crio-119408774b89de2fe25ee44b691fff8763b6ccc3fad1fd5d1ad81e0042cc4d91 WatchSource:0}: Error finding container 119408774b89de2fe25ee44b691fff8763b6ccc3fad1fd5d1ad81e0042cc4d91: Status 404 returned error can't find the container with id 119408774b89de2fe25ee44b691fff8763b6ccc3fad1fd5d1ad81e0042cc4d91
Aug 04 18:37:01.204081 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:01.204048    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv" event={"ID":"5befe5b4-51c0-4117-9d74-cb0e758b5d33","Type":"ContainerStarted","Data":"114da32b13d24669690c143994bacc7e44a1d280b2f6cbc553501593b10f075a"}
Aug 04 18:37:01.204230 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:01.204098    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv" event={"ID":"5befe5b4-51c0-4117-9d74-cb0e758b5d33","Type":"ContainerStarted","Data":"119408774b89de2fe25ee44b691fff8763b6ccc3fad1fd5d1ad81e0042cc4d91"}
Aug 04 18:37:01.204230 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:01.204134    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv"
Aug 04 18:37:01.205421 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:01.205397    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-6f74856975-5l6vm" event={"ID":"3fff781b-238a-4425-9952-198c8fdbecd8","Type":"ContainerStarted","Data":"be6d649ce0fa56d2716d83784c076267bce86736b2e77cb768aa279c62604868"}
Aug 04 18:37:01.206974 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:01.206955    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj" event={"ID":"6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f","Type":"ContainerStarted","Data":"c883dd56a14c0aa1a82ce1153dccb7811b89f39793b58b9dd810d479044e93de"}
Aug 04 18:37:01.207065 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:01.207020    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj" event={"ID":"6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f","Type":"ContainerStarted","Data":"97f8481e3e263d2f9f6084f265f700e36a7bfbcd9f38509c18e26409eb16492b"}
Aug 04 18:37:01.207065 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:01.207050    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj"
Aug 04 18:37:01.230797 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:01.230719    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv" podStartSLOduration=1.230707253 podStartE2EDuration="1.230707253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:37:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:37:01.22719549 +0000 UTC m=+2703.410355667" watchObservedRunningTime="2026-08-04 18:37:01.230707253 +0000 UTC m=+2703.413867419"
Aug 04 18:37:01.249196 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:01.249145    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj" podStartSLOduration=1.249128384 podStartE2EDuration="1.249128384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:37:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:37:01.247266543 +0000 UTC m=+2703.430426718" watchObservedRunningTime="2026-08-04 18:37:01.249128384 +0000 UTC m=+2703.432288565"
Aug 04 18:37:02.214138 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:02.214103    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-6f74856975-5l6vm" event={"ID":"3fff781b-238a-4425-9952-198c8fdbecd8","Type":"ContainerStarted","Data":"4e587da3f6892320d9b676a77626449628b9ee0973df03984deb47c9ad0c73fb"}
Aug 04 18:37:02.214561 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:02.214415    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-6f74856975-5l6vm"
Aug 04 18:37:02.236487 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:02.236429    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-6f74856975-5l6vm" podStartSLOduration=2.676685484 podStartE2EDuration="3.236411214s" totalImagesPullingTime="559.72573ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:36:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:37:02.235392155 +0000 UTC m=+2704.418552333" watchObservedRunningTime="2026-08-04 18:37:02.236411214 +0000 UTC m=+2704.419571393"
Aug 04 18:37:08.230634 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:08.230608    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-6f74856975-5l6vm"
Aug 04 18:37:12.215967 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:12.215935    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv"
Aug 04 18:37:12.216397 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:12.216045    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj"
Aug 04 18:37:47.292541 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.292501    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw"]
Aug 04 18:37:47.292977 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.292714    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw" podUID="b8ba4898-2212-4585-a7f4-58173a9a5fee" containerName="maas-api" containerID="cri-o://6870c0e07e177df6164ba6710f7e1cd8d6d46813467906b1baf19e7d2eb15f1b" gracePeriod=30
Aug 04 18:37:47.419779 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.419750    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq"]
Aug 04 18:37:47.420040 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.419983    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq" podUID="0f00cebc-ae97-4766-80b8-a38d2c281f53" containerName="payload-processing" containerID="cri-o://ca8533757001887ff6b68dd49fc5eef2019a49e7d110ca9ab3cbc2d59072b550" gracePeriod=30
Aug 04 18:37:47.433828 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.433804    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc"]
Aug 04 18:37:47.452899 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.452869    2560 generic.go:356] "Generic (PLEG): container finished" podID="b8ba4898-2212-4585-a7f4-58173a9a5fee" containerID="6870c0e07e177df6164ba6710f7e1cd8d6d46813467906b1baf19e7d2eb15f1b" exitCode=0
Aug 04 18:37:47.453070 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.452953    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw" event={"ID":"b8ba4898-2212-4585-a7f4-58173a9a5fee","Type":"ContainerDied","Data":"6870c0e07e177df6164ba6710f7e1cd8d6d46813467906b1baf19e7d2eb15f1b"}
Aug 04 18:37:47.453266 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.453240    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc" podUID="3fff060f-820e-4f9f-83af-da10bd526d4f" containerName="payload-pre-processing" containerID="cri-o://19c7564f692371e3fccedcc23f425f4db0d4c24fa7da29779c71c974d3346a38" gracePeriod=30
Aug 04 18:37:47.574357 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.574336    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw"
Aug 04 18:37:47.689092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.689065    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq"
Aug 04 18:37:47.729446 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.729419    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc"
Aug 04 18:37:47.744799 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.744765    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xfwb\" (UniqueName: \"kubernetes.io/projected/b8ba4898-2212-4585-a7f4-58173a9a5fee-kube-api-access-4xfwb\") pod \"b8ba4898-2212-4585-a7f4-58173a9a5fee\" (UID: \"b8ba4898-2212-4585-a7f4-58173a9a5fee\") "
Aug 04 18:37:47.744944 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.744842    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8ba4898-2212-4585-a7f4-58173a9a5fee-maas-api-tls\") pod \"b8ba4898-2212-4585-a7f4-58173a9a5fee\" (UID: \"b8ba4898-2212-4585-a7f4-58173a9a5fee\") "
Aug 04 18:37:47.745044 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.744946    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8ba4898-2212-4585-a7f4-58173a9a5fee-openshift-service-ca\") pod \"b8ba4898-2212-4585-a7f4-58173a9a5fee\" (UID: \"b8ba4898-2212-4585-a7f4-58173a9a5fee\") "
Aug 04 18:37:47.745304 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.745275    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8ba4898-2212-4585-a7f4-58173a9a5fee-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b8ba4898-2212-4585-a7f4-58173a9a5fee" (UID: "b8ba4898-2212-4585-a7f4-58173a9a5fee"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:37:47.746972 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.746949    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8ba4898-2212-4585-a7f4-58173a9a5fee-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b8ba4898-2212-4585-a7f4-58173a9a5fee" (UID: "b8ba4898-2212-4585-a7f4-58173a9a5fee"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:37:47.747117 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.747008    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8ba4898-2212-4585-a7f4-58173a9a5fee-kube-api-access-4xfwb" (OuterVolumeSpecName: "kube-api-access-4xfwb") pod "b8ba4898-2212-4585-a7f4-58173a9a5fee" (UID: "b8ba4898-2212-4585-a7f4-58173a9a5fee"). InnerVolumeSpecName "kube-api-access-4xfwb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:37:47.846333 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.846257    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fft67\" (UniqueName: \"kubernetes.io/projected/3fff060f-820e-4f9f-83af-da10bd526d4f-kube-api-access-fft67\") pod \"3fff060f-820e-4f9f-83af-da10bd526d4f\" (UID: \"3fff060f-820e-4f9f-83af-da10bd526d4f\") "
Aug 04 18:37:47.846500 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.846342    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fff060f-820e-4f9f-83af-da10bd526d4f-plugins-config-volume\") pod \"3fff060f-820e-4f9f-83af-da10bd526d4f\" (UID: \"3fff060f-820e-4f9f-83af-da10bd526d4f\") "
Aug 04 18:37:47.846500 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.846445    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tf6v\" (UniqueName: \"kubernetes.io/projected/0f00cebc-ae97-4766-80b8-a38d2c281f53-kube-api-access-4tf6v\") pod \"0f00cebc-ae97-4766-80b8-a38d2c281f53\" (UID: \"0f00cebc-ae97-4766-80b8-a38d2c281f53\") "
Aug 04 18:37:47.846623 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.846548    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f00cebc-ae97-4766-80b8-a38d2c281f53-plugins-config-volume\") pod \"0f00cebc-ae97-4766-80b8-a38d2c281f53\" (UID: \"0f00cebc-ae97-4766-80b8-a38d2c281f53\") "
Aug 04 18:37:47.846674 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.846646    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fff060f-820e-4f9f-83af-da10bd526d4f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3fff060f-820e-4f9f-83af-da10bd526d4f" (UID: "3fff060f-820e-4f9f-83af-da10bd526d4f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:37:47.846923 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.846898    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8ba4898-2212-4585-a7f4-58173a9a5fee-maas-api-tls\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:37:47.846923 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.846906    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f00cebc-ae97-4766-80b8-a38d2c281f53-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0f00cebc-ae97-4766-80b8-a38d2c281f53" (UID: "0f00cebc-ae97-4766-80b8-a38d2c281f53"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:37:47.847277 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.846922    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fff060f-820e-4f9f-83af-da10bd526d4f-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:37:47.847277 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.846963    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8ba4898-2212-4585-a7f4-58173a9a5fee-openshift-service-ca\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:37:47.847277 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.846976    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xfwb\" (UniqueName: \"kubernetes.io/projected/b8ba4898-2212-4585-a7f4-58173a9a5fee-kube-api-access-4xfwb\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:37:47.848438 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.848417    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fff060f-820e-4f9f-83af-da10bd526d4f-kube-api-access-fft67" (OuterVolumeSpecName: "kube-api-access-fft67") pod "3fff060f-820e-4f9f-83af-da10bd526d4f" (UID: "3fff060f-820e-4f9f-83af-da10bd526d4f"). InnerVolumeSpecName "kube-api-access-fft67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:37:47.848550 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.848536    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f00cebc-ae97-4766-80b8-a38d2c281f53-kube-api-access-4tf6v" (OuterVolumeSpecName: "kube-api-access-4tf6v") pod "0f00cebc-ae97-4766-80b8-a38d2c281f53" (UID: "0f00cebc-ae97-4766-80b8-a38d2c281f53"). InnerVolumeSpecName "kube-api-access-4tf6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:37:47.947409 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.947379    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f00cebc-ae97-4766-80b8-a38d2c281f53-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:37:47.947409 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.947409    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fft67\" (UniqueName: \"kubernetes.io/projected/3fff060f-820e-4f9f-83af-da10bd526d4f-kube-api-access-fft67\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:37:47.947588 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:47.947420    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tf6v\" (UniqueName: \"kubernetes.io/projected/0f00cebc-ae97-4766-80b8-a38d2c281f53-kube-api-access-4tf6v\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:37:48.465534 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:48.465503    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw"
Aug 04 18:37:48.465976 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:48.465534    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw" event={"ID":"b8ba4898-2212-4585-a7f4-58173a9a5fee","Type":"ContainerDied","Data":"abc530d467ffb8e03a80cbdd162aa230204502c9f797b1aefb98b6228ca032d3"}
Aug 04 18:37:48.465976 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:48.465622    2560 scope.go:117] "RemoveContainer" containerID="6870c0e07e177df6164ba6710f7e1cd8d6d46813467906b1baf19e7d2eb15f1b"
Aug 04 18:37:48.466947 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:48.466923    2560 generic.go:356] "Generic (PLEG): container finished" podID="0f00cebc-ae97-4766-80b8-a38d2c281f53" containerID="ca8533757001887ff6b68dd49fc5eef2019a49e7d110ca9ab3cbc2d59072b550" exitCode=0
Aug 04 18:37:48.467055 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:48.467005    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq"
Aug 04 18:37:48.467055 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:48.467016    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq" event={"ID":"0f00cebc-ae97-4766-80b8-a38d2c281f53","Type":"ContainerDied","Data":"ca8533757001887ff6b68dd49fc5eef2019a49e7d110ca9ab3cbc2d59072b550"}
Aug 04 18:37:48.467263 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:48.467060    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq" event={"ID":"0f00cebc-ae97-4766-80b8-a38d2c281f53","Type":"ContainerDied","Data":"4efa94458df7f4c25e614619278edf432a8f1923e5fd33780c44b5b7322e046c"}
Aug 04 18:37:48.468549 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:48.468527    2560 generic.go:356] "Generic (PLEG): container finished" podID="3fff060f-820e-4f9f-83af-da10bd526d4f" containerID="19c7564f692371e3fccedcc23f425f4db0d4c24fa7da29779c71c974d3346a38" exitCode=0
Aug 04 18:37:48.468665 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:48.468558    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc" event={"ID":"3fff060f-820e-4f9f-83af-da10bd526d4f","Type":"ContainerDied","Data":"19c7564f692371e3fccedcc23f425f4db0d4c24fa7da29779c71c974d3346a38"}
Aug 04 18:37:48.468665 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:48.468573    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc"
Aug 04 18:37:48.468665 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:48.468591    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc" event={"ID":"3fff060f-820e-4f9f-83af-da10bd526d4f","Type":"ContainerDied","Data":"685e13e080734af90647a6efbcd2387ec674dd328cbb0ebdc577d05697229bf7"}
Aug 04 18:37:48.475696 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:48.475674    2560 scope.go:117] "RemoveContainer" containerID="ca8533757001887ff6b68dd49fc5eef2019a49e7d110ca9ab3cbc2d59072b550"
Aug 04 18:37:48.483717 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:48.483702    2560 scope.go:117] "RemoveContainer" containerID="ca8533757001887ff6b68dd49fc5eef2019a49e7d110ca9ab3cbc2d59072b550"
Aug 04 18:37:48.483963 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:37:48.483945    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca8533757001887ff6b68dd49fc5eef2019a49e7d110ca9ab3cbc2d59072b550\": container with ID starting with ca8533757001887ff6b68dd49fc5eef2019a49e7d110ca9ab3cbc2d59072b550 not found: ID does not exist" containerID="ca8533757001887ff6b68dd49fc5eef2019a49e7d110ca9ab3cbc2d59072b550"
Aug 04 18:37:48.484052 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:48.483968    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca8533757001887ff6b68dd49fc5eef2019a49e7d110ca9ab3cbc2d59072b550"} err="failed to get container status \"ca8533757001887ff6b68dd49fc5eef2019a49e7d110ca9ab3cbc2d59072b550\": rpc error: code = NotFound desc = could not find container \"ca8533757001887ff6b68dd49fc5eef2019a49e7d110ca9ab3cbc2d59072b550\": container with ID starting with ca8533757001887ff6b68dd49fc5eef2019a49e7d110ca9ab3cbc2d59072b550 not found: ID does not exist"
Aug 04 18:37:48.484052 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:48.483983    2560 scope.go:117] "RemoveContainer" containerID="19c7564f692371e3fccedcc23f425f4db0d4c24fa7da29779c71c974d3346a38"
Aug 04 18:37:48.492930 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:48.492910    2560 scope.go:117] "RemoveContainer" containerID="19c7564f692371e3fccedcc23f425f4db0d4c24fa7da29779c71c974d3346a38"
Aug 04 18:37:48.493184 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:37:48.493168    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19c7564f692371e3fccedcc23f425f4db0d4c24fa7da29779c71c974d3346a38\": container with ID starting with 19c7564f692371e3fccedcc23f425f4db0d4c24fa7da29779c71c974d3346a38 not found: ID does not exist" containerID="19c7564f692371e3fccedcc23f425f4db0d4c24fa7da29779c71c974d3346a38"
Aug 04 18:37:48.493248 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:48.493191    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19c7564f692371e3fccedcc23f425f4db0d4c24fa7da29779c71c974d3346a38"} err="failed to get container status \"19c7564f692371e3fccedcc23f425f4db0d4c24fa7da29779c71c974d3346a38\": rpc error: code = NotFound desc = could not find container \"19c7564f692371e3fccedcc23f425f4db0d4c24fa7da29779c71c974d3346a38\": container with ID starting with 19c7564f692371e3fccedcc23f425f4db0d4c24fa7da29779c71c974d3346a38 not found: ID does not exist"
Aug 04 18:37:48.493398 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:48.493372    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw"]
Aug 04 18:37:48.496069 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:48.496049    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a08d36-5f9c6ddd8-z9bgw"]
Aug 04 18:37:48.507771 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:48.507751    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc"]
Aug 04 18:37:48.509921 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:48.509898    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a08d36-775cbb76f6-wd5rc"]
Aug 04 18:37:48.521571 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:48.521554    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq"]
Aug 04 18:37:48.526241 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:48.526219    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a08d36-79b747cb9b-6tzkq"]
Aug 04 18:37:50.353763 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:50.353729    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f00cebc-ae97-4766-80b8-a38d2c281f53" path="/var/lib/kubelet/pods/0f00cebc-ae97-4766-80b8-a38d2c281f53/volumes"
Aug 04 18:37:50.354275 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:50.354251    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fff060f-820e-4f9f-83af-da10bd526d4f" path="/var/lib/kubelet/pods/3fff060f-820e-4f9f-83af-da10bd526d4f/volumes"
Aug 04 18:37:50.354644 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:37:50.354623    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8ba4898-2212-4585-a7f4-58173a9a5fee" path="/var/lib/kubelet/pods/b8ba4898-2212-4585-a7f4-58173a9a5fee/volumes"
Aug 04 18:38:10.468028 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:10.467965    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-6f74856975-5l6vm"]
Aug 04 18:38:10.468547 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:10.468227    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-6f74856975-5l6vm" podUID="3fff781b-238a-4425-9952-198c8fdbecd8" containerName="maas-api" containerID="cri-o://4e587da3f6892320d9b676a77626449628b9ee0973df03984deb47c9ad0c73fb" gracePeriod=30
Aug 04 18:38:10.529290 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:10.529259    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv"]
Aug 04 18:38:10.529534 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:10.529506    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv" podUID="5befe5b4-51c0-4117-9d74-cb0e758b5d33" containerName="payload-processing" containerID="cri-o://114da32b13d24669690c143994bacc7e44a1d280b2f6cbc553501593b10f075a" gracePeriod=30
Aug 04 18:38:10.538388 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:10.538364    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj"]
Aug 04 18:38:10.538606 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:10.538569    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj" podUID="6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f" containerName="payload-pre-processing" containerID="cri-o://c883dd56a14c0aa1a82ce1153dccb7811b89f39793b58b9dd810d479044e93de" gracePeriod=30
Aug 04 18:38:10.585106 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:10.585078    2560 generic.go:356] "Generic (PLEG): container finished" podID="3fff781b-238a-4425-9952-198c8fdbecd8" containerID="4e587da3f6892320d9b676a77626449628b9ee0973df03984deb47c9ad0c73fb" exitCode=0
Aug 04 18:38:10.585234 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:10.585149    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-6f74856975-5l6vm" event={"ID":"3fff781b-238a-4425-9952-198c8fdbecd8","Type":"ContainerDied","Data":"4e587da3f6892320d9b676a77626449628b9ee0973df03984deb47c9ad0c73fb"}
Aug 04 18:38:10.800237 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:10.800204    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv"
Aug 04 18:38:10.808742 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:10.808718    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-6f74856975-5l6vm"
Aug 04 18:38:10.833127 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:10.833109    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5befe5b4-51c0-4117-9d74-cb0e758b5d33-plugins-config-volume\") pod \"5befe5b4-51c0-4117-9d74-cb0e758b5d33\" (UID: \"5befe5b4-51c0-4117-9d74-cb0e758b5d33\") "
Aug 04 18:38:10.833211 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:10.833185    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vlfkq\" (UniqueName: \"kubernetes.io/projected/5befe5b4-51c0-4117-9d74-cb0e758b5d33-kube-api-access-vlfkq\") pod \"5befe5b4-51c0-4117-9d74-cb0e758b5d33\" (UID: \"5befe5b4-51c0-4117-9d74-cb0e758b5d33\") "
Aug 04 18:38:10.833510 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:10.833484    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5befe5b4-51c0-4117-9d74-cb0e758b5d33-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5befe5b4-51c0-4117-9d74-cb0e758b5d33" (UID: "5befe5b4-51c0-4117-9d74-cb0e758b5d33"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:38:10.835427 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:10.835399    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5befe5b4-51c0-4117-9d74-cb0e758b5d33-kube-api-access-vlfkq" (OuterVolumeSpecName: "kube-api-access-vlfkq") pod "5befe5b4-51c0-4117-9d74-cb0e758b5d33" (UID: "5befe5b4-51c0-4117-9d74-cb0e758b5d33"). InnerVolumeSpecName "kube-api-access-vlfkq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:38:10.838371 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:10.838351    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj"
Aug 04 18:38:10.934243 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:10.934211    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fff781b-238a-4425-9952-198c8fdbecd8-maas-api-tls\") pod \"3fff781b-238a-4425-9952-198c8fdbecd8\" (UID: \"3fff781b-238a-4425-9952-198c8fdbecd8\") "
Aug 04 18:38:10.934406 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:10.934277    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fjvj\" (UniqueName: \"kubernetes.io/projected/6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f-kube-api-access-5fjvj\") pod \"6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f\" (UID: \"6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f\") "
Aug 04 18:38:10.934406 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:10.934343    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fff781b-238a-4425-9952-198c8fdbecd8-openshift-service-ca\") pod \"3fff781b-238a-4425-9952-198c8fdbecd8\" (UID: \"3fff781b-238a-4425-9952-198c8fdbecd8\") "
Aug 04 18:38:10.934406 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:10.934378    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pcdtm\" (UniqueName: \"kubernetes.io/projected/3fff781b-238a-4425-9952-198c8fdbecd8-kube-api-access-pcdtm\") pod \"3fff781b-238a-4425-9952-198c8fdbecd8\" (UID: \"3fff781b-238a-4425-9952-198c8fdbecd8\") "
Aug 04 18:38:10.934406 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:10.934395    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f-plugins-config-volume\") pod \"6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f\" (UID: \"6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f\") "
Aug 04 18:38:10.934646 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:10.934587    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5befe5b4-51c0-4117-9d74-cb0e758b5d33-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:38:10.934646 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:10.934599    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vlfkq\" (UniqueName: \"kubernetes.io/projected/5befe5b4-51c0-4117-9d74-cb0e758b5d33-kube-api-access-vlfkq\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:38:10.934732 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:10.934705    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fff781b-238a-4425-9952-198c8fdbecd8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3fff781b-238a-4425-9952-198c8fdbecd8" (UID: "3fff781b-238a-4425-9952-198c8fdbecd8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:38:10.934806 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:10.934784    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f" (UID: "6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:38:10.936461 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:10.936441    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fff781b-238a-4425-9952-198c8fdbecd8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3fff781b-238a-4425-9952-198c8fdbecd8" (UID: "3fff781b-238a-4425-9952-198c8fdbecd8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:38:10.936563 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:10.936542    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f-kube-api-access-5fjvj" (OuterVolumeSpecName: "kube-api-access-5fjvj") pod "6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f" (UID: "6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f"). InnerVolumeSpecName "kube-api-access-5fjvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:38:10.936563 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:10.936549    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fff781b-238a-4425-9952-198c8fdbecd8-kube-api-access-pcdtm" (OuterVolumeSpecName: "kube-api-access-pcdtm") pod "3fff781b-238a-4425-9952-198c8fdbecd8" (UID: "3fff781b-238a-4425-9952-198c8fdbecd8"). InnerVolumeSpecName "kube-api-access-pcdtm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:38:11.035178 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:11.035126    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcdtm\" (UniqueName: \"kubernetes.io/projected/3fff781b-238a-4425-9952-198c8fdbecd8-kube-api-access-pcdtm\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:38:11.035178 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:11.035147    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:38:11.035178 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:11.035157    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fff781b-238a-4425-9952-198c8fdbecd8-maas-api-tls\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:38:11.035178 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:11.035165    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fjvj\" (UniqueName: \"kubernetes.io/projected/6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f-kube-api-access-5fjvj\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:38:11.035178 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:11.035175    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fff781b-238a-4425-9952-198c8fdbecd8-openshift-service-ca\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:38:11.593395 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:11.593360    2560 generic.go:356] "Generic (PLEG): container finished" podID="5befe5b4-51c0-4117-9d74-cb0e758b5d33" containerID="114da32b13d24669690c143994bacc7e44a1d280b2f6cbc553501593b10f075a" exitCode=0
Aug 04 18:38:11.593838 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:11.593432    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv"
Aug 04 18:38:11.593838 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:11.593446    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv" event={"ID":"5befe5b4-51c0-4117-9d74-cb0e758b5d33","Type":"ContainerDied","Data":"114da32b13d24669690c143994bacc7e44a1d280b2f6cbc553501593b10f075a"}
Aug 04 18:38:11.593838 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:11.593549    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv" event={"ID":"5befe5b4-51c0-4117-9d74-cb0e758b5d33","Type":"ContainerDied","Data":"119408774b89de2fe25ee44b691fff8763b6ccc3fad1fd5d1ad81e0042cc4d91"}
Aug 04 18:38:11.593838 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:11.593582    2560 scope.go:117] "RemoveContainer" containerID="114da32b13d24669690c143994bacc7e44a1d280b2f6cbc553501593b10f075a"
Aug 04 18:38:11.595157 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:11.595136    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-6f74856975-5l6vm"
Aug 04 18:38:11.595294 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:11.595209    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-6f74856975-5l6vm" event={"ID":"3fff781b-238a-4425-9952-198c8fdbecd8","Type":"ContainerDied","Data":"be6d649ce0fa56d2716d83784c076267bce86736b2e77cb768aa279c62604868"}
Aug 04 18:38:11.596549 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:11.596525    2560 generic.go:356] "Generic (PLEG): container finished" podID="6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f" containerID="c883dd56a14c0aa1a82ce1153dccb7811b89f39793b58b9dd810d479044e93de" exitCode=0
Aug 04 18:38:11.596655 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:11.596564    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj" event={"ID":"6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f","Type":"ContainerDied","Data":"c883dd56a14c0aa1a82ce1153dccb7811b89f39793b58b9dd810d479044e93de"}
Aug 04 18:38:11.596655 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:11.596578    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj"
Aug 04 18:38:11.596655 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:11.596602    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj" event={"ID":"6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f","Type":"ContainerDied","Data":"97f8481e3e263d2f9f6084f265f700e36a7bfbcd9f38509c18e26409eb16492b"}
Aug 04 18:38:11.604675 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:11.604658    2560 scope.go:117] "RemoveContainer" containerID="114da32b13d24669690c143994bacc7e44a1d280b2f6cbc553501593b10f075a"
Aug 04 18:38:11.604970 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:38:11.604947    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"114da32b13d24669690c143994bacc7e44a1d280b2f6cbc553501593b10f075a\": container with ID starting with 114da32b13d24669690c143994bacc7e44a1d280b2f6cbc553501593b10f075a not found: ID does not exist" containerID="114da32b13d24669690c143994bacc7e44a1d280b2f6cbc553501593b10f075a"
Aug 04 18:38:11.605060 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:11.605008    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"114da32b13d24669690c143994bacc7e44a1d280b2f6cbc553501593b10f075a"} err="failed to get container status \"114da32b13d24669690c143994bacc7e44a1d280b2f6cbc553501593b10f075a\": rpc error: code = NotFound desc = could not find container \"114da32b13d24669690c143994bacc7e44a1d280b2f6cbc553501593b10f075a\": container with ID starting with 114da32b13d24669690c143994bacc7e44a1d280b2f6cbc553501593b10f075a not found: ID does not exist"
Aug 04 18:38:11.605060 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:11.605029    2560 scope.go:117] "RemoveContainer" containerID="4e587da3f6892320d9b676a77626449628b9ee0973df03984deb47c9ad0c73fb"
Aug 04 18:38:11.613061 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:11.613047    2560 scope.go:117] "RemoveContainer" containerID="c883dd56a14c0aa1a82ce1153dccb7811b89f39793b58b9dd810d479044e93de"
Aug 04 18:38:11.620396 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:11.620381    2560 scope.go:117] "RemoveContainer" containerID="c883dd56a14c0aa1a82ce1153dccb7811b89f39793b58b9dd810d479044e93de"
Aug 04 18:38:11.620640 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:38:11.620625    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c883dd56a14c0aa1a82ce1153dccb7811b89f39793b58b9dd810d479044e93de\": container with ID starting with c883dd56a14c0aa1a82ce1153dccb7811b89f39793b58b9dd810d479044e93de not found: ID does not exist" containerID="c883dd56a14c0aa1a82ce1153dccb7811b89f39793b58b9dd810d479044e93de"
Aug 04 18:38:11.620685 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:11.620645    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c883dd56a14c0aa1a82ce1153dccb7811b89f39793b58b9dd810d479044e93de"} err="failed to get container status \"c883dd56a14c0aa1a82ce1153dccb7811b89f39793b58b9dd810d479044e93de\": rpc error: code = NotFound desc = could not find container \"c883dd56a14c0aa1a82ce1153dccb7811b89f39793b58b9dd810d479044e93de\": container with ID starting with c883dd56a14c0aa1a82ce1153dccb7811b89f39793b58b9dd810d479044e93de not found: ID does not exist"
Aug 04 18:38:11.624127 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:11.624104    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv"]
Aug 04 18:38:11.632062 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:11.632040    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a77bad-55bbf4b6d6-qwnkv"]
Aug 04 18:38:11.647563 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:11.647544    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj"]
Aug 04 18:38:11.655388 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:11.655368    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a77bad-754c57784d-4wcvj"]
Aug 04 18:38:11.670893 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:11.670874    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-6f74856975-5l6vm"]
Aug 04 18:38:11.680740 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:11.680721    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a77bad-6f74856975-5l6vm"]
Aug 04 18:38:12.353444 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:12.353413    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fff781b-238a-4425-9952-198c8fdbecd8" path="/var/lib/kubelet/pods/3fff781b-238a-4425-9952-198c8fdbecd8/volumes"
Aug 04 18:38:12.353762 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:12.353750    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5befe5b4-51c0-4117-9d74-cb0e758b5d33" path="/var/lib/kubelet/pods/5befe5b4-51c0-4117-9d74-cb0e758b5d33/volumes"
Aug 04 18:38:12.354080 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:12.354068    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f" path="/var/lib/kubelet/pods/6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f/volumes"
Aug 04 18:38:23.223189 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.223116    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx"]
Aug 04 18:38:23.223555 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.223506    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f00cebc-ae97-4766-80b8-a38d2c281f53" containerName="payload-processing"
Aug 04 18:38:23.223555 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.223522    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f00cebc-ae97-4766-80b8-a38d2c281f53" containerName="payload-processing"
Aug 04 18:38:23.223555 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.223532    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8ba4898-2212-4585-a7f4-58173a9a5fee" containerName="maas-api"
Aug 04 18:38:23.223555 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.223538    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8ba4898-2212-4585-a7f4-58173a9a5fee" containerName="maas-api"
Aug 04 18:38:23.223555 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.223546    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f" containerName="payload-pre-processing"
Aug 04 18:38:23.223555 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.223551    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f" containerName="payload-pre-processing"
Aug 04 18:38:23.223739 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.223561    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fff060f-820e-4f9f-83af-da10bd526d4f" containerName="payload-pre-processing"
Aug 04 18:38:23.223739 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.223568    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fff060f-820e-4f9f-83af-da10bd526d4f" containerName="payload-pre-processing"
Aug 04 18:38:23.223739 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.223593    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5befe5b4-51c0-4117-9d74-cb0e758b5d33" containerName="payload-processing"
Aug 04 18:38:23.223739 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.223600    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5befe5b4-51c0-4117-9d74-cb0e758b5d33" containerName="payload-processing"
Aug 04 18:38:23.223739 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.223610    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fff781b-238a-4425-9952-198c8fdbecd8" containerName="maas-api"
Aug 04 18:38:23.223739 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.223615    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fff781b-238a-4425-9952-198c8fdbecd8" containerName="maas-api"
Aug 04 18:38:23.223739 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.223674    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f00cebc-ae97-4766-80b8-a38d2c281f53" containerName="payload-processing"
Aug 04 18:38:23.223739 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.223687    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8ba4898-2212-4585-a7f4-58173a9a5fee" containerName="maas-api"
Aug 04 18:38:23.223739 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.223692    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fff781b-238a-4425-9952-198c8fdbecd8" containerName="maas-api"
Aug 04 18:38:23.223739 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.223698    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fff060f-820e-4f9f-83af-da10bd526d4f" containerName="payload-pre-processing"
Aug 04 18:38:23.223739 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.223704    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f9e4bb1-04fd-4d0a-b9e4-61fec8ef342f" containerName="payload-pre-processing"
Aug 04 18:38:23.223739 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.223712    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5befe5b4-51c0-4117-9d74-cb0e758b5d33" containerName="payload-processing"
Aug 04 18:38:23.228352 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.228332    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx"
Aug 04 18:38:23.232736 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.232705    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-5c8685-serving-cert\""
Aug 04 18:38:23.248891 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.248857    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx"]
Aug 04 18:38:23.345407 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.345382    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9d6f224-ceb0-4f3f-8377-6467e9b99d14-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx\" (UID: \"f9d6f224-ceb0-4f3f-8377-6467e9b99d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx"
Aug 04 18:38:23.345540 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.345420    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9d6f224-ceb0-4f3f-8377-6467e9b99d14-maas-api-tls\") pod \"maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx\" (UID: \"f9d6f224-ceb0-4f3f-8377-6467e9b99d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx"
Aug 04 18:38:23.345540 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.345448    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l25m\" (UniqueName: \"kubernetes.io/projected/f9d6f224-ceb0-4f3f-8377-6467e9b99d14-kube-api-access-9l25m\") pod \"maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx\" (UID: \"f9d6f224-ceb0-4f3f-8377-6467e9b99d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx"
Aug 04 18:38:23.446051 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.446018    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9d6f224-ceb0-4f3f-8377-6467e9b99d14-maas-api-tls\") pod \"maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx\" (UID: \"f9d6f224-ceb0-4f3f-8377-6467e9b99d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx"
Aug 04 18:38:23.446173 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.446093    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l25m\" (UniqueName: \"kubernetes.io/projected/f9d6f224-ceb0-4f3f-8377-6467e9b99d14-kube-api-access-9l25m\") pod \"maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx\" (UID: \"f9d6f224-ceb0-4f3f-8377-6467e9b99d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx"
Aug 04 18:38:23.446173 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:38:23.446155    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-serving-cert: secret "maas-api-e2e-shared-a-5c8685-serving-cert" not found
Aug 04 18:38:23.446285 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.446191    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9d6f224-ceb0-4f3f-8377-6467e9b99d14-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx\" (UID: \"f9d6f224-ceb0-4f3f-8377-6467e9b99d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx"
Aug 04 18:38:23.446285 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:38:23.446222    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9d6f224-ceb0-4f3f-8377-6467e9b99d14-maas-api-tls podName:f9d6f224-ceb0-4f3f-8377-6467e9b99d14 nodeName:}" failed. No retries permitted until 2026-08-04 18:38:23.946206154 +0000 UTC m=+2786.129366308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f9d6f224-ceb0-4f3f-8377-6467e9b99d14-maas-api-tls") pod "maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx" (UID: "f9d6f224-ceb0-4f3f-8377-6467e9b99d14") : secret "maas-api-e2e-shared-a-5c8685-serving-cert" not found
Aug 04 18:38:23.446837 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.446817    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9d6f224-ceb0-4f3f-8377-6467e9b99d14-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx\" (UID: \"f9d6f224-ceb0-4f3f-8377-6467e9b99d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx"
Aug 04 18:38:23.458023 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.457976    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l25m\" (UniqueName: \"kubernetes.io/projected/f9d6f224-ceb0-4f3f-8377-6467e9b99d14-kube-api-access-9l25m\") pod \"maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx\" (UID: \"f9d6f224-ceb0-4f3f-8377-6467e9b99d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx"
Aug 04 18:38:23.950833 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.950796    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9d6f224-ceb0-4f3f-8377-6467e9b99d14-maas-api-tls\") pod \"maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx\" (UID: \"f9d6f224-ceb0-4f3f-8377-6467e9b99d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx"
Aug 04 18:38:23.953347 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.953322    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9d6f224-ceb0-4f3f-8377-6467e9b99d14-maas-api-tls\") pod \"maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx\" (UID: \"f9d6f224-ceb0-4f3f-8377-6467e9b99d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx"
Aug 04 18:38:23.964058 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.964034    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v"]
Aug 04 18:38:23.967715 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.967698    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v"
Aug 04 18:38:23.970168 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.970149    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-5c8685\""
Aug 04 18:38:23.970272 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.970166    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-5c8685-dockercfg-mnhbb\""
Aug 04 18:38:23.979086 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.979065    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v"]
Aug 04 18:38:23.999977 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:23.999959    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2"]
Aug 04 18:38:24.003344 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.003329    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2"
Aug 04 18:38:24.014160 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.014137    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2"]
Aug 04 18:38:24.051650 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.051623    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed1e3367-3fc6-454c-bf49-4a66dd82e41b-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v\" (UID: \"ed1e3367-3fc6-454c-bf49-4a66dd82e41b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v"
Aug 04 18:38:24.051746 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.051706    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ck9b5\" (UniqueName: \"kubernetes.io/projected/ed1e3367-3fc6-454c-bf49-4a66dd82e41b-kube-api-access-ck9b5\") pod \"payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v\" (UID: \"ed1e3367-3fc6-454c-bf49-4a66dd82e41b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v"
Aug 04 18:38:24.142475 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.142447    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx"
Aug 04 18:38:24.152675 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.152649    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ck9b5\" (UniqueName: \"kubernetes.io/projected/ed1e3367-3fc6-454c-bf49-4a66dd82e41b-kube-api-access-ck9b5\") pod \"payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v\" (UID: \"ed1e3367-3fc6-454c-bf49-4a66dd82e41b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v"
Aug 04 18:38:24.152817 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.152719    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed1e3367-3fc6-454c-bf49-4a66dd82e41b-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v\" (UID: \"ed1e3367-3fc6-454c-bf49-4a66dd82e41b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v"
Aug 04 18:38:24.152817 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.152776    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx9bx\" (UniqueName: \"kubernetes.io/projected/9d085af0-a8cb-48ed-86ce-d6e190d7afd9-kube-api-access-vx9bx\") pod \"payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2\" (UID: \"9d085af0-a8cb-48ed-86ce-d6e190d7afd9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2"
Aug 04 18:38:24.152939 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.152828    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d085af0-a8cb-48ed-86ce-d6e190d7afd9-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2\" (UID: \"9d085af0-a8cb-48ed-86ce-d6e190d7afd9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2"
Aug 04 18:38:24.153518 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.153491    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed1e3367-3fc6-454c-bf49-4a66dd82e41b-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v\" (UID: \"ed1e3367-3fc6-454c-bf49-4a66dd82e41b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v"
Aug 04 18:38:24.161967 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.161930    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ck9b5\" (UniqueName: \"kubernetes.io/projected/ed1e3367-3fc6-454c-bf49-4a66dd82e41b-kube-api-access-ck9b5\") pod \"payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v\" (UID: \"ed1e3367-3fc6-454c-bf49-4a66dd82e41b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v"
Aug 04 18:38:24.254327 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.254291    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx9bx\" (UniqueName: \"kubernetes.io/projected/9d085af0-a8cb-48ed-86ce-d6e190d7afd9-kube-api-access-vx9bx\") pod \"payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2\" (UID: \"9d085af0-a8cb-48ed-86ce-d6e190d7afd9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2"
Aug 04 18:38:24.254717 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.254368    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d085af0-a8cb-48ed-86ce-d6e190d7afd9-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2\" (UID: \"9d085af0-a8cb-48ed-86ce-d6e190d7afd9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2"
Aug 04 18:38:24.255347 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.255317    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d085af0-a8cb-48ed-86ce-d6e190d7afd9-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2\" (UID: \"9d085af0-a8cb-48ed-86ce-d6e190d7afd9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2"
Aug 04 18:38:24.263774 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.263750    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx9bx\" (UniqueName: \"kubernetes.io/projected/9d085af0-a8cb-48ed-86ce-d6e190d7afd9-kube-api-access-vx9bx\") pod \"payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2\" (UID: \"9d085af0-a8cb-48ed-86ce-d6e190d7afd9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2"
Aug 04 18:38:24.278199 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.278178    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v"
Aug 04 18:38:24.283462 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.283372    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx"]
Aug 04 18:38:24.285839 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:38:24.285813    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9d6f224_ceb0_4f3f_8377_6467e9b99d14.slice/crio-2d467f15d0dc6e107c082c5b33cb766000319cc308dcadbd136aee713c01af04 WatchSource:0}: Error finding container 2d467f15d0dc6e107c082c5b33cb766000319cc308dcadbd136aee713c01af04: Status 404 returned error can't find the container with id 2d467f15d0dc6e107c082c5b33cb766000319cc308dcadbd136aee713c01af04
Aug 04 18:38:24.313043 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.313021    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2"
Aug 04 18:38:24.434250 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.434218    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v"]
Aug 04 18:38:24.437436 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:38:24.437385    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded1e3367_3fc6_454c_bf49_4a66dd82e41b.slice/crio-ff9e43958419a5a5eb94177fff62270c60a3da5acdedb3d93963ebd50ab893dc WatchSource:0}: Error finding container ff9e43958419a5a5eb94177fff62270c60a3da5acdedb3d93963ebd50ab893dc: Status 404 returned error can't find the container with id ff9e43958419a5a5eb94177fff62270c60a3da5acdedb3d93963ebd50ab893dc
Aug 04 18:38:24.463927 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.463903    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2"]
Aug 04 18:38:24.466117 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:38:24.466092    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d085af0_a8cb_48ed_86ce_d6e190d7afd9.slice/crio-b39d6a8a971eb77566170f93d6484714baa5e093dda717eab8a91ec331905959 WatchSource:0}: Error finding container b39d6a8a971eb77566170f93d6484714baa5e093dda717eab8a91ec331905959: Status 404 returned error can't find the container with id b39d6a8a971eb77566170f93d6484714baa5e093dda717eab8a91ec331905959
Aug 04 18:38:24.662473 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.662441    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v" event={"ID":"ed1e3367-3fc6-454c-bf49-4a66dd82e41b","Type":"ContainerStarted","Data":"f4b052a27bc2f579de0ed6b6cdd697b7c3aa4d3a5fff00b57f9f859a6ff969c2"}
Aug 04 18:38:24.662655 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.662495    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v" event={"ID":"ed1e3367-3fc6-454c-bf49-4a66dd82e41b","Type":"ContainerStarted","Data":"ff9e43958419a5a5eb94177fff62270c60a3da5acdedb3d93963ebd50ab893dc"}
Aug 04 18:38:24.662655 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.662526    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v"
Aug 04 18:38:24.664045 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.664019    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2" event={"ID":"9d085af0-a8cb-48ed-86ce-d6e190d7afd9","Type":"ContainerStarted","Data":"271e85b91865d7b9c748f63cb3fa15a46a3299b08ed4dd790dbac1cb5de9743d"}
Aug 04 18:38:24.664183 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.664060    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2" event={"ID":"9d085af0-a8cb-48ed-86ce-d6e190d7afd9","Type":"ContainerStarted","Data":"b39d6a8a971eb77566170f93d6484714baa5e093dda717eab8a91ec331905959"}
Aug 04 18:38:24.664183 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.664078    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2"
Aug 04 18:38:24.665223 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.665204    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx" event={"ID":"f9d6f224-ceb0-4f3f-8377-6467e9b99d14","Type":"ContainerStarted","Data":"2d467f15d0dc6e107c082c5b33cb766000319cc308dcadbd136aee713c01af04"}
Aug 04 18:38:24.681155 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.681115    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v" podStartSLOduration=1.6811016680000002 podStartE2EDuration="1.681101668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:38:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:38:24.679371155 +0000 UTC m=+2786.862531331" watchObservedRunningTime="2026-08-04 18:38:24.681101668 +0000 UTC m=+2786.864261843"
Aug 04 18:38:24.696129 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:24.696087    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2" podStartSLOduration=1.696073745 podStartE2EDuration="1.696073745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:38:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:38:24.694621101 +0000 UTC m=+2786.877781278" watchObservedRunningTime="2026-08-04 18:38:24.696073745 +0000 UTC m=+2786.879233922"
Aug 04 18:38:25.672581 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:25.672548    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx" event={"ID":"f9d6f224-ceb0-4f3f-8377-6467e9b99d14","Type":"ContainerStarted","Data":"f854ce127d3c1e899cc2832a86fa4fb5f240d2744585fbd4f8595eb116f10cb9"}
Aug 04 18:38:25.672976 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:25.672632    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx"
Aug 04 18:38:25.693379 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:25.693336    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx" podStartSLOduration=2.2246809499999998 podStartE2EDuration="2.693323582s" totalImagesPullingTime="468.642632ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:38:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:38:25.69178444 +0000 UTC m=+2787.874944617" watchObservedRunningTime="2026-08-04 18:38:25.693323582 +0000 UTC m=+2787.876483809"
Aug 04 18:38:31.683558 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:31.683525    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx"
Aug 04 18:38:35.674684 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:35.674655    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2"
Aug 04 18:38:35.675074 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:35.674809    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v"
Aug 04 18:38:52.185443 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.185410    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p"]
Aug 04 18:38:52.189669 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.189646    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p"
Aug 04 18:38:52.192032 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.191983    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-bc18f1-serving-cert\""
Aug 04 18:38:52.212400 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.212371    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p"]
Aug 04 18:38:52.285649 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.285616    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20d4d278-4eae-4298-8f17-878684d2b21c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p\" (UID: \"20d4d278-4eae-4298-8f17-878684d2b21c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p"
Aug 04 18:38:52.285794 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.285655    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20d4d278-4eae-4298-8f17-878684d2b21c-maas-api-tls\") pod \"maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p\" (UID: \"20d4d278-4eae-4298-8f17-878684d2b21c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p"
Aug 04 18:38:52.285794 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.285752    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdgg9\" (UniqueName: \"kubernetes.io/projected/20d4d278-4eae-4298-8f17-878684d2b21c-kube-api-access-sdgg9\") pod \"maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p\" (UID: \"20d4d278-4eae-4298-8f17-878684d2b21c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p"
Aug 04 18:38:52.386592 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.386564    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdgg9\" (UniqueName: \"kubernetes.io/projected/20d4d278-4eae-4298-8f17-878684d2b21c-kube-api-access-sdgg9\") pod \"maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p\" (UID: \"20d4d278-4eae-4298-8f17-878684d2b21c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p"
Aug 04 18:38:52.386726 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.386611    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20d4d278-4eae-4298-8f17-878684d2b21c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p\" (UID: \"20d4d278-4eae-4298-8f17-878684d2b21c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p"
Aug 04 18:38:52.386726 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.386639    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20d4d278-4eae-4298-8f17-878684d2b21c-maas-api-tls\") pod \"maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p\" (UID: \"20d4d278-4eae-4298-8f17-878684d2b21c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p"
Aug 04 18:38:52.386844 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:38:52.386746    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-serving-cert: secret "maas-api-e2e-shared-b-bc18f1-serving-cert" not found
Aug 04 18:38:52.386844 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:38:52.386803    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20d4d278-4eae-4298-8f17-878684d2b21c-maas-api-tls podName:20d4d278-4eae-4298-8f17-878684d2b21c nodeName:}" failed. No retries permitted until 2026-08-04 18:38:52.886784521 +0000 UTC m=+2815.069944678 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/20d4d278-4eae-4298-8f17-878684d2b21c-maas-api-tls") pod "maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p" (UID: "20d4d278-4eae-4298-8f17-878684d2b21c") : secret "maas-api-e2e-shared-b-bc18f1-serving-cert" not found
Aug 04 18:38:52.387295 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.387276    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20d4d278-4eae-4298-8f17-878684d2b21c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p\" (UID: \"20d4d278-4eae-4298-8f17-878684d2b21c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p"
Aug 04 18:38:52.398346 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.398321    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdgg9\" (UniqueName: \"kubernetes.io/projected/20d4d278-4eae-4298-8f17-878684d2b21c-kube-api-access-sdgg9\") pod \"maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p\" (UID: \"20d4d278-4eae-4298-8f17-878684d2b21c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p"
Aug 04 18:38:52.844832 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.844799    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w"]
Aug 04 18:38:52.848647 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.848628    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm"]
Aug 04 18:38:52.848776 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.848750    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w"
Aug 04 18:38:52.851112 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.851092    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-bc18f1\""
Aug 04 18:38:52.851812 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.851793    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm"
Aug 04 18:38:52.853806 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.853792    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-bc18f1-dockercfg-k9wvv\""
Aug 04 18:38:52.858427 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.858408    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w"]
Aug 04 18:38:52.860739 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.860719    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm"]
Aug 04 18:38:52.889380 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.889345    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec7e9cc7-74e3-4eb1-9035-f2186991f9f3-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w\" (UID: \"ec7e9cc7-74e3-4eb1-9035-f2186991f9f3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w"
Aug 04 18:38:52.889380 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.889376    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmp9d\" (UniqueName: \"kubernetes.io/projected/35429f65-863d-45be-a5ad-1bee106b1ba5-kube-api-access-xmp9d\") pod \"payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm\" (UID: \"35429f65-863d-45be-a5ad-1bee106b1ba5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm"
Aug 04 18:38:52.889549 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.889400    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20d4d278-4eae-4298-8f17-878684d2b21c-maas-api-tls\") pod \"maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p\" (UID: \"20d4d278-4eae-4298-8f17-878684d2b21c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p"
Aug 04 18:38:52.889549 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.889483    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35429f65-863d-45be-a5ad-1bee106b1ba5-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm\" (UID: \"35429f65-863d-45be-a5ad-1bee106b1ba5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm"
Aug 04 18:38:52.889672 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.889554    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96zdx\" (UniqueName: \"kubernetes.io/projected/ec7e9cc7-74e3-4eb1-9035-f2186991f9f3-kube-api-access-96zdx\") pod \"payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w\" (UID: \"ec7e9cc7-74e3-4eb1-9035-f2186991f9f3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w"
Aug 04 18:38:52.891917 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.891899    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20d4d278-4eae-4298-8f17-878684d2b21c-maas-api-tls\") pod \"maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p\" (UID: \"20d4d278-4eae-4298-8f17-878684d2b21c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p"
Aug 04 18:38:52.990139 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.990117    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96zdx\" (UniqueName: \"kubernetes.io/projected/ec7e9cc7-74e3-4eb1-9035-f2186991f9f3-kube-api-access-96zdx\") pod \"payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w\" (UID: \"ec7e9cc7-74e3-4eb1-9035-f2186991f9f3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w"
Aug 04 18:38:52.990274 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.990222    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec7e9cc7-74e3-4eb1-9035-f2186991f9f3-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w\" (UID: \"ec7e9cc7-74e3-4eb1-9035-f2186991f9f3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w"
Aug 04 18:38:52.990274 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.990250    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmp9d\" (UniqueName: \"kubernetes.io/projected/35429f65-863d-45be-a5ad-1bee106b1ba5-kube-api-access-xmp9d\") pod \"payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm\" (UID: \"35429f65-863d-45be-a5ad-1bee106b1ba5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm"
Aug 04 18:38:52.990394 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.990283    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35429f65-863d-45be-a5ad-1bee106b1ba5-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm\" (UID: \"35429f65-863d-45be-a5ad-1bee106b1ba5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm"
Aug 04 18:38:52.990936 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.990904    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35429f65-863d-45be-a5ad-1bee106b1ba5-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm\" (UID: \"35429f65-863d-45be-a5ad-1bee106b1ba5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm"
Aug 04 18:38:52.991038 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.990939    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec7e9cc7-74e3-4eb1-9035-f2186991f9f3-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w\" (UID: \"ec7e9cc7-74e3-4eb1-9035-f2186991f9f3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w"
Aug 04 18:38:52.999275 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.999252    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96zdx\" (UniqueName: \"kubernetes.io/projected/ec7e9cc7-74e3-4eb1-9035-f2186991f9f3-kube-api-access-96zdx\") pod \"payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w\" (UID: \"ec7e9cc7-74e3-4eb1-9035-f2186991f9f3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w"
Aug 04 18:38:52.999361 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:52.999276    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmp9d\" (UniqueName: \"kubernetes.io/projected/35429f65-863d-45be-a5ad-1bee106b1ba5-kube-api-access-xmp9d\") pod \"payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm\" (UID: \"35429f65-863d-45be-a5ad-1bee106b1ba5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm"
Aug 04 18:38:53.111449 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:53.111377    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p"
Aug 04 18:38:53.161944 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:53.161903    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w"
Aug 04 18:38:53.170324 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:53.170299    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm"
Aug 04 18:38:53.278449 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:53.278417    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p"]
Aug 04 18:38:53.280917 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:38:53.280876    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20d4d278_4eae_4298_8f17_878684d2b21c.slice/crio-642f13c308ec0338b58fe6fbf794ec1091fbfd4ff1cf4c1c0066d5e11dae7cf6 WatchSource:0}: Error finding container 642f13c308ec0338b58fe6fbf794ec1091fbfd4ff1cf4c1c0066d5e11dae7cf6: Status 404 returned error can't find the container with id 642f13c308ec0338b58fe6fbf794ec1091fbfd4ff1cf4c1c0066d5e11dae7cf6
Aug 04 18:38:53.540809 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:53.540643    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm"]
Aug 04 18:38:53.543227 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:38:53.543189    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35429f65_863d_45be_a5ad_1bee106b1ba5.slice/crio-08dcec4beb60cc0f0d5bcfbe3b53a0f9576d0aea65abea7fb79c29256ca40094 WatchSource:0}: Error finding container 08dcec4beb60cc0f0d5bcfbe3b53a0f9576d0aea65abea7fb79c29256ca40094: Status 404 returned error can't find the container with id 08dcec4beb60cc0f0d5bcfbe3b53a0f9576d0aea65abea7fb79c29256ca40094
Aug 04 18:38:53.563215 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:53.563194    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w"]
Aug 04 18:38:53.565779 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:38:53.565759    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec7e9cc7_74e3_4eb1_9035_f2186991f9f3.slice/crio-0bd7352c1fc14540809cfd8669ad0e5499a9ae3cb674509e63461399fb7a094f WatchSource:0}: Error finding container 0bd7352c1fc14540809cfd8669ad0e5499a9ae3cb674509e63461399fb7a094f: Status 404 returned error can't find the container with id 0bd7352c1fc14540809cfd8669ad0e5499a9ae3cb674509e63461399fb7a094f
Aug 04 18:38:53.809491 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:53.809393    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm" event={"ID":"35429f65-863d-45be-a5ad-1bee106b1ba5","Type":"ContainerStarted","Data":"26db26eaeed416cb7704cb05b6bec09bdfcaca5ff937b32726dce1c2aaac2e13"}
Aug 04 18:38:53.809491 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:53.809456    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm" event={"ID":"35429f65-863d-45be-a5ad-1bee106b1ba5","Type":"ContainerStarted","Data":"08dcec4beb60cc0f0d5bcfbe3b53a0f9576d0aea65abea7fb79c29256ca40094"}
Aug 04 18:38:53.809491 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:53.809487    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm"
Aug 04 18:38:53.811020 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:53.810977    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p" event={"ID":"20d4d278-4eae-4298-8f17-878684d2b21c","Type":"ContainerStarted","Data":"37ed8ec9ffd75706087e509570e634aff1f17014fc6b806bd45ed14482d24b19"}
Aug 04 18:38:53.811198 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:53.811039    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p" event={"ID":"20d4d278-4eae-4298-8f17-878684d2b21c","Type":"ContainerStarted","Data":"642f13c308ec0338b58fe6fbf794ec1091fbfd4ff1cf4c1c0066d5e11dae7cf6"}
Aug 04 18:38:53.811198 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:53.811060    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p"
Aug 04 18:38:53.812524 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:53.812504    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w" event={"ID":"ec7e9cc7-74e3-4eb1-9035-f2186991f9f3","Type":"ContainerStarted","Data":"83361c7573cc9164a730e877e7a8ad5a28ff51af44ff63d4f6b71efbecd9cd9a"}
Aug 04 18:38:53.812613 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:53.812543    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w" event={"ID":"ec7e9cc7-74e3-4eb1-9035-f2186991f9f3","Type":"ContainerStarted","Data":"0bd7352c1fc14540809cfd8669ad0e5499a9ae3cb674509e63461399fb7a094f"}
Aug 04 18:38:53.812664 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:53.812629    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w"
Aug 04 18:38:53.833138 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:53.833089    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm" podStartSLOduration=1.833069657 podStartE2EDuration="1.833069657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:38:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:38:53.831011674 +0000 UTC m=+2816.014171844" watchObservedRunningTime="2026-08-04 18:38:53.833069657 +0000 UTC m=+2816.016229837"
Aug 04 18:38:53.905887 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:53.905843    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p" podStartSLOduration=1.5470063710000002 podStartE2EDuration="1.905829507s" totalImagesPullingTime="358.823136ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:38:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:38:53.877906724 +0000 UTC m=+2816.061066901" watchObservedRunningTime="2026-08-04 18:38:53.905829507 +0000 UTC m=+2816.088989681"
Aug 04 18:38:53.906036 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:53.905921    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w" podStartSLOduration=1.905915316 podStartE2EDuration="1.905915316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:38:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:38:53.904307089 +0000 UTC m=+2816.087467264" watchObservedRunningTime="2026-08-04 18:38:53.905915316 +0000 UTC m=+2816.089075489"
Aug 04 18:38:59.823934 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:38:59.823907    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p"
Aug 04 18:39:04.820109 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:39:04.820069    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm"
Aug 04 18:39:04.820787 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:39:04.820324    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w"
Aug 04 18:41:58.547474 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:41:58.547347    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/2.log"
Aug 04 18:41:58.553803 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:41:58.553780    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 18:41:58.555494 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:41:58.555476    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/2.log"
Aug 04 18:41:58.561654 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:41:58.561634    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 18:44:27.208366 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:27.208292    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"]
Aug 04 18:44:27.211882 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:27.211863    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"
Aug 04 18:44:27.214454 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:27.214432    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5c8685\"/\"kube-root-ca.crt\""
Aug 04 18:44:27.215192 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:27.215170    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5c8685\"/\"openshift-service-ca.crt\""
Aug 04 18:44:27.215294 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:27.215232    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5c8685\"/\"default-dockercfg-bfqmh\""
Aug 04 18:44:27.215359 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:27.215301    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5c8685\"/\"rate-test-model-5c8685-kserve-self-signed-certs\""
Aug 04 18:44:27.222979 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:27.222951    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"]
Aug 04 18:44:27.307134 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:27.307099    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37f2b861-3a95-4630-9b57-47e8cc147b9d-model-cache\") pod \"rate-test-model-5c8685-kserve-854646fd79-nr2s6\" (UID: \"37f2b861-3a95-4630-9b57-47e8cc147b9d\") " pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"
Aug 04 18:44:27.307134 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:27.307135    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37f2b861-3a95-4630-9b57-47e8cc147b9d-tls-certs\") pod \"rate-test-model-5c8685-kserve-854646fd79-nr2s6\" (UID: \"37f2b861-3a95-4630-9b57-47e8cc147b9d\") " pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"
Aug 04 18:44:27.307363 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:27.307207    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fjz2\" (UniqueName: \"kubernetes.io/projected/37f2b861-3a95-4630-9b57-47e8cc147b9d-kube-api-access-7fjz2\") pod \"rate-test-model-5c8685-kserve-854646fd79-nr2s6\" (UID: \"37f2b861-3a95-4630-9b57-47e8cc147b9d\") " pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"
Aug 04 18:44:27.307363 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:27.307253    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37f2b861-3a95-4630-9b57-47e8cc147b9d-home\") pod \"rate-test-model-5c8685-kserve-854646fd79-nr2s6\" (UID: \"37f2b861-3a95-4630-9b57-47e8cc147b9d\") " pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"
Aug 04 18:44:27.307363 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:27.307278    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37f2b861-3a95-4630-9b57-47e8cc147b9d-dshm\") pod \"rate-test-model-5c8685-kserve-854646fd79-nr2s6\" (UID: \"37f2b861-3a95-4630-9b57-47e8cc147b9d\") " pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"
Aug 04 18:44:27.307363 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:27.307302    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37f2b861-3a95-4630-9b57-47e8cc147b9d-tmp-dir\") pod \"rate-test-model-5c8685-kserve-854646fd79-nr2s6\" (UID: \"37f2b861-3a95-4630-9b57-47e8cc147b9d\") " pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"
Aug 04 18:44:27.408674 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:27.408631    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fjz2\" (UniqueName: \"kubernetes.io/projected/37f2b861-3a95-4630-9b57-47e8cc147b9d-kube-api-access-7fjz2\") pod \"rate-test-model-5c8685-kserve-854646fd79-nr2s6\" (UID: \"37f2b861-3a95-4630-9b57-47e8cc147b9d\") " pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"
Aug 04 18:44:27.408890 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:27.408693    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37f2b861-3a95-4630-9b57-47e8cc147b9d-home\") pod \"rate-test-model-5c8685-kserve-854646fd79-nr2s6\" (UID: \"37f2b861-3a95-4630-9b57-47e8cc147b9d\") " pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"
Aug 04 18:44:27.408890 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:27.408730    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37f2b861-3a95-4630-9b57-47e8cc147b9d-dshm\") pod \"rate-test-model-5c8685-kserve-854646fd79-nr2s6\" (UID: \"37f2b861-3a95-4630-9b57-47e8cc147b9d\") " pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"
Aug 04 18:44:27.408890 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:27.408764    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37f2b861-3a95-4630-9b57-47e8cc147b9d-tmp-dir\") pod \"rate-test-model-5c8685-kserve-854646fd79-nr2s6\" (UID: \"37f2b861-3a95-4630-9b57-47e8cc147b9d\") " pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"
Aug 04 18:44:27.408890 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:27.408838    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37f2b861-3a95-4630-9b57-47e8cc147b9d-model-cache\") pod \"rate-test-model-5c8685-kserve-854646fd79-nr2s6\" (UID: \"37f2b861-3a95-4630-9b57-47e8cc147b9d\") " pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"
Aug 04 18:44:27.408890 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:27.408862    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37f2b861-3a95-4630-9b57-47e8cc147b9d-tls-certs\") pod \"rate-test-model-5c8685-kserve-854646fd79-nr2s6\" (UID: \"37f2b861-3a95-4630-9b57-47e8cc147b9d\") " pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"
Aug 04 18:44:27.409234 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:27.409210    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37f2b861-3a95-4630-9b57-47e8cc147b9d-home\") pod \"rate-test-model-5c8685-kserve-854646fd79-nr2s6\" (UID: \"37f2b861-3a95-4630-9b57-47e8cc147b9d\") " pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"
Aug 04 18:44:27.409291 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:27.409231    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37f2b861-3a95-4630-9b57-47e8cc147b9d-model-cache\") pod \"rate-test-model-5c8685-kserve-854646fd79-nr2s6\" (UID: \"37f2b861-3a95-4630-9b57-47e8cc147b9d\") " pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"
Aug 04 18:44:27.409341 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:27.409300    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37f2b861-3a95-4630-9b57-47e8cc147b9d-tmp-dir\") pod \"rate-test-model-5c8685-kserve-854646fd79-nr2s6\" (UID: \"37f2b861-3a95-4630-9b57-47e8cc147b9d\") " pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"
Aug 04 18:44:27.411357 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:27.411327    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37f2b861-3a95-4630-9b57-47e8cc147b9d-dshm\") pod \"rate-test-model-5c8685-kserve-854646fd79-nr2s6\" (UID: \"37f2b861-3a95-4630-9b57-47e8cc147b9d\") " pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"
Aug 04 18:44:27.411724 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:27.411703    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37f2b861-3a95-4630-9b57-47e8cc147b9d-tls-certs\") pod \"rate-test-model-5c8685-kserve-854646fd79-nr2s6\" (UID: \"37f2b861-3a95-4630-9b57-47e8cc147b9d\") " pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"
Aug 04 18:44:27.419132 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:27.419110    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fjz2\" (UniqueName: \"kubernetes.io/projected/37f2b861-3a95-4630-9b57-47e8cc147b9d-kube-api-access-7fjz2\") pod \"rate-test-model-5c8685-kserve-854646fd79-nr2s6\" (UID: \"37f2b861-3a95-4630-9b57-47e8cc147b9d\") " pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"
Aug 04 18:44:27.521642 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:27.521553    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"
Aug 04 18:44:27.663160 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:27.663133    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"]
Aug 04 18:44:27.665182 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:44:27.665156    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod37f2b861_3a95_4630_9b57_47e8cc147b9d.slice/crio-274a9d97366490efd376592b3fd5b3be226131883b5ac6c4944c32d478b2716e WatchSource:0}: Error finding container 274a9d97366490efd376592b3fd5b3be226131883b5ac6c4944c32d478b2716e: Status 404 returned error can't find the container with id 274a9d97366490efd376592b3fd5b3be226131883b5ac6c4944c32d478b2716e
Aug 04 18:44:28.483138 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:28.483101    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6" event={"ID":"37f2b861-3a95-4630-9b57-47e8cc147b9d","Type":"ContainerStarted","Data":"922e4fe4872cc861b177ddd823a411079c51d92b521ffa9ff7e9cbade3bc2cb1"}
Aug 04 18:44:28.483551 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:28.483146    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6" event={"ID":"37f2b861-3a95-4630-9b57-47e8cc147b9d","Type":"ContainerStarted","Data":"274a9d97366490efd376592b3fd5b3be226131883b5ac6c4944c32d478b2716e"}
Aug 04 18:44:28.506086 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:28.506036    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6" podStartSLOduration=1.506019438 podStartE2EDuration="1.506019438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:44:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:44:28.504798198 +0000 UTC m=+3150.687958373" watchObservedRunningTime="2026-08-04 18:44:28.506019438 +0000 UTC m=+3150.689179646"
Aug 04 18:44:37.522172 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:37.522137    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"
Aug 04 18:44:37.522551 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:37.522183    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"
Aug 04 18:44:37.534376 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:37.534354    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"
Aug 04 18:44:37.545283 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:44:37.545261    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"
Aug 04 18:46:16.381408 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:16.381371    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"]
Aug 04 18:46:31.401234 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:31.401186    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6" podUID="37f2b861-3a95-4630-9b57-47e8cc147b9d" containerName="main" containerID="cri-o://922e4fe4872cc861b177ddd823a411079c51d92b521ffa9ff7e9cbade3bc2cb1" gracePeriod=45
Aug 04 18:46:31.658314 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:31.658247    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"
Aug 04 18:46:31.715026 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:31.714972    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37f2b861-3a95-4630-9b57-47e8cc147b9d-tls-certs\") pod \"37f2b861-3a95-4630-9b57-47e8cc147b9d\" (UID: \"37f2b861-3a95-4630-9b57-47e8cc147b9d\") "
Aug 04 18:46:31.715026 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:31.715035    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37f2b861-3a95-4630-9b57-47e8cc147b9d-home\") pod \"37f2b861-3a95-4630-9b57-47e8cc147b9d\" (UID: \"37f2b861-3a95-4630-9b57-47e8cc147b9d\") "
Aug 04 18:46:31.715276 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:31.715121    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37f2b861-3a95-4630-9b57-47e8cc147b9d-tmp-dir\") pod \"37f2b861-3a95-4630-9b57-47e8cc147b9d\" (UID: \"37f2b861-3a95-4630-9b57-47e8cc147b9d\") "
Aug 04 18:46:31.715276 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:31.715160    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37f2b861-3a95-4630-9b57-47e8cc147b9d-dshm\") pod \"37f2b861-3a95-4630-9b57-47e8cc147b9d\" (UID: \"37f2b861-3a95-4630-9b57-47e8cc147b9d\") "
Aug 04 18:46:31.715276 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:31.715180    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37f2b861-3a95-4630-9b57-47e8cc147b9d-model-cache\") pod \"37f2b861-3a95-4630-9b57-47e8cc147b9d\" (UID: \"37f2b861-3a95-4630-9b57-47e8cc147b9d\") "
Aug 04 18:46:31.715276 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:31.715227    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fjz2\" (UniqueName: \"kubernetes.io/projected/37f2b861-3a95-4630-9b57-47e8cc147b9d-kube-api-access-7fjz2\") pod \"37f2b861-3a95-4630-9b57-47e8cc147b9d\" (UID: \"37f2b861-3a95-4630-9b57-47e8cc147b9d\") "
Aug 04 18:46:31.715505 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:31.715286    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37f2b861-3a95-4630-9b57-47e8cc147b9d-home" (OuterVolumeSpecName: "home") pod "37f2b861-3a95-4630-9b57-47e8cc147b9d" (UID: "37f2b861-3a95-4630-9b57-47e8cc147b9d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:46:31.715505 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:31.715457    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37f2b861-3a95-4630-9b57-47e8cc147b9d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "37f2b861-3a95-4630-9b57-47e8cc147b9d" (UID: "37f2b861-3a95-4630-9b57-47e8cc147b9d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:46:31.715505 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:31.715470    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37f2b861-3a95-4630-9b57-47e8cc147b9d-model-cache" (OuterVolumeSpecName: "model-cache") pod "37f2b861-3a95-4630-9b57-47e8cc147b9d" (UID: "37f2b861-3a95-4630-9b57-47e8cc147b9d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:46:31.715671 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:31.715600    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37f2b861-3a95-4630-9b57-47e8cc147b9d-tmp-dir\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:46:31.715671 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:31.715622    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37f2b861-3a95-4630-9b57-47e8cc147b9d-model-cache\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:46:31.715671 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:31.715636    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37f2b861-3a95-4630-9b57-47e8cc147b9d-home\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:46:31.717439 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:31.717410    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37f2b861-3a95-4630-9b57-47e8cc147b9d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "37f2b861-3a95-4630-9b57-47e8cc147b9d" (UID: "37f2b861-3a95-4630-9b57-47e8cc147b9d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:46:31.717558 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:31.717440    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37f2b861-3a95-4630-9b57-47e8cc147b9d-kube-api-access-7fjz2" (OuterVolumeSpecName: "kube-api-access-7fjz2") pod "37f2b861-3a95-4630-9b57-47e8cc147b9d" (UID: "37f2b861-3a95-4630-9b57-47e8cc147b9d"). InnerVolumeSpecName "kube-api-access-7fjz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:46:31.717558 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:31.717445    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37f2b861-3a95-4630-9b57-47e8cc147b9d-dshm" (OuterVolumeSpecName: "dshm") pod "37f2b861-3a95-4630-9b57-47e8cc147b9d" (UID: "37f2b861-3a95-4630-9b57-47e8cc147b9d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:46:31.816300 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:31.816264    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37f2b861-3a95-4630-9b57-47e8cc147b9d-dshm\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:46:31.816300 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:31.816296    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fjz2\" (UniqueName: \"kubernetes.io/projected/37f2b861-3a95-4630-9b57-47e8cc147b9d-kube-api-access-7fjz2\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:46:31.816300 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:31.816308    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37f2b861-3a95-4630-9b57-47e8cc147b9d-tls-certs\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:46:32.099224 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:32.099191    2560 generic.go:356] "Generic (PLEG): container finished" podID="37f2b861-3a95-4630-9b57-47e8cc147b9d" containerID="922e4fe4872cc861b177ddd823a411079c51d92b521ffa9ff7e9cbade3bc2cb1" exitCode=0
Aug 04 18:46:32.099411 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:32.099239    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"
Aug 04 18:46:32.099411 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:32.099243    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6" event={"ID":"37f2b861-3a95-4630-9b57-47e8cc147b9d","Type":"ContainerDied","Data":"922e4fe4872cc861b177ddd823a411079c51d92b521ffa9ff7e9cbade3bc2cb1"}
Aug 04 18:46:32.099411 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:32.099363    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6" event={"ID":"37f2b861-3a95-4630-9b57-47e8cc147b9d","Type":"ContainerDied","Data":"274a9d97366490efd376592b3fd5b3be226131883b5ac6c4944c32d478b2716e"}
Aug 04 18:46:32.099411 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:32.099390    2560 scope.go:117] "RemoveContainer" containerID="922e4fe4872cc861b177ddd823a411079c51d92b521ffa9ff7e9cbade3bc2cb1"
Aug 04 18:46:32.108754 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:32.108735    2560 scope.go:117] "RemoveContainer" containerID="922e4fe4872cc861b177ddd823a411079c51d92b521ffa9ff7e9cbade3bc2cb1"
Aug 04 18:46:32.109090 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:46:32.109071    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"922e4fe4872cc861b177ddd823a411079c51d92b521ffa9ff7e9cbade3bc2cb1\": container with ID starting with 922e4fe4872cc861b177ddd823a411079c51d92b521ffa9ff7e9cbade3bc2cb1 not found: ID does not exist" containerID="922e4fe4872cc861b177ddd823a411079c51d92b521ffa9ff7e9cbade3bc2cb1"
Aug 04 18:46:32.109164 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:32.109104    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"922e4fe4872cc861b177ddd823a411079c51d92b521ffa9ff7e9cbade3bc2cb1"} err="failed to get container status \"922e4fe4872cc861b177ddd823a411079c51d92b521ffa9ff7e9cbade3bc2cb1\": rpc error: code = NotFound desc = could not find container \"922e4fe4872cc861b177ddd823a411079c51d92b521ffa9ff7e9cbade3bc2cb1\": container with ID starting with 922e4fe4872cc861b177ddd823a411079c51d92b521ffa9ff7e9cbade3bc2cb1 not found: ID does not exist"
Aug 04 18:46:32.126585 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:32.126563    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"]
Aug 04 18:46:32.133170 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:32.133143    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5c8685/rate-test-model-5c8685-kserve-854646fd79-nr2s6"]
Aug 04 18:46:32.354161 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:32.354085    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37f2b861-3a95-4630-9b57-47e8cc147b9d" path="/var/lib/kubelet/pods/37f2b861-3a95-4630-9b57-47e8cc147b9d/volumes"
Aug 04 18:46:51.199620 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:51.199590    2560 generic.go:356] "Generic (PLEG): container finished" podID="76b5a999-8b81-45a0-9663-9daabaca87fa" containerID="e4ce1740c2fce74f1e28f58d8795dd058ab90623c8914afb8bfb857ddf5702a9" exitCode=1
Aug 04 18:46:51.200045 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:51.199665    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57c8d79cc7-97k8r" event={"ID":"76b5a999-8b81-45a0-9663-9daabaca87fa","Type":"ContainerDied","Data":"e4ce1740c2fce74f1e28f58d8795dd058ab90623c8914afb8bfb857ddf5702a9"}
Aug 04 18:46:51.200108 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:51.200048    2560 scope.go:117] "RemoveContainer" containerID="e4ce1740c2fce74f1e28f58d8795dd058ab90623c8914afb8bfb857ddf5702a9"
Aug 04 18:46:51.200426 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:51.200408    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 18:46:51.201312 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:51.201266    2560 generic.go:356] "Generic (PLEG): container finished" podID="58d7dbcb-b524-4c25-8112-ac3cb702e76c" containerID="b23b8453b8071bb25f525b3dc1f0f65f0359d9eaf51e12982fb1ee7e6ec78b9f" exitCode=1
Aug 04 18:46:51.201312 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:51.201290    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-srjp7" event={"ID":"58d7dbcb-b524-4c25-8112-ac3cb702e76c","Type":"ContainerDied","Data":"b23b8453b8071bb25f525b3dc1f0f65f0359d9eaf51e12982fb1ee7e6ec78b9f"}
Aug 04 18:46:51.201552 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:51.201539    2560 scope.go:117] "RemoveContainer" containerID="b23b8453b8071bb25f525b3dc1f0f65f0359d9eaf51e12982fb1ee7e6ec78b9f"
Aug 04 18:46:52.208232 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:52.208197    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-srjp7" event={"ID":"58d7dbcb-b524-4c25-8112-ac3cb702e76c","Type":"ContainerStarted","Data":"048b3f47b257f636bd36d1f0520a89579ab004ce5c81918a14030e505f3562e3"}
Aug 04 18:46:52.208710 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:52.208425    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-srjp7"
Aug 04 18:46:52.209743 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:52.209721    2560 generic.go:356] "Generic (PLEG): container finished" podID="b748fb50-d006-4dc4-900d-bbebd806fc17" containerID="9d1e7bc73abfe341ea34398d4b6bc47b051bee9b2f4dfa739c7178480bd88a64" exitCode=1
Aug 04 18:46:52.209853 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:52.209757    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-55bf5c9f58-7gdxs" event={"ID":"b748fb50-d006-4dc4-900d-bbebd806fc17","Type":"ContainerDied","Data":"9d1e7bc73abfe341ea34398d4b6bc47b051bee9b2f4dfa739c7178480bd88a64"}
Aug 04 18:46:52.210064 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:52.210049    2560 scope.go:117] "RemoveContainer" containerID="9d1e7bc73abfe341ea34398d4b6bc47b051bee9b2f4dfa739c7178480bd88a64"
Aug 04 18:46:52.594670 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:52.594587    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 04 18:46:53.218479 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:53.218431    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57c8d79cc7-97k8r" event={"ID":"76b5a999-8b81-45a0-9663-9daabaca87fa","Type":"ContainerStarted","Data":"b71d565235b0740c6bd074450bbc43e0d256690cb5513ca9b43cfc67fa52844a"}
Aug 04 18:46:53.218948 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:53.218504    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-57c8d79cc7-97k8r"
Aug 04 18:46:53.220778 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:53.220753    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-55bf5c9f58-7gdxs" event={"ID":"b748fb50-d006-4dc4-900d-bbebd806fc17","Type":"ContainerStarted","Data":"3ac9566c6e57212fa0f12d31c04e04405e2a77e4269de4d07e9fa7c4c5f174a2"}
Aug 04 18:46:53.221062 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:53.221043    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-55bf5c9f58-7gdxs"
Aug 04 18:46:58.581077 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:58.580956    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/2.log"
Aug 04 18:46:58.594304 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:58.587098    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 18:46:58.594304 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:58.590850    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/2.log"
Aug 04 18:46:58.597261 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:46:58.597240    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 18:47:00.255638 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:00.255604    2560 generic.go:356] "Generic (PLEG): container finished" podID="58d7dbcb-b524-4c25-8112-ac3cb702e76c" containerID="048b3f47b257f636bd36d1f0520a89579ab004ce5c81918a14030e505f3562e3" exitCode=1
Aug 04 18:47:00.256105 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:00.255679    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-srjp7" event={"ID":"58d7dbcb-b524-4c25-8112-ac3cb702e76c","Type":"ContainerDied","Data":"048b3f47b257f636bd36d1f0520a89579ab004ce5c81918a14030e505f3562e3"}
Aug 04 18:47:00.256105 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:00.255770    2560 scope.go:117] "RemoveContainer" containerID="b23b8453b8071bb25f525b3dc1f0f65f0359d9eaf51e12982fb1ee7e6ec78b9f"
Aug 04 18:47:00.256213 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:00.256200    2560 scope.go:117] "RemoveContainer" containerID="048b3f47b257f636bd36d1f0520a89579ab004ce5c81918a14030e505f3562e3"
Aug 04 18:47:00.256431 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:47:00.256407    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=kserve-controller-manager-d6f4b65b8-srjp7_opendatahub(58d7dbcb-b524-4c25-8112-ac3cb702e76c)\"" pod="opendatahub/kserve-controller-manager-d6f4b65b8-srjp7" podUID="58d7dbcb-b524-4c25-8112-ac3cb702e76c"
Aug 04 18:47:04.227872 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:04.227839    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-57c8d79cc7-97k8r"
Aug 04 18:47:04.228311 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:04.227945    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-55bf5c9f58-7gdxs"
Aug 04 18:47:07.031507 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:07.031457    2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/kserve-controller-manager-d6f4b65b8-srjp7"
Aug 04 18:47:07.031929 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:07.031820    2560 scope.go:117] "RemoveContainer" containerID="048b3f47b257f636bd36d1f0520a89579ab004ce5c81918a14030e505f3562e3"
Aug 04 18:47:07.032060 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:47:07.032038    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=kserve-controller-manager-d6f4b65b8-srjp7_opendatahub(58d7dbcb-b524-4c25-8112-ac3cb702e76c)\"" pod="opendatahub/kserve-controller-manager-d6f4b65b8-srjp7" podUID="58d7dbcb-b524-4c25-8112-ac3cb702e76c"
Aug 04 18:47:18.349838 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:18.349801    2560 scope.go:117] "RemoveContainer" containerID="048b3f47b257f636bd36d1f0520a89579ab004ce5c81918a14030e505f3562e3"
Aug 04 18:47:19.364658 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:19.364579    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-srjp7" event={"ID":"58d7dbcb-b524-4c25-8112-ac3cb702e76c","Type":"ContainerStarted","Data":"6cd52197f91e67414987ad9c349e7a972488aa2f5c2adf28f5cd09219b813c7e"}
Aug 04 18:47:19.365040 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:19.364862    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-srjp7"
Aug 04 18:47:27.331420 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:27.331390    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr"]
Aug 04 18:47:27.332012 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:27.331965    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37f2b861-3a95-4630-9b57-47e8cc147b9d" containerName="main"
Aug 04 18:47:27.332012 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:27.332007    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="37f2b861-3a95-4630-9b57-47e8cc147b9d" containerName="main"
Aug 04 18:47:27.332128 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:27.332115    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="37f2b861-3a95-4630-9b57-47e8cc147b9d" containerName="main"
Aug 04 18:47:27.335393 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:27.335375    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr"
Aug 04 18:47:27.347939 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:27.347920    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-6a6d9f-serving-cert\""
Aug 04 18:47:27.361166 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:27.361140    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr"]
Aug 04 18:47:27.432756 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:27.432720    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfb66d6b-b813-4b58-ae53-326f08bd3878-maas-api-tls\") pod \"maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr\" (UID: \"bfb66d6b-b813-4b58-ae53-326f08bd3878\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr"
Aug 04 18:47:27.432921 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:27.432760    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs7z8\" (UniqueName: \"kubernetes.io/projected/bfb66d6b-b813-4b58-ae53-326f08bd3878-kube-api-access-hs7z8\") pod \"maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr\" (UID: \"bfb66d6b-b813-4b58-ae53-326f08bd3878\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr"
Aug 04 18:47:27.432921 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:27.432854    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfb66d6b-b813-4b58-ae53-326f08bd3878-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr\" (UID: \"bfb66d6b-b813-4b58-ae53-326f08bd3878\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr"
Aug 04 18:47:27.534168 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:27.534137    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfb66d6b-b813-4b58-ae53-326f08bd3878-maas-api-tls\") pod \"maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr\" (UID: \"bfb66d6b-b813-4b58-ae53-326f08bd3878\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr"
Aug 04 18:47:27.534324 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:27.534177    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs7z8\" (UniqueName: \"kubernetes.io/projected/bfb66d6b-b813-4b58-ae53-326f08bd3878-kube-api-access-hs7z8\") pod \"maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr\" (UID: \"bfb66d6b-b813-4b58-ae53-326f08bd3878\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr"
Aug 04 18:47:27.534324 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:27.534253    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfb66d6b-b813-4b58-ae53-326f08bd3878-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr\" (UID: \"bfb66d6b-b813-4b58-ae53-326f08bd3878\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr"
Aug 04 18:47:27.534843 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:27.534820    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfb66d6b-b813-4b58-ae53-326f08bd3878-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr\" (UID: \"bfb66d6b-b813-4b58-ae53-326f08bd3878\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr"
Aug 04 18:47:27.536541 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:27.536517    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfb66d6b-b813-4b58-ae53-326f08bd3878-maas-api-tls\") pod \"maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr\" (UID: \"bfb66d6b-b813-4b58-ae53-326f08bd3878\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr"
Aug 04 18:47:27.547344 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:27.547322    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs7z8\" (UniqueName: \"kubernetes.io/projected/bfb66d6b-b813-4b58-ae53-326f08bd3878-kube-api-access-hs7z8\") pod \"maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr\" (UID: \"bfb66d6b-b813-4b58-ae53-326f08bd3878\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr"
Aug 04 18:47:27.648783 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:27.648702    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr"
Aug 04 18:47:27.797236 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:27.797203    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr"]
Aug 04 18:47:27.800405 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:47:27.800366    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfb66d6b_b813_4b58_ae53_326f08bd3878.slice/crio-ce6312930dcda55b17a08e474e70344f3a378004869eecce8a18fbbdeee17ca4 WatchSource:0}: Error finding container ce6312930dcda55b17a08e474e70344f3a378004869eecce8a18fbbdeee17ca4: Status 404 returned error can't find the container with id ce6312930dcda55b17a08e474e70344f3a378004869eecce8a18fbbdeee17ca4
Aug 04 18:47:28.414870 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:28.414835    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr" event={"ID":"bfb66d6b-b813-4b58-ae53-326f08bd3878","Type":"ContainerStarted","Data":"ce6312930dcda55b17a08e474e70344f3a378004869eecce8a18fbbdeee17ca4"}
Aug 04 18:47:28.649876 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:28.649832    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r"]
Aug 04 18:47:28.655228 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:28.655205    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r"
Aug 04 18:47:28.658794 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:28.658775    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-6a6d9f-dockercfg-6ptkj\""
Aug 04 18:47:28.675455 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:28.675405    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-6a6d9f\""
Aug 04 18:47:28.683129 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:28.683107    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r"]
Aug 04 18:47:28.740199 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:28.740166    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8"]
Aug 04 18:47:28.744017 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:28.743982    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8"
Aug 04 18:47:28.745435 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:28.745405    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ff9e4ac-14b4-400d-855b-4804b76c43e2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r\" (UID: \"7ff9e4ac-14b4-400d-855b-4804b76c43e2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r"
Aug 04 18:47:28.745525 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:28.745490    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j4tp\" (UniqueName: \"kubernetes.io/projected/7ff9e4ac-14b4-400d-855b-4804b76c43e2-kube-api-access-2j4tp\") pod \"payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r\" (UID: \"7ff9e4ac-14b4-400d-855b-4804b76c43e2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r"
Aug 04 18:47:28.773112 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:28.773086    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8"]
Aug 04 18:47:28.846096 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:28.846062    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19ad9e9e-84ef-4724-8492-1bbd185b9eab-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8\" (UID: \"19ad9e9e-84ef-4724-8492-1bbd185b9eab\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8"
Aug 04 18:47:28.846306 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:28.846113    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ff9e4ac-14b4-400d-855b-4804b76c43e2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r\" (UID: \"7ff9e4ac-14b4-400d-855b-4804b76c43e2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r"
Aug 04 18:47:28.846306 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:28.846139    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xxqn\" (UniqueName: \"kubernetes.io/projected/19ad9e9e-84ef-4724-8492-1bbd185b9eab-kube-api-access-4xxqn\") pod \"payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8\" (UID: \"19ad9e9e-84ef-4724-8492-1bbd185b9eab\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8"
Aug 04 18:47:28.846306 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:28.846280    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j4tp\" (UniqueName: \"kubernetes.io/projected/7ff9e4ac-14b4-400d-855b-4804b76c43e2-kube-api-access-2j4tp\") pod \"payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r\" (UID: \"7ff9e4ac-14b4-400d-855b-4804b76c43e2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r"
Aug 04 18:47:28.846821 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:28.846800    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ff9e4ac-14b4-400d-855b-4804b76c43e2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r\" (UID: \"7ff9e4ac-14b4-400d-855b-4804b76c43e2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r"
Aug 04 18:47:28.862469 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:28.862433    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j4tp\" (UniqueName: \"kubernetes.io/projected/7ff9e4ac-14b4-400d-855b-4804b76c43e2-kube-api-access-2j4tp\") pod \"payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r\" (UID: \"7ff9e4ac-14b4-400d-855b-4804b76c43e2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r"
Aug 04 18:47:28.947439 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:28.947357    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19ad9e9e-84ef-4724-8492-1bbd185b9eab-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8\" (UID: \"19ad9e9e-84ef-4724-8492-1bbd185b9eab\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8"
Aug 04 18:47:28.947439 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:28.947404    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xxqn\" (UniqueName: \"kubernetes.io/projected/19ad9e9e-84ef-4724-8492-1bbd185b9eab-kube-api-access-4xxqn\") pod \"payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8\" (UID: \"19ad9e9e-84ef-4724-8492-1bbd185b9eab\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8"
Aug 04 18:47:28.948100 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:28.948082    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19ad9e9e-84ef-4724-8492-1bbd185b9eab-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8\" (UID: \"19ad9e9e-84ef-4724-8492-1bbd185b9eab\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8"
Aug 04 18:47:28.958642 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:28.958613    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xxqn\" (UniqueName: \"kubernetes.io/projected/19ad9e9e-84ef-4724-8492-1bbd185b9eab-kube-api-access-4xxqn\") pod \"payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8\" (UID: \"19ad9e9e-84ef-4724-8492-1bbd185b9eab\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8"
Aug 04 18:47:28.966167 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:28.966144    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r"
Aug 04 18:47:29.054519 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:29.054474    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8"
Aug 04 18:47:29.421545 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:29.421507    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr" event={"ID":"bfb66d6b-b813-4b58-ae53-326f08bd3878","Type":"ContainerStarted","Data":"1cc21b92435cee3fea0439ac36b01f00b3c71147aa6306365c97da7904accdf7"}
Aug 04 18:47:29.422055 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:29.421626    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr"
Aug 04 18:47:30.615340 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:47:30.615310    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ff9e4ac_14b4_400d_855b_4804b76c43e2.slice/crio-a1f89deaea74c54046cdc8c09af6e5aace8afdc95a2110c0b10176e0b7e3689b WatchSource:0}: Error finding container a1f89deaea74c54046cdc8c09af6e5aace8afdc95a2110c0b10176e0b7e3689b: Status 404 returned error can't find the container with id a1f89deaea74c54046cdc8c09af6e5aace8afdc95a2110c0b10176e0b7e3689b
Aug 04 18:47:30.629641 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:30.629621    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8"]
Aug 04 18:47:30.630290 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:47:30.630268    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19ad9e9e_84ef_4724_8492_1bbd185b9eab.slice/crio-351db590b8215a0e1aeeb3da96a3206aa3767ab59ee4fc4519958f30edee9d77 WatchSource:0}: Error finding container 351db590b8215a0e1aeeb3da96a3206aa3767ab59ee4fc4519958f30edee9d77: Status 404 returned error can't find the container with id 351db590b8215a0e1aeeb3da96a3206aa3767ab59ee4fc4519958f30edee9d77
Aug 04 18:47:30.635154 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:30.633382    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r"]
Aug 04 18:47:30.650142 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:30.650103    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr" podStartSLOduration=2.986415523 podStartE2EDuration="3.650085203s" totalImagesPullingTime="663.66968ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:47:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:47:30.649605445 +0000 UTC m=+3332.832765622" watchObservedRunningTime="2026-08-04 18:47:30.650085203 +0000 UTC m=+3332.833245382"
Aug 04 18:47:31.433428 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:31.433386    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8" event={"ID":"19ad9e9e-84ef-4724-8492-1bbd185b9eab","Type":"ContainerStarted","Data":"2420f4bf9f09f868d0b5dac43610c9ac387acc168025855c5bc03fae237160d2"}
Aug 04 18:47:31.433634 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:31.433462    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8" event={"ID":"19ad9e9e-84ef-4724-8492-1bbd185b9eab","Type":"ContainerStarted","Data":"351db590b8215a0e1aeeb3da96a3206aa3767ab59ee4fc4519958f30edee9d77"}
Aug 04 18:47:31.433733 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:31.433633    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8"
Aug 04 18:47:31.435098 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:31.435069    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r" event={"ID":"7ff9e4ac-14b4-400d-855b-4804b76c43e2","Type":"ContainerStarted","Data":"04e3882a53cf8c17e0883c3b7464b1d5cda4b610a04e1a733f67bc116f5a6350"}
Aug 04 18:47:31.435206 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:31.435107    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r" event={"ID":"7ff9e4ac-14b4-400d-855b-4804b76c43e2","Type":"ContainerStarted","Data":"a1f89deaea74c54046cdc8c09af6e5aace8afdc95a2110c0b10176e0b7e3689b"}
Aug 04 18:47:31.435206 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:31.435150    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r"
Aug 04 18:47:31.457211 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:31.457159    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8" podStartSLOduration=3.457142493 podStartE2EDuration="3.457142493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:47:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:47:31.456918454 +0000 UTC m=+3333.640078630" watchObservedRunningTime="2026-08-04 18:47:31.457142493 +0000 UTC m=+3333.640302651"
Aug 04 18:47:31.490354 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:31.490319    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r" podStartSLOduration=3.490305383 podStartE2EDuration="3.490305383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:47:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:47:31.488943 +0000 UTC m=+3333.672103294" watchObservedRunningTime="2026-08-04 18:47:31.490305383 +0000 UTC m=+3333.673465560"
Aug 04 18:47:35.432248 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:35.432220    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr"
Aug 04 18:47:42.441972 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:42.441933    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8"
Aug 04 18:47:42.442459 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:42.442293    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r"
Aug 04 18:47:50.378477 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:50.378449    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-srjp7"
Aug 04 18:47:55.015049 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:55.015017    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz"]
Aug 04 18:47:55.018812 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:55.018791    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz"
Aug 04 18:47:55.024104 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:55.024081    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-071def-dockercfg-xb267\""
Aug 04 18:47:55.024217 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:55.024117    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-071def\""
Aug 04 18:47:55.065229 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:55.065196    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz"]
Aug 04 18:47:55.068259 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:55.068228    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8"]
Aug 04 18:47:55.072649 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:55.072624    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8"
Aug 04 18:47:55.091168 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:55.091119    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d547860-b313-4977-a2d2-c05253219539-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8\" (UID: \"9d547860-b313-4977-a2d2-c05253219539\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8"
Aug 04 18:47:55.091333 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:55.091201    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsghj\" (UniqueName: \"kubernetes.io/projected/6bb7a6fe-b7c7-4676-b0ce-87c1137cae07-kube-api-access-dsghj\") pod \"payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz\" (UID: \"6bb7a6fe-b7c7-4676-b0ce-87c1137cae07\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz"
Aug 04 18:47:55.091333 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:55.091309    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbfbl\" (UniqueName: \"kubernetes.io/projected/9d547860-b313-4977-a2d2-c05253219539-kube-api-access-zbfbl\") pod \"payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8\" (UID: \"9d547860-b313-4977-a2d2-c05253219539\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8"
Aug 04 18:47:55.091420 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:55.091362    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bb7a6fe-b7c7-4676-b0ce-87c1137cae07-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz\" (UID: \"6bb7a6fe-b7c7-4676-b0ce-87c1137cae07\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz"
Aug 04 18:47:55.101203 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:55.101167    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8"]
Aug 04 18:47:55.192374 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:55.192337    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d547860-b313-4977-a2d2-c05253219539-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8\" (UID: \"9d547860-b313-4977-a2d2-c05253219539\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8"
Aug 04 18:47:55.192563 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:55.192383    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsghj\" (UniqueName: \"kubernetes.io/projected/6bb7a6fe-b7c7-4676-b0ce-87c1137cae07-kube-api-access-dsghj\") pod \"payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz\" (UID: \"6bb7a6fe-b7c7-4676-b0ce-87c1137cae07\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz"
Aug 04 18:47:55.192563 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:55.192441    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbfbl\" (UniqueName: \"kubernetes.io/projected/9d547860-b313-4977-a2d2-c05253219539-kube-api-access-zbfbl\") pod \"payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8\" (UID: \"9d547860-b313-4977-a2d2-c05253219539\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8"
Aug 04 18:47:55.192563 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:55.192486    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bb7a6fe-b7c7-4676-b0ce-87c1137cae07-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz\" (UID: \"6bb7a6fe-b7c7-4676-b0ce-87c1137cae07\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz"
Aug 04 18:47:55.193154 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:55.193136    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d547860-b313-4977-a2d2-c05253219539-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8\" (UID: \"9d547860-b313-4977-a2d2-c05253219539\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8"
Aug 04 18:47:55.193240 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:55.193223    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bb7a6fe-b7c7-4676-b0ce-87c1137cae07-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz\" (UID: \"6bb7a6fe-b7c7-4676-b0ce-87c1137cae07\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz"
Aug 04 18:47:55.205706 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:55.205673    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsghj\" (UniqueName: \"kubernetes.io/projected/6bb7a6fe-b7c7-4676-b0ce-87c1137cae07-kube-api-access-dsghj\") pod \"payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz\" (UID: \"6bb7a6fe-b7c7-4676-b0ce-87c1137cae07\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz"
Aug 04 18:47:55.209916 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:55.209887    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbfbl\" (UniqueName: \"kubernetes.io/projected/9d547860-b313-4977-a2d2-c05253219539-kube-api-access-zbfbl\") pod \"payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8\" (UID: \"9d547860-b313-4977-a2d2-c05253219539\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8"
Aug 04 18:47:55.329747 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:55.329652    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz"
Aug 04 18:47:55.383650 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:55.383585    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8"
Aug 04 18:47:55.494683 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:55.494637    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz"]
Aug 04 18:47:55.560614 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:55.560583    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8"]
Aug 04 18:47:55.565111 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:47:55.565078    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d547860_b313_4977_a2d2_c05253219539.slice/crio-46583a205df1230ae9df98828af118cc34165875d5abaa58be9ce2ef4d113dca WatchSource:0}: Error finding container 46583a205df1230ae9df98828af118cc34165875d5abaa58be9ce2ef4d113dca: Status 404 returned error can't find the container with id 46583a205df1230ae9df98828af118cc34165875d5abaa58be9ce2ef4d113dca
Aug 04 18:47:55.569801 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:55.569772    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz" event={"ID":"6bb7a6fe-b7c7-4676-b0ce-87c1137cae07","Type":"ContainerStarted","Data":"e3bbd1d0cb9754d2a7789277a76be002b53ffd7ff132e5cb7c6ebf69b6aaf50d"}
Aug 04 18:47:56.576757 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:56.576720    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz" event={"ID":"6bb7a6fe-b7c7-4676-b0ce-87c1137cae07","Type":"ContainerStarted","Data":"1a5efe98eaee3a613e2cdd646fef1f5643451a2086f1245db0b2f7b43afd0184"}
Aug 04 18:47:56.577174 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:56.576788    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz"
Aug 04 18:47:56.578310 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:56.578285    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8" event={"ID":"9d547860-b313-4977-a2d2-c05253219539","Type":"ContainerStarted","Data":"3f8f2a36245cf8d6be2222f143aa6fa6de6f31f0b5f1458e159f67bfab6fd7d9"}
Aug 04 18:47:56.578401 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:56.578326    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8" event={"ID":"9d547860-b313-4977-a2d2-c05253219539","Type":"ContainerStarted","Data":"46583a205df1230ae9df98828af118cc34165875d5abaa58be9ce2ef4d113dca"}
Aug 04 18:47:56.578401 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:56.578346    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8"
Aug 04 18:47:56.597334 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:56.597290    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz" podStartSLOduration=2.59727752 podStartE2EDuration="2.59727752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:47:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:47:56.594827294 +0000 UTC m=+3358.777987473" watchObservedRunningTime="2026-08-04 18:47:56.59727752 +0000 UTC m=+3358.780437699"
Aug 04 18:47:56.619533 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:47:56.619475    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8" podStartSLOduration=1.6194546060000001 podStartE2EDuration="1.619454606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:47:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:47:56.615640088 +0000 UTC m=+3358.798800263" watchObservedRunningTime="2026-08-04 18:47:56.619454606 +0000 UTC m=+3358.802614783"
Aug 04 18:48:07.585496 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:48:07.585465    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz"
Aug 04 18:48:07.586105 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:48:07.585860    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8"
Aug 04 18:48:17.624565 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:48:17.624535    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-6wqzc_12873b28-6874-43e7-b589-16b8f7f7fef4/payload-processing/0.log"
Aug 04 18:48:17.754368 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:48:17.754321    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8_9d547860-b313-4977-a2d2-c05253219539/payload-processing/0.log"
Aug 04 18:49:12.989722 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:12.989693    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r_7ff9e4ac-14b4-400d-855b-4804b76c43e2/payload-processing/0.log"
Aug 04 18:49:13.157116 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:13.157083    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-6wqzc_12873b28-6874-43e7-b589-16b8f7f7fef4/payload-processing/0.log"
Aug 04 18:49:13.281643 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:13.281571    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8_9d547860-b313-4977-a2d2-c05253219539/payload-processing/0.log"
Aug 04 18:49:21.809807 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:21.809774    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq"]
Aug 04 18:49:21.813647 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:21.813630    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq"
Aug 04 18:49:21.816875 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:21.816853    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-f2f60a-dockercfg-dqt7d\""
Aug 04 18:49:21.817031 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:21.816920    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-f2f60a\""
Aug 04 18:49:21.826779 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:21.826757    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq"]
Aug 04 18:49:21.908369 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:21.908337    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d095b18a-feca-4015-a4c4-689e24253d8d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq\" (UID: \"d095b18a-feca-4015-a4c4-689e24253d8d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq"
Aug 04 18:49:21.908525 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:21.908466    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-685t8\" (UniqueName: \"kubernetes.io/projected/d095b18a-feca-4015-a4c4-689e24253d8d-kube-api-access-685t8\") pod \"payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq\" (UID: \"d095b18a-feca-4015-a4c4-689e24253d8d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq"
Aug 04 18:49:21.943822 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:21.943793    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v"]
Aug 04 18:49:21.947479 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:21.947464    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v"
Aug 04 18:49:21.962204 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:21.962176    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v"]
Aug 04 18:49:22.009811 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:22.009768    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d095b18a-feca-4015-a4c4-689e24253d8d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq\" (UID: \"d095b18a-feca-4015-a4c4-689e24253d8d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq"
Aug 04 18:49:22.010024 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:22.009868    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ab0c0c-3753-4181-9085-bed884dac850-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v\" (UID: \"69ab0c0c-3753-4181-9085-bed884dac850\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v"
Aug 04 18:49:22.010024 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:22.009943    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-685t8\" (UniqueName: \"kubernetes.io/projected/d095b18a-feca-4015-a4c4-689e24253d8d-kube-api-access-685t8\") pod \"payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq\" (UID: \"d095b18a-feca-4015-a4c4-689e24253d8d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq"
Aug 04 18:49:22.010163 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:22.010044    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhhgw\" (UniqueName: \"kubernetes.io/projected/69ab0c0c-3753-4181-9085-bed884dac850-kube-api-access-lhhgw\") pod \"payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v\" (UID: \"69ab0c0c-3753-4181-9085-bed884dac850\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v"
Aug 04 18:49:22.010575 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:22.010550    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d095b18a-feca-4015-a4c4-689e24253d8d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq\" (UID: \"d095b18a-feca-4015-a4c4-689e24253d8d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq"
Aug 04 18:49:22.020122 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:22.020092    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-685t8\" (UniqueName: \"kubernetes.io/projected/d095b18a-feca-4015-a4c4-689e24253d8d-kube-api-access-685t8\") pod \"payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq\" (UID: \"d095b18a-feca-4015-a4c4-689e24253d8d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq"
Aug 04 18:49:22.111568 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:22.111502    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ab0c0c-3753-4181-9085-bed884dac850-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v\" (UID: \"69ab0c0c-3753-4181-9085-bed884dac850\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v"
Aug 04 18:49:22.111688 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:22.111621    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhhgw\" (UniqueName: \"kubernetes.io/projected/69ab0c0c-3753-4181-9085-bed884dac850-kube-api-access-lhhgw\") pod \"payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v\" (UID: \"69ab0c0c-3753-4181-9085-bed884dac850\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v"
Aug 04 18:49:22.112230 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:22.112211    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ab0c0c-3753-4181-9085-bed884dac850-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v\" (UID: \"69ab0c0c-3753-4181-9085-bed884dac850\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v"
Aug 04 18:49:22.120798 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:22.120773    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhhgw\" (UniqueName: \"kubernetes.io/projected/69ab0c0c-3753-4181-9085-bed884dac850-kube-api-access-lhhgw\") pod \"payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v\" (UID: \"69ab0c0c-3753-4181-9085-bed884dac850\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v"
Aug 04 18:49:22.124626 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:22.124599    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq"
Aug 04 18:49:22.257643 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:22.257616    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v"
Aug 04 18:49:22.266471 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:22.266449    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq"]
Aug 04 18:49:22.269027 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:49:22.268982    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd095b18a_feca_4015_a4c4_689e24253d8d.slice/crio-5e3ef1f78aa990fd78311a59effa3d9e2ea6ebac734c7865945c872f63952372 WatchSource:0}: Error finding container 5e3ef1f78aa990fd78311a59effa3d9e2ea6ebac734c7865945c872f63952372: Status 404 returned error can't find the container with id 5e3ef1f78aa990fd78311a59effa3d9e2ea6ebac734c7865945c872f63952372
Aug 04 18:49:22.620134 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:22.620108    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v"]
Aug 04 18:49:22.622445 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:49:22.622415    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69ab0c0c_3753_4181_9085_bed884dac850.slice/crio-c6b55c00152f0906f45508fda74f5ac39e40c09ddb3b9ef85488c5e4744171ee WatchSource:0}: Error finding container c6b55c00152f0906f45508fda74f5ac39e40c09ddb3b9ef85488c5e4744171ee: Status 404 returned error can't find the container with id c6b55c00152f0906f45508fda74f5ac39e40c09ddb3b9ef85488c5e4744171ee
Aug 04 18:49:23.041803 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:23.041769    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq" event={"ID":"d095b18a-feca-4015-a4c4-689e24253d8d","Type":"ContainerStarted","Data":"ae694947bb133201d485066387bf7e4e9d6523c27086559b45759ecc58a03c6b"}
Aug 04 18:49:23.042280 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:23.041832    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq"
Aug 04 18:49:23.042280 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:23.041859    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq" event={"ID":"d095b18a-feca-4015-a4c4-689e24253d8d","Type":"ContainerStarted","Data":"5e3ef1f78aa990fd78311a59effa3d9e2ea6ebac734c7865945c872f63952372"}
Aug 04 18:49:23.043413 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:23.043392    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v" event={"ID":"69ab0c0c-3753-4181-9085-bed884dac850","Type":"ContainerStarted","Data":"4c8dc833d2a94adf1ae269782de048da0308fed9d4841c45ed758d579a5a24cb"}
Aug 04 18:49:23.043521 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:23.043428    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v" event={"ID":"69ab0c0c-3753-4181-9085-bed884dac850","Type":"ContainerStarted","Data":"c6b55c00152f0906f45508fda74f5ac39e40c09ddb3b9ef85488c5e4744171ee"}
Aug 04 18:49:23.043521 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:23.043455    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v"
Aug 04 18:49:23.069352 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:23.069319    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq" podStartSLOduration=2.069306922 podStartE2EDuration="2.069306922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:49:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:49:23.067720849 +0000 UTC m=+3445.250881026" watchObservedRunningTime="2026-08-04 18:49:23.069306922 +0000 UTC m=+3445.252467097"
Aug 04 18:49:23.099659 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:23.099613    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v" podStartSLOduration=2.099599414 podStartE2EDuration="2.099599414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:49:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:49:23.097287762 +0000 UTC m=+3445.280447951" watchObservedRunningTime="2026-08-04 18:49:23.099599414 +0000 UTC m=+3445.282759649"
Aug 04 18:49:34.051642 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:34.051611    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v"
Aug 04 18:49:34.052039 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:34.051677    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq"
Aug 04 18:49:49.349246 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:49.349212    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v"]
Aug 04 18:49:49.349637 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:49.349434    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v" podUID="69ab0c0c-3753-4181-9085-bed884dac850" containerName="payload-processing" containerID="cri-o://4c8dc833d2a94adf1ae269782de048da0308fed9d4841c45ed758d579a5a24cb" gracePeriod=30
Aug 04 18:49:49.359154 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:49.356979    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq"]
Aug 04 18:49:49.359154 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:49.357278    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq" podUID="d095b18a-feca-4015-a4c4-689e24253d8d" containerName="payload-pre-processing" containerID="cri-o://ae694947bb133201d485066387bf7e4e9d6523c27086559b45759ecc58a03c6b" gracePeriod=30
Aug 04 18:49:49.625285 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:49.625262    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v"
Aug 04 18:49:49.654497 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:49.654475    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq"
Aug 04 18:49:49.766328 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:49.766301    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d095b18a-feca-4015-a4c4-689e24253d8d-plugins-config-volume\") pod \"d095b18a-feca-4015-a4c4-689e24253d8d\" (UID: \"d095b18a-feca-4015-a4c4-689e24253d8d\") "
Aug 04 18:49:49.766481 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:49.766392    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ab0c0c-3753-4181-9085-bed884dac850-plugins-config-volume\") pod \"69ab0c0c-3753-4181-9085-bed884dac850\" (UID: \"69ab0c0c-3753-4181-9085-bed884dac850\") "
Aug 04 18:49:49.766481 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:49.766439    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-685t8\" (UniqueName: \"kubernetes.io/projected/d095b18a-feca-4015-a4c4-689e24253d8d-kube-api-access-685t8\") pod \"d095b18a-feca-4015-a4c4-689e24253d8d\" (UID: \"d095b18a-feca-4015-a4c4-689e24253d8d\") "
Aug 04 18:49:49.766481 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:49.766454    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhhgw\" (UniqueName: \"kubernetes.io/projected/69ab0c0c-3753-4181-9085-bed884dac850-kube-api-access-lhhgw\") pod \"69ab0c0c-3753-4181-9085-bed884dac850\" (UID: \"69ab0c0c-3753-4181-9085-bed884dac850\") "
Aug 04 18:49:49.766748 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:49.766719    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d095b18a-feca-4015-a4c4-689e24253d8d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d095b18a-feca-4015-a4c4-689e24253d8d" (UID: "d095b18a-feca-4015-a4c4-689e24253d8d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:49:49.766836 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:49.766746    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69ab0c0c-3753-4181-9085-bed884dac850-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "69ab0c0c-3753-4181-9085-bed884dac850" (UID: "69ab0c0c-3753-4181-9085-bed884dac850"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:49:49.768571 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:49.768549    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69ab0c0c-3753-4181-9085-bed884dac850-kube-api-access-lhhgw" (OuterVolumeSpecName: "kube-api-access-lhhgw") pod "69ab0c0c-3753-4181-9085-bed884dac850" (UID: "69ab0c0c-3753-4181-9085-bed884dac850"). InnerVolumeSpecName "kube-api-access-lhhgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:49:49.768701 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:49.768683    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d095b18a-feca-4015-a4c4-689e24253d8d-kube-api-access-685t8" (OuterVolumeSpecName: "kube-api-access-685t8") pod "d095b18a-feca-4015-a4c4-689e24253d8d" (UID: "d095b18a-feca-4015-a4c4-689e24253d8d"). InnerVolumeSpecName "kube-api-access-685t8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:49:49.867825 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:49.867797    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-685t8\" (UniqueName: \"kubernetes.io/projected/d095b18a-feca-4015-a4c4-689e24253d8d-kube-api-access-685t8\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:49:49.867825 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:49.867821    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhhgw\" (UniqueName: \"kubernetes.io/projected/69ab0c0c-3753-4181-9085-bed884dac850-kube-api-access-lhhgw\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:49:49.867825 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:49.867830    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d095b18a-feca-4015-a4c4-689e24253d8d-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:49:49.868090 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:49.867842    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ab0c0c-3753-4181-9085-bed884dac850-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:49:50.195003 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:50.194964    2560 generic.go:356] "Generic (PLEG): container finished" podID="d095b18a-feca-4015-a4c4-689e24253d8d" containerID="ae694947bb133201d485066387bf7e4e9d6523c27086559b45759ecc58a03c6b" exitCode=0
Aug 04 18:49:50.195170 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:50.195026    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq"
Aug 04 18:49:50.195170 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:50.195021    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq" event={"ID":"d095b18a-feca-4015-a4c4-689e24253d8d","Type":"ContainerDied","Data":"ae694947bb133201d485066387bf7e4e9d6523c27086559b45759ecc58a03c6b"}
Aug 04 18:49:50.195170 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:50.195076    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq" event={"ID":"d095b18a-feca-4015-a4c4-689e24253d8d","Type":"ContainerDied","Data":"5e3ef1f78aa990fd78311a59effa3d9e2ea6ebac734c7865945c872f63952372"}
Aug 04 18:49:50.195170 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:50.195100    2560 scope.go:117] "RemoveContainer" containerID="ae694947bb133201d485066387bf7e4e9d6523c27086559b45759ecc58a03c6b"
Aug 04 18:49:50.196632 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:50.196599    2560 generic.go:356] "Generic (PLEG): container finished" podID="69ab0c0c-3753-4181-9085-bed884dac850" containerID="4c8dc833d2a94adf1ae269782de048da0308fed9d4841c45ed758d579a5a24cb" exitCode=0
Aug 04 18:49:50.196755 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:50.196650    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v"
Aug 04 18:49:50.196755 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:50.196713    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v" event={"ID":"69ab0c0c-3753-4181-9085-bed884dac850","Type":"ContainerDied","Data":"4c8dc833d2a94adf1ae269782de048da0308fed9d4841c45ed758d579a5a24cb"}
Aug 04 18:49:50.196865 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:50.196775    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v" event={"ID":"69ab0c0c-3753-4181-9085-bed884dac850","Type":"ContainerDied","Data":"c6b55c00152f0906f45508fda74f5ac39e40c09ddb3b9ef85488c5e4744171ee"}
Aug 04 18:49:50.204232 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:50.204209    2560 scope.go:117] "RemoveContainer" containerID="ae694947bb133201d485066387bf7e4e9d6523c27086559b45759ecc58a03c6b"
Aug 04 18:49:50.204519 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:49:50.204496    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae694947bb133201d485066387bf7e4e9d6523c27086559b45759ecc58a03c6b\": container with ID starting with ae694947bb133201d485066387bf7e4e9d6523c27086559b45759ecc58a03c6b not found: ID does not exist" containerID="ae694947bb133201d485066387bf7e4e9d6523c27086559b45759ecc58a03c6b"
Aug 04 18:49:50.204633 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:50.204525    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae694947bb133201d485066387bf7e4e9d6523c27086559b45759ecc58a03c6b"} err="failed to get container status \"ae694947bb133201d485066387bf7e4e9d6523c27086559b45759ecc58a03c6b\": rpc error: code = NotFound desc = could not find container \"ae694947bb133201d485066387bf7e4e9d6523c27086559b45759ecc58a03c6b\": container with ID starting with ae694947bb133201d485066387bf7e4e9d6523c27086559b45759ecc58a03c6b not found: ID does not exist"
Aug 04 18:49:50.204633 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:50.204541    2560 scope.go:117] "RemoveContainer" containerID="4c8dc833d2a94adf1ae269782de048da0308fed9d4841c45ed758d579a5a24cb"
Aug 04 18:49:50.213155 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:50.213140    2560 scope.go:117] "RemoveContainer" containerID="4c8dc833d2a94adf1ae269782de048da0308fed9d4841c45ed758d579a5a24cb"
Aug 04 18:49:50.213383 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:49:50.213366    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c8dc833d2a94adf1ae269782de048da0308fed9d4841c45ed758d579a5a24cb\": container with ID starting with 4c8dc833d2a94adf1ae269782de048da0308fed9d4841c45ed758d579a5a24cb not found: ID does not exist" containerID="4c8dc833d2a94adf1ae269782de048da0308fed9d4841c45ed758d579a5a24cb"
Aug 04 18:49:50.213432 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:50.213391    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c8dc833d2a94adf1ae269782de048da0308fed9d4841c45ed758d579a5a24cb"} err="failed to get container status \"4c8dc833d2a94adf1ae269782de048da0308fed9d4841c45ed758d579a5a24cb\": rpc error: code = NotFound desc = could not find container \"4c8dc833d2a94adf1ae269782de048da0308fed9d4841c45ed758d579a5a24cb\": container with ID starting with 4c8dc833d2a94adf1ae269782de048da0308fed9d4841c45ed758d579a5a24cb not found: ID does not exist"
Aug 04 18:49:50.228286 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:50.228262    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v"]
Aug 04 18:49:50.249770 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:50.249732    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f2f60a-5f977cf86b-dqp9v"]
Aug 04 18:49:50.260727 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:50.260705    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq"]
Aug 04 18:49:50.266810 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:50.266791    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f2f60a-6ff9ff8988-pdqlq"]
Aug 04 18:49:50.353713 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:50.353687    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69ab0c0c-3753-4181-9085-bed884dac850" path="/var/lib/kubelet/pods/69ab0c0c-3753-4181-9085-bed884dac850/volumes"
Aug 04 18:49:50.354043 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:49:50.354030    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d095b18a-feca-4015-a4c4-689e24253d8d" path="/var/lib/kubelet/pods/d095b18a-feca-4015-a4c4-689e24253d8d/volumes"
Aug 04 18:50:20.360330 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.360250    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr"]
Aug 04 18:50:20.361214 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.360487    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr" podUID="bfb66d6b-b813-4b58-ae53-326f08bd3878" containerName="maas-api" containerID="cri-o://1cc21b92435cee3fea0439ac36b01f00b3c71147aa6306365c97da7904accdf7" gracePeriod=30
Aug 04 18:50:20.427407 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.427371    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr" podUID="bfb66d6b-b813-4b58-ae53-326f08bd3878" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.84:8443/health\": dial tcp 10.132.0.84:8443: connect: connection refused"
Aug 04 18:50:20.442633 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.442602    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r"]
Aug 04 18:50:20.442844 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.442816    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r" podUID="7ff9e4ac-14b4-400d-855b-4804b76c43e2" containerName="payload-processing" containerID="cri-o://04e3882a53cf8c17e0883c3b7464b1d5cda4b610a04e1a733f67bc116f5a6350" gracePeriod=30
Aug 04 18:50:20.447350 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.447323    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8"]
Aug 04 18:50:20.447579 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.447549    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8" podUID="19ad9e9e-84ef-4724-8492-1bbd185b9eab" containerName="payload-pre-processing" containerID="cri-o://2420f4bf9f09f868d0b5dac43610c9ac387acc168025855c5bc03fae237160d2" gracePeriod=30
Aug 04 18:50:20.763391 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.763367    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r"
Aug 04 18:50:20.766967 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.766945    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr"
Aug 04 18:50:20.773078 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.773059    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8"
Aug 04 18:50:20.863530 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.863495    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xxqn\" (UniqueName: \"kubernetes.io/projected/19ad9e9e-84ef-4724-8492-1bbd185b9eab-kube-api-access-4xxqn\") pod \"19ad9e9e-84ef-4724-8492-1bbd185b9eab\" (UID: \"19ad9e9e-84ef-4724-8492-1bbd185b9eab\") "
Aug 04 18:50:20.863709 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.863588    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19ad9e9e-84ef-4724-8492-1bbd185b9eab-plugins-config-volume\") pod \"19ad9e9e-84ef-4724-8492-1bbd185b9eab\" (UID: \"19ad9e9e-84ef-4724-8492-1bbd185b9eab\") "
Aug 04 18:50:20.863757 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.863734    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfb66d6b-b813-4b58-ae53-326f08bd3878-maas-api-tls\") pod \"bfb66d6b-b813-4b58-ae53-326f08bd3878\" (UID: \"bfb66d6b-b813-4b58-ae53-326f08bd3878\") "
Aug 04 18:50:20.863835 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.863804    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hs7z8\" (UniqueName: \"kubernetes.io/projected/bfb66d6b-b813-4b58-ae53-326f08bd3878-kube-api-access-hs7z8\") pod \"bfb66d6b-b813-4b58-ae53-326f08bd3878\" (UID: \"bfb66d6b-b813-4b58-ae53-326f08bd3878\") "
Aug 04 18:50:20.863901 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.863846    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfb66d6b-b813-4b58-ae53-326f08bd3878-openshift-service-ca\") pod \"bfb66d6b-b813-4b58-ae53-326f08bd3878\" (UID: \"bfb66d6b-b813-4b58-ae53-326f08bd3878\") "
Aug 04 18:50:20.863901 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.863877    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2j4tp\" (UniqueName: \"kubernetes.io/projected/7ff9e4ac-14b4-400d-855b-4804b76c43e2-kube-api-access-2j4tp\") pod \"7ff9e4ac-14b4-400d-855b-4804b76c43e2\" (UID: \"7ff9e4ac-14b4-400d-855b-4804b76c43e2\") "
Aug 04 18:50:20.864046 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.863906    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ff9e4ac-14b4-400d-855b-4804b76c43e2-plugins-config-volume\") pod \"7ff9e4ac-14b4-400d-855b-4804b76c43e2\" (UID: \"7ff9e4ac-14b4-400d-855b-4804b76c43e2\") "
Aug 04 18:50:20.864046 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.863956    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19ad9e9e-84ef-4724-8492-1bbd185b9eab-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "19ad9e9e-84ef-4724-8492-1bbd185b9eab" (UID: "19ad9e9e-84ef-4724-8492-1bbd185b9eab"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:50:20.864261 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.864235    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19ad9e9e-84ef-4724-8492-1bbd185b9eab-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:50:20.864578 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.864555    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ff9e4ac-14b4-400d-855b-4804b76c43e2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7ff9e4ac-14b4-400d-855b-4804b76c43e2" (UID: "7ff9e4ac-14b4-400d-855b-4804b76c43e2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:50:20.864578 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.864560    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfb66d6b-b813-4b58-ae53-326f08bd3878-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bfb66d6b-b813-4b58-ae53-326f08bd3878" (UID: "bfb66d6b-b813-4b58-ae53-326f08bd3878"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:50:20.866235 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.866211    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19ad9e9e-84ef-4724-8492-1bbd185b9eab-kube-api-access-4xxqn" (OuterVolumeSpecName: "kube-api-access-4xxqn") pod "19ad9e9e-84ef-4724-8492-1bbd185b9eab" (UID: "19ad9e9e-84ef-4724-8492-1bbd185b9eab"). InnerVolumeSpecName "kube-api-access-4xxqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:50:20.866680 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.866660    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfb66d6b-b813-4b58-ae53-326f08bd3878-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bfb66d6b-b813-4b58-ae53-326f08bd3878" (UID: "bfb66d6b-b813-4b58-ae53-326f08bd3878"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:50:20.866738 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.866691    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfb66d6b-b813-4b58-ae53-326f08bd3878-kube-api-access-hs7z8" (OuterVolumeSpecName: "kube-api-access-hs7z8") pod "bfb66d6b-b813-4b58-ae53-326f08bd3878" (UID: "bfb66d6b-b813-4b58-ae53-326f08bd3878"). InnerVolumeSpecName "kube-api-access-hs7z8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:50:20.866869 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.866850    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ff9e4ac-14b4-400d-855b-4804b76c43e2-kube-api-access-2j4tp" (OuterVolumeSpecName: "kube-api-access-2j4tp") pod "7ff9e4ac-14b4-400d-855b-4804b76c43e2" (UID: "7ff9e4ac-14b4-400d-855b-4804b76c43e2"). InnerVolumeSpecName "kube-api-access-2j4tp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:50:20.965534 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.965493    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xxqn\" (UniqueName: \"kubernetes.io/projected/19ad9e9e-84ef-4724-8492-1bbd185b9eab-kube-api-access-4xxqn\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:50:20.965534 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.965531    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfb66d6b-b813-4b58-ae53-326f08bd3878-maas-api-tls\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:50:20.965534 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.965543    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hs7z8\" (UniqueName: \"kubernetes.io/projected/bfb66d6b-b813-4b58-ae53-326f08bd3878-kube-api-access-hs7z8\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:50:20.965801 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.965551    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfb66d6b-b813-4b58-ae53-326f08bd3878-openshift-service-ca\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:50:20.965801 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.965560    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2j4tp\" (UniqueName: \"kubernetes.io/projected/7ff9e4ac-14b4-400d-855b-4804b76c43e2-kube-api-access-2j4tp\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:50:20.965801 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:20.965573    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ff9e4ac-14b4-400d-855b-4804b76c43e2-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:50:21.369538 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:21.369456    2560 generic.go:356] "Generic (PLEG): container finished" podID="bfb66d6b-b813-4b58-ae53-326f08bd3878" containerID="1cc21b92435cee3fea0439ac36b01f00b3c71147aa6306365c97da7904accdf7" exitCode=0
Aug 04 18:50:21.369538 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:21.369510    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr"
Aug 04 18:50:21.370157 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:21.369537    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr" event={"ID":"bfb66d6b-b813-4b58-ae53-326f08bd3878","Type":"ContainerDied","Data":"1cc21b92435cee3fea0439ac36b01f00b3c71147aa6306365c97da7904accdf7"}
Aug 04 18:50:21.370157 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:21.369595    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr" event={"ID":"bfb66d6b-b813-4b58-ae53-326f08bd3878","Type":"ContainerDied","Data":"ce6312930dcda55b17a08e474e70344f3a378004869eecce8a18fbbdeee17ca4"}
Aug 04 18:50:21.370157 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:21.369627    2560 scope.go:117] "RemoveContainer" containerID="1cc21b92435cee3fea0439ac36b01f00b3c71147aa6306365c97da7904accdf7"
Aug 04 18:50:21.371276 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:21.371253    2560 generic.go:356] "Generic (PLEG): container finished" podID="7ff9e4ac-14b4-400d-855b-4804b76c43e2" containerID="04e3882a53cf8c17e0883c3b7464b1d5cda4b610a04e1a733f67bc116f5a6350" exitCode=0
Aug 04 18:50:21.371393 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:21.371293    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r"
Aug 04 18:50:21.371393 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:21.371337    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r" event={"ID":"7ff9e4ac-14b4-400d-855b-4804b76c43e2","Type":"ContainerDied","Data":"04e3882a53cf8c17e0883c3b7464b1d5cda4b610a04e1a733f67bc116f5a6350"}
Aug 04 18:50:21.371495 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:21.371403    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r" event={"ID":"7ff9e4ac-14b4-400d-855b-4804b76c43e2","Type":"ContainerDied","Data":"a1f89deaea74c54046cdc8c09af6e5aace8afdc95a2110c0b10176e0b7e3689b"}
Aug 04 18:50:21.372853 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:21.372829    2560 generic.go:356] "Generic (PLEG): container finished" podID="19ad9e9e-84ef-4724-8492-1bbd185b9eab" containerID="2420f4bf9f09f868d0b5dac43610c9ac387acc168025855c5bc03fae237160d2" exitCode=0
Aug 04 18:50:21.372999 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:21.372851    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8" event={"ID":"19ad9e9e-84ef-4724-8492-1bbd185b9eab","Type":"ContainerDied","Data":"2420f4bf9f09f868d0b5dac43610c9ac387acc168025855c5bc03fae237160d2"}
Aug 04 18:50:21.372999 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:21.372872    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8"
Aug 04 18:50:21.372999 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:21.372892    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8" event={"ID":"19ad9e9e-84ef-4724-8492-1bbd185b9eab","Type":"ContainerDied","Data":"351db590b8215a0e1aeeb3da96a3206aa3767ab59ee4fc4519958f30edee9d77"}
Aug 04 18:50:21.389161 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:21.389143    2560 scope.go:117] "RemoveContainer" containerID="1cc21b92435cee3fea0439ac36b01f00b3c71147aa6306365c97da7904accdf7"
Aug 04 18:50:21.389468 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:50:21.389444    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cc21b92435cee3fea0439ac36b01f00b3c71147aa6306365c97da7904accdf7\": container with ID starting with 1cc21b92435cee3fea0439ac36b01f00b3c71147aa6306365c97da7904accdf7 not found: ID does not exist" containerID="1cc21b92435cee3fea0439ac36b01f00b3c71147aa6306365c97da7904accdf7"
Aug 04 18:50:21.389544 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:21.389477    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cc21b92435cee3fea0439ac36b01f00b3c71147aa6306365c97da7904accdf7"} err="failed to get container status \"1cc21b92435cee3fea0439ac36b01f00b3c71147aa6306365c97da7904accdf7\": rpc error: code = NotFound desc = could not find container \"1cc21b92435cee3fea0439ac36b01f00b3c71147aa6306365c97da7904accdf7\": container with ID starting with 1cc21b92435cee3fea0439ac36b01f00b3c71147aa6306365c97da7904accdf7 not found: ID does not exist"
Aug 04 18:50:21.389544 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:21.389493    2560 scope.go:117] "RemoveContainer" containerID="04e3882a53cf8c17e0883c3b7464b1d5cda4b610a04e1a733f67bc116f5a6350"
Aug 04 18:50:21.397450 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:21.397430    2560 scope.go:117] "RemoveContainer" containerID="04e3882a53cf8c17e0883c3b7464b1d5cda4b610a04e1a733f67bc116f5a6350"
Aug 04 18:50:21.397690 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:50:21.397673    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04e3882a53cf8c17e0883c3b7464b1d5cda4b610a04e1a733f67bc116f5a6350\": container with ID starting with 04e3882a53cf8c17e0883c3b7464b1d5cda4b610a04e1a733f67bc116f5a6350 not found: ID does not exist" containerID="04e3882a53cf8c17e0883c3b7464b1d5cda4b610a04e1a733f67bc116f5a6350"
Aug 04 18:50:21.397754 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:21.397696    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04e3882a53cf8c17e0883c3b7464b1d5cda4b610a04e1a733f67bc116f5a6350"} err="failed to get container status \"04e3882a53cf8c17e0883c3b7464b1d5cda4b610a04e1a733f67bc116f5a6350\": rpc error: code = NotFound desc = could not find container \"04e3882a53cf8c17e0883c3b7464b1d5cda4b610a04e1a733f67bc116f5a6350\": container with ID starting with 04e3882a53cf8c17e0883c3b7464b1d5cda4b610a04e1a733f67bc116f5a6350 not found: ID does not exist"
Aug 04 18:50:21.397754 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:21.397712    2560 scope.go:117] "RemoveContainer" containerID="2420f4bf9f09f868d0b5dac43610c9ac387acc168025855c5bc03fae237160d2"
Aug 04 18:50:21.406105 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:21.406088    2560 scope.go:117] "RemoveContainer" containerID="2420f4bf9f09f868d0b5dac43610c9ac387acc168025855c5bc03fae237160d2"
Aug 04 18:50:21.406334 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:21.406313    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r"]
Aug 04 18:50:21.406430 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:50:21.406338    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2420f4bf9f09f868d0b5dac43610c9ac387acc168025855c5bc03fae237160d2\": container with ID starting with 2420f4bf9f09f868d0b5dac43610c9ac387acc168025855c5bc03fae237160d2 not found: ID does not exist" containerID="2420f4bf9f09f868d0b5dac43610c9ac387acc168025855c5bc03fae237160d2"
Aug 04 18:50:21.406430 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:21.406361    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2420f4bf9f09f868d0b5dac43610c9ac387acc168025855c5bc03fae237160d2"} err="failed to get container status \"2420f4bf9f09f868d0b5dac43610c9ac387acc168025855c5bc03fae237160d2\": rpc error: code = NotFound desc = could not find container \"2420f4bf9f09f868d0b5dac43610c9ac387acc168025855c5bc03fae237160d2\": container with ID starting with 2420f4bf9f09f868d0b5dac43610c9ac387acc168025855c5bc03fae237160d2 not found: ID does not exist"
Aug 04 18:50:21.414176 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:21.414156    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-6a6d9f-86c67d8986-jnv9r"]
Aug 04 18:50:21.426417 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:21.426395    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8"]
Aug 04 18:50:21.432287 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:21.432266    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6a6d9f-7d4484744-vqdp8"]
Aug 04 18:50:21.450177 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:21.450148    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr"]
Aug 04 18:50:21.456306 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:21.456286    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a6d9f-5cc4f9f485-6ffhr"]
Aug 04 18:50:22.354194 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:22.354159    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19ad9e9e-84ef-4724-8492-1bbd185b9eab" path="/var/lib/kubelet/pods/19ad9e9e-84ef-4724-8492-1bbd185b9eab/volumes"
Aug 04 18:50:22.354588 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:22.354572    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ff9e4ac-14b4-400d-855b-4804b76c43e2" path="/var/lib/kubelet/pods/7ff9e4ac-14b4-400d-855b-4804b76c43e2/volumes"
Aug 04 18:50:22.354973 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:50:22.354959    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfb66d6b-b813-4b58-ae53-326f08bd3878" path="/var/lib/kubelet/pods/bfb66d6b-b813-4b58-ae53-326f08bd3878/volumes"
Aug 04 18:51:19.918811 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:19.918776    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8"]
Aug 04 18:51:19.919326 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:19.919058    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8" podUID="9d547860-b313-4977-a2d2-c05253219539" containerName="payload-processing" containerID="cri-o://3f8f2a36245cf8d6be2222f143aa6fa6de6f31f0b5f1458e159f67bfab6fd7d9" gracePeriod=30
Aug 04 18:51:19.926287 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:19.926143    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz"]
Aug 04 18:51:19.926403 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:19.926357    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz" podUID="6bb7a6fe-b7c7-4676-b0ce-87c1137cae07" containerName="payload-pre-processing" containerID="cri-o://1a5efe98eaee3a613e2cdd646fef1f5643451a2086f1245db0b2f7b43afd0184" gracePeriod=30
Aug 04 18:51:20.185343 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.185288    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8"
Aug 04 18:51:20.216780 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.216756    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz"
Aug 04 18:51:20.305257 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.305230    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsghj\" (UniqueName: \"kubernetes.io/projected/6bb7a6fe-b7c7-4676-b0ce-87c1137cae07-kube-api-access-dsghj\") pod \"6bb7a6fe-b7c7-4676-b0ce-87c1137cae07\" (UID: \"6bb7a6fe-b7c7-4676-b0ce-87c1137cae07\") "
Aug 04 18:51:20.305407 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.305284    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bb7a6fe-b7c7-4676-b0ce-87c1137cae07-plugins-config-volume\") pod \"6bb7a6fe-b7c7-4676-b0ce-87c1137cae07\" (UID: \"6bb7a6fe-b7c7-4676-b0ce-87c1137cae07\") "
Aug 04 18:51:20.305407 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.305327    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d547860-b313-4977-a2d2-c05253219539-plugins-config-volume\") pod \"9d547860-b313-4977-a2d2-c05253219539\" (UID: \"9d547860-b313-4977-a2d2-c05253219539\") "
Aug 04 18:51:20.305407 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.305393    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbfbl\" (UniqueName: \"kubernetes.io/projected/9d547860-b313-4977-a2d2-c05253219539-kube-api-access-zbfbl\") pod \"9d547860-b313-4977-a2d2-c05253219539\" (UID: \"9d547860-b313-4977-a2d2-c05253219539\") "
Aug 04 18:51:20.305714 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.305687    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bb7a6fe-b7c7-4676-b0ce-87c1137cae07-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6bb7a6fe-b7c7-4676-b0ce-87c1137cae07" (UID: "6bb7a6fe-b7c7-4676-b0ce-87c1137cae07"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:51:20.305843 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.305743    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d547860-b313-4977-a2d2-c05253219539-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9d547860-b313-4977-a2d2-c05253219539" (UID: "9d547860-b313-4977-a2d2-c05253219539"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:51:20.307498 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.307475    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bb7a6fe-b7c7-4676-b0ce-87c1137cae07-kube-api-access-dsghj" (OuterVolumeSpecName: "kube-api-access-dsghj") pod "6bb7a6fe-b7c7-4676-b0ce-87c1137cae07" (UID: "6bb7a6fe-b7c7-4676-b0ce-87c1137cae07"). InnerVolumeSpecName "kube-api-access-dsghj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:51:20.307701 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.307673    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d547860-b313-4977-a2d2-c05253219539-kube-api-access-zbfbl" (OuterVolumeSpecName: "kube-api-access-zbfbl") pod "9d547860-b313-4977-a2d2-c05253219539" (UID: "9d547860-b313-4977-a2d2-c05253219539"). InnerVolumeSpecName "kube-api-access-zbfbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:51:20.406100 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.406079    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bb7a6fe-b7c7-4676-b0ce-87c1137cae07-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:51:20.406100 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.406102    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d547860-b313-4977-a2d2-c05253219539-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:51:20.406238 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.406111    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbfbl\" (UniqueName: \"kubernetes.io/projected/9d547860-b313-4977-a2d2-c05253219539-kube-api-access-zbfbl\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:51:20.406238 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.406119    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsghj\" (UniqueName: \"kubernetes.io/projected/6bb7a6fe-b7c7-4676-b0ce-87c1137cae07-kube-api-access-dsghj\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:51:20.681177 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.681138    2560 generic.go:356] "Generic (PLEG): container finished" podID="9d547860-b313-4977-a2d2-c05253219539" containerID="3f8f2a36245cf8d6be2222f143aa6fa6de6f31f0b5f1458e159f67bfab6fd7d9" exitCode=0
Aug 04 18:51:20.681337 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.681187    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8"
Aug 04 18:51:20.681337 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.681223    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8" event={"ID":"9d547860-b313-4977-a2d2-c05253219539","Type":"ContainerDied","Data":"3f8f2a36245cf8d6be2222f143aa6fa6de6f31f0b5f1458e159f67bfab6fd7d9"}
Aug 04 18:51:20.681337 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.681275    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8" event={"ID":"9d547860-b313-4977-a2d2-c05253219539","Type":"ContainerDied","Data":"46583a205df1230ae9df98828af118cc34165875d5abaa58be9ce2ef4d113dca"}
Aug 04 18:51:20.681337 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.681296    2560 scope.go:117] "RemoveContainer" containerID="3f8f2a36245cf8d6be2222f143aa6fa6de6f31f0b5f1458e159f67bfab6fd7d9"
Aug 04 18:51:20.683022 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.682728    2560 generic.go:356] "Generic (PLEG): container finished" podID="6bb7a6fe-b7c7-4676-b0ce-87c1137cae07" containerID="1a5efe98eaee3a613e2cdd646fef1f5643451a2086f1245db0b2f7b43afd0184" exitCode=0
Aug 04 18:51:20.683022 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.682776    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz" event={"ID":"6bb7a6fe-b7c7-4676-b0ce-87c1137cae07","Type":"ContainerDied","Data":"1a5efe98eaee3a613e2cdd646fef1f5643451a2086f1245db0b2f7b43afd0184"}
Aug 04 18:51:20.683022 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.682781    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz"
Aug 04 18:51:20.683022 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.682807    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz" event={"ID":"6bb7a6fe-b7c7-4676-b0ce-87c1137cae07","Type":"ContainerDied","Data":"e3bbd1d0cb9754d2a7789277a76be002b53ffd7ff132e5cb7c6ebf69b6aaf50d"}
Aug 04 18:51:20.690723 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.690706    2560 scope.go:117] "RemoveContainer" containerID="3f8f2a36245cf8d6be2222f143aa6fa6de6f31f0b5f1458e159f67bfab6fd7d9"
Aug 04 18:51:20.690982 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:51:20.690960    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f8f2a36245cf8d6be2222f143aa6fa6de6f31f0b5f1458e159f67bfab6fd7d9\": container with ID starting with 3f8f2a36245cf8d6be2222f143aa6fa6de6f31f0b5f1458e159f67bfab6fd7d9 not found: ID does not exist" containerID="3f8f2a36245cf8d6be2222f143aa6fa6de6f31f0b5f1458e159f67bfab6fd7d9"
Aug 04 18:51:20.691060 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.691010    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f8f2a36245cf8d6be2222f143aa6fa6de6f31f0b5f1458e159f67bfab6fd7d9"} err="failed to get container status \"3f8f2a36245cf8d6be2222f143aa6fa6de6f31f0b5f1458e159f67bfab6fd7d9\": rpc error: code = NotFound desc = could not find container \"3f8f2a36245cf8d6be2222f143aa6fa6de6f31f0b5f1458e159f67bfab6fd7d9\": container with ID starting with 3f8f2a36245cf8d6be2222f143aa6fa6de6f31f0b5f1458e159f67bfab6fd7d9 not found: ID does not exist"
Aug 04 18:51:20.691060 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.691028    2560 scope.go:117] "RemoveContainer" containerID="1a5efe98eaee3a613e2cdd646fef1f5643451a2086f1245db0b2f7b43afd0184"
Aug 04 18:51:20.699175 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.699158    2560 scope.go:117] "RemoveContainer" containerID="1a5efe98eaee3a613e2cdd646fef1f5643451a2086f1245db0b2f7b43afd0184"
Aug 04 18:51:20.699378 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.699358    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz"]
Aug 04 18:51:20.699471 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:51:20.699411    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a5efe98eaee3a613e2cdd646fef1f5643451a2086f1245db0b2f7b43afd0184\": container with ID starting with 1a5efe98eaee3a613e2cdd646fef1f5643451a2086f1245db0b2f7b43afd0184 not found: ID does not exist" containerID="1a5efe98eaee3a613e2cdd646fef1f5643451a2086f1245db0b2f7b43afd0184"
Aug 04 18:51:20.699471 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.699428    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a5efe98eaee3a613e2cdd646fef1f5643451a2086f1245db0b2f7b43afd0184"} err="failed to get container status \"1a5efe98eaee3a613e2cdd646fef1f5643451a2086f1245db0b2f7b43afd0184\": rpc error: code = NotFound desc = could not find container \"1a5efe98eaee3a613e2cdd646fef1f5643451a2086f1245db0b2f7b43afd0184\": container with ID starting with 1a5efe98eaee3a613e2cdd646fef1f5643451a2086f1245db0b2f7b43afd0184 not found: ID does not exist"
Aug 04 18:51:20.703614 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.703594    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-071def-59b95f5f76-h5nkz"]
Aug 04 18:51:20.715329 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.715307    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8"]
Aug 04 18:51:20.720315 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:20.720298    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-071def-7bd8f6c59d-74hw8"]
Aug 04 18:51:22.353899 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:22.353866    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bb7a6fe-b7c7-4676-b0ce-87c1137cae07" path="/var/lib/kubelet/pods/6bb7a6fe-b7c7-4676-b0ce-87c1137cae07/volumes"
Aug 04 18:51:22.354293 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:22.354249    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d547860-b313-4977-a2d2-c05253219539" path="/var/lib/kubelet/pods/9d547860-b313-4977-a2d2-c05253219539/volumes"
Aug 04 18:51:35.176840 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.176801    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx"]
Aug 04 18:51:35.177396 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.177078    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx" podUID="f9d6f224-ceb0-4f3f-8377-6467e9b99d14" containerName="maas-api" containerID="cri-o://f854ce127d3c1e899cc2832a86fa4fb5f240d2744585fbd4f8595eb116f10cb9" gracePeriod=30
Aug 04 18:51:35.240902 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.240874    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v"]
Aug 04 18:51:35.241361 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.241311    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v" podUID="ed1e3367-3fc6-454c-bf49-4a66dd82e41b" containerName="payload-processing" containerID="cri-o://f4b052a27bc2f579de0ed6b6cdd697b7c3aa4d3a5fff00b57f9f859a6ff969c2" gracePeriod=30
Aug 04 18:51:35.250826 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.250793    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2"]
Aug 04 18:51:35.251179 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.251123    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2" podUID="9d085af0-a8cb-48ed-86ce-d6e190d7afd9" containerName="payload-pre-processing" containerID="cri-o://271e85b91865d7b9c748f63cb3fa15a46a3299b08ed4dd790dbac1cb5de9743d" gracePeriod=30
Aug 04 18:51:35.536468 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.536432    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v"
Aug 04 18:51:35.562521 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.562487    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx"
Aug 04 18:51:35.605781 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.605759    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2"
Aug 04 18:51:35.637705 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.637680    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ck9b5\" (UniqueName: \"kubernetes.io/projected/ed1e3367-3fc6-454c-bf49-4a66dd82e41b-kube-api-access-ck9b5\") pod \"ed1e3367-3fc6-454c-bf49-4a66dd82e41b\" (UID: \"ed1e3367-3fc6-454c-bf49-4a66dd82e41b\") "
Aug 04 18:51:35.637842 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.637739    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9d6f224-ceb0-4f3f-8377-6467e9b99d14-maas-api-tls\") pod \"f9d6f224-ceb0-4f3f-8377-6467e9b99d14\" (UID: \"f9d6f224-ceb0-4f3f-8377-6467e9b99d14\") "
Aug 04 18:51:35.637842 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.637796    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d085af0-a8cb-48ed-86ce-d6e190d7afd9-plugins-config-volume\") pod \"9d085af0-a8cb-48ed-86ce-d6e190d7afd9\" (UID: \"9d085af0-a8cb-48ed-86ce-d6e190d7afd9\") "
Aug 04 18:51:35.637842 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.637827    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vx9bx\" (UniqueName: \"kubernetes.io/projected/9d085af0-a8cb-48ed-86ce-d6e190d7afd9-kube-api-access-vx9bx\") pod \"9d085af0-a8cb-48ed-86ce-d6e190d7afd9\" (UID: \"9d085af0-a8cb-48ed-86ce-d6e190d7afd9\") "
Aug 04 18:51:35.638035 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.637870    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9d6f224-ceb0-4f3f-8377-6467e9b99d14-openshift-service-ca\") pod \"f9d6f224-ceb0-4f3f-8377-6467e9b99d14\" (UID: \"f9d6f224-ceb0-4f3f-8377-6467e9b99d14\") "
Aug 04 18:51:35.638035 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.637903    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9l25m\" (UniqueName: \"kubernetes.io/projected/f9d6f224-ceb0-4f3f-8377-6467e9b99d14-kube-api-access-9l25m\") pod \"f9d6f224-ceb0-4f3f-8377-6467e9b99d14\" (UID: \"f9d6f224-ceb0-4f3f-8377-6467e9b99d14\") "
Aug 04 18:51:35.638035 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.637950    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed1e3367-3fc6-454c-bf49-4a66dd82e41b-plugins-config-volume\") pod \"ed1e3367-3fc6-454c-bf49-4a66dd82e41b\" (UID: \"ed1e3367-3fc6-454c-bf49-4a66dd82e41b\") "
Aug 04 18:51:35.638298 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.638264    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d085af0-a8cb-48ed-86ce-d6e190d7afd9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9d085af0-a8cb-48ed-86ce-d6e190d7afd9" (UID: "9d085af0-a8cb-48ed-86ce-d6e190d7afd9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:51:35.638648 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.638597    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9d6f224-ceb0-4f3f-8377-6467e9b99d14-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f9d6f224-ceb0-4f3f-8377-6467e9b99d14" (UID: "f9d6f224-ceb0-4f3f-8377-6467e9b99d14"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:51:35.638820 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.638639    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed1e3367-3fc6-454c-bf49-4a66dd82e41b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ed1e3367-3fc6-454c-bf49-4a66dd82e41b" (UID: "ed1e3367-3fc6-454c-bf49-4a66dd82e41b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:51:35.640080 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.640057    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9d6f224-ceb0-4f3f-8377-6467e9b99d14-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f9d6f224-ceb0-4f3f-8377-6467e9b99d14" (UID: "f9d6f224-ceb0-4f3f-8377-6467e9b99d14"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:51:35.640200 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.640179    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d085af0-a8cb-48ed-86ce-d6e190d7afd9-kube-api-access-vx9bx" (OuterVolumeSpecName: "kube-api-access-vx9bx") pod "9d085af0-a8cb-48ed-86ce-d6e190d7afd9" (UID: "9d085af0-a8cb-48ed-86ce-d6e190d7afd9"). InnerVolumeSpecName "kube-api-access-vx9bx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:51:35.640269 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.640224    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed1e3367-3fc6-454c-bf49-4a66dd82e41b-kube-api-access-ck9b5" (OuterVolumeSpecName: "kube-api-access-ck9b5") pod "ed1e3367-3fc6-454c-bf49-4a66dd82e41b" (UID: "ed1e3367-3fc6-454c-bf49-4a66dd82e41b"). InnerVolumeSpecName "kube-api-access-ck9b5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:51:35.640942 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.640921    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9d6f224-ceb0-4f3f-8377-6467e9b99d14-kube-api-access-9l25m" (OuterVolumeSpecName: "kube-api-access-9l25m") pod "f9d6f224-ceb0-4f3f-8377-6467e9b99d14" (UID: "f9d6f224-ceb0-4f3f-8377-6467e9b99d14"). InnerVolumeSpecName "kube-api-access-9l25m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:51:35.739576 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.739506    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ck9b5\" (UniqueName: \"kubernetes.io/projected/ed1e3367-3fc6-454c-bf49-4a66dd82e41b-kube-api-access-ck9b5\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:51:35.739576 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.739532    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9d6f224-ceb0-4f3f-8377-6467e9b99d14-maas-api-tls\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:51:35.739576 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.739542    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d085af0-a8cb-48ed-86ce-d6e190d7afd9-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:51:35.739576 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.739552    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vx9bx\" (UniqueName: \"kubernetes.io/projected/9d085af0-a8cb-48ed-86ce-d6e190d7afd9-kube-api-access-vx9bx\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:51:35.739576 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.739561    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9d6f224-ceb0-4f3f-8377-6467e9b99d14-openshift-service-ca\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:51:35.739576 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.739571    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9l25m\" (UniqueName: \"kubernetes.io/projected/f9d6f224-ceb0-4f3f-8377-6467e9b99d14-kube-api-access-9l25m\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:51:35.739576 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.739579    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed1e3367-3fc6-454c-bf49-4a66dd82e41b-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:51:35.761063 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.761039    2560 generic.go:356] "Generic (PLEG): container finished" podID="9d085af0-a8cb-48ed-86ce-d6e190d7afd9" containerID="271e85b91865d7b9c748f63cb3fa15a46a3299b08ed4dd790dbac1cb5de9743d" exitCode=0
Aug 04 18:51:35.761195 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.761082    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2"
Aug 04 18:51:35.761195 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.761124    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2" event={"ID":"9d085af0-a8cb-48ed-86ce-d6e190d7afd9","Type":"ContainerDied","Data":"271e85b91865d7b9c748f63cb3fa15a46a3299b08ed4dd790dbac1cb5de9743d"}
Aug 04 18:51:35.761195 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.761175    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2" event={"ID":"9d085af0-a8cb-48ed-86ce-d6e190d7afd9","Type":"ContainerDied","Data":"b39d6a8a971eb77566170f93d6484714baa5e093dda717eab8a91ec331905959"}
Aug 04 18:51:35.761344 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.761207    2560 scope.go:117] "RemoveContainer" containerID="271e85b91865d7b9c748f63cb3fa15a46a3299b08ed4dd790dbac1cb5de9743d"
Aug 04 18:51:35.762512 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.762491    2560 generic.go:356] "Generic (PLEG): container finished" podID="f9d6f224-ceb0-4f3f-8377-6467e9b99d14" containerID="f854ce127d3c1e899cc2832a86fa4fb5f240d2744585fbd4f8595eb116f10cb9" exitCode=0
Aug 04 18:51:35.762614 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.762530    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx"
Aug 04 18:51:35.762614 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.762577    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx" event={"ID":"f9d6f224-ceb0-4f3f-8377-6467e9b99d14","Type":"ContainerDied","Data":"f854ce127d3c1e899cc2832a86fa4fb5f240d2744585fbd4f8595eb116f10cb9"}
Aug 04 18:51:35.762813 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.762791    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx" event={"ID":"f9d6f224-ceb0-4f3f-8377-6467e9b99d14","Type":"ContainerDied","Data":"2d467f15d0dc6e107c082c5b33cb766000319cc308dcadbd136aee713c01af04"}
Aug 04 18:51:35.764263 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.764241    2560 generic.go:356] "Generic (PLEG): container finished" podID="ed1e3367-3fc6-454c-bf49-4a66dd82e41b" containerID="f4b052a27bc2f579de0ed6b6cdd697b7c3aa4d3a5fff00b57f9f859a6ff969c2" exitCode=0
Aug 04 18:51:35.764354 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.764284    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v" event={"ID":"ed1e3367-3fc6-454c-bf49-4a66dd82e41b","Type":"ContainerDied","Data":"f4b052a27bc2f579de0ed6b6cdd697b7c3aa4d3a5fff00b57f9f859a6ff969c2"}
Aug 04 18:51:35.764354 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.764323    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v" event={"ID":"ed1e3367-3fc6-454c-bf49-4a66dd82e41b","Type":"ContainerDied","Data":"ff9e43958419a5a5eb94177fff62270c60a3da5acdedb3d93963ebd50ab893dc"}
Aug 04 18:51:35.764354 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.764288    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v"
Aug 04 18:51:35.772438 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.772416    2560 scope.go:117] "RemoveContainer" containerID="271e85b91865d7b9c748f63cb3fa15a46a3299b08ed4dd790dbac1cb5de9743d"
Aug 04 18:51:35.772778 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:51:35.772715    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"271e85b91865d7b9c748f63cb3fa15a46a3299b08ed4dd790dbac1cb5de9743d\": container with ID starting with 271e85b91865d7b9c748f63cb3fa15a46a3299b08ed4dd790dbac1cb5de9743d not found: ID does not exist" containerID="271e85b91865d7b9c748f63cb3fa15a46a3299b08ed4dd790dbac1cb5de9743d"
Aug 04 18:51:35.772778 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.772740    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"271e85b91865d7b9c748f63cb3fa15a46a3299b08ed4dd790dbac1cb5de9743d"} err="failed to get container status \"271e85b91865d7b9c748f63cb3fa15a46a3299b08ed4dd790dbac1cb5de9743d\": rpc error: code = NotFound desc = could not find container \"271e85b91865d7b9c748f63cb3fa15a46a3299b08ed4dd790dbac1cb5de9743d\": container with ID starting with 271e85b91865d7b9c748f63cb3fa15a46a3299b08ed4dd790dbac1cb5de9743d not found: ID does not exist"
Aug 04 18:51:35.772778 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.772755    2560 scope.go:117] "RemoveContainer" containerID="f854ce127d3c1e899cc2832a86fa4fb5f240d2744585fbd4f8595eb116f10cb9"
Aug 04 18:51:35.780895 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.780881    2560 scope.go:117] "RemoveContainer" containerID="f854ce127d3c1e899cc2832a86fa4fb5f240d2744585fbd4f8595eb116f10cb9"
Aug 04 18:51:35.781171 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:51:35.781153    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f854ce127d3c1e899cc2832a86fa4fb5f240d2744585fbd4f8595eb116f10cb9\": container with ID starting with f854ce127d3c1e899cc2832a86fa4fb5f240d2744585fbd4f8595eb116f10cb9 not found: ID does not exist" containerID="f854ce127d3c1e899cc2832a86fa4fb5f240d2744585fbd4f8595eb116f10cb9"
Aug 04 18:51:35.781240 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.781176    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f854ce127d3c1e899cc2832a86fa4fb5f240d2744585fbd4f8595eb116f10cb9"} err="failed to get container status \"f854ce127d3c1e899cc2832a86fa4fb5f240d2744585fbd4f8595eb116f10cb9\": rpc error: code = NotFound desc = could not find container \"f854ce127d3c1e899cc2832a86fa4fb5f240d2744585fbd4f8595eb116f10cb9\": container with ID starting with f854ce127d3c1e899cc2832a86fa4fb5f240d2744585fbd4f8595eb116f10cb9 not found: ID does not exist"
Aug 04 18:51:35.781240 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.781193    2560 scope.go:117] "RemoveContainer" containerID="f4b052a27bc2f579de0ed6b6cdd697b7c3aa4d3a5fff00b57f9f859a6ff969c2"
Aug 04 18:51:35.787762 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.787741    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx"]
Aug 04 18:51:35.788750 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.788728    2560 scope.go:117] "RemoveContainer" containerID="f4b052a27bc2f579de0ed6b6cdd697b7c3aa4d3a5fff00b57f9f859a6ff969c2"
Aug 04 18:51:35.788970 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:51:35.788955    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4b052a27bc2f579de0ed6b6cdd697b7c3aa4d3a5fff00b57f9f859a6ff969c2\": container with ID starting with f4b052a27bc2f579de0ed6b6cdd697b7c3aa4d3a5fff00b57f9f859a6ff969c2 not found: ID does not exist" containerID="f4b052a27bc2f579de0ed6b6cdd697b7c3aa4d3a5fff00b57f9f859a6ff969c2"
Aug 04 18:51:35.789050 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.788975    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4b052a27bc2f579de0ed6b6cdd697b7c3aa4d3a5fff00b57f9f859a6ff969c2"} err="failed to get container status \"f4b052a27bc2f579de0ed6b6cdd697b7c3aa4d3a5fff00b57f9f859a6ff969c2\": rpc error: code = NotFound desc = could not find container \"f4b052a27bc2f579de0ed6b6cdd697b7c3aa4d3a5fff00b57f9f859a6ff969c2\": container with ID starting with f4b052a27bc2f579de0ed6b6cdd697b7c3aa4d3a5fff00b57f9f859a6ff969c2 not found: ID does not exist"
Aug 04 18:51:35.792818 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.792800    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5c8685-84495bc7cb-jj7cx"]
Aug 04 18:51:35.804656 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.804634    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2"]
Aug 04 18:51:35.809049 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.809031    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-5c8685-587fbd979b-2kpr2"]
Aug 04 18:51:35.820814 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.820793    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v"]
Aug 04 18:51:35.826567 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:35.826549    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5c8685-76d5ddf9db-p964v"]
Aug 04 18:51:36.356104 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:36.356068    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d085af0-a8cb-48ed-86ce-d6e190d7afd9" path="/var/lib/kubelet/pods/9d085af0-a8cb-48ed-86ce-d6e190d7afd9/volumes"
Aug 04 18:51:36.356610 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:36.356591    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed1e3367-3fc6-454c-bf49-4a66dd82e41b" path="/var/lib/kubelet/pods/ed1e3367-3fc6-454c-bf49-4a66dd82e41b/volumes"
Aug 04 18:51:36.357126 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:36.357108    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9d6f224-ceb0-4f3f-8377-6467e9b99d14" path="/var/lib/kubelet/pods/f9d6f224-ceb0-4f3f-8377-6467e9b99d14/volumes"
Aug 04 18:51:48.285699 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.285664    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p"]
Aug 04 18:51:48.286171 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.285921    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p" podUID="20d4d278-4eae-4298-8f17-878684d2b21c" containerName="maas-api" containerID="cri-o://37ed8ec9ffd75706087e509570e634aff1f17014fc6b806bd45ed14482d24b19" gracePeriod=30
Aug 04 18:51:48.356588 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.356227    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm"]
Aug 04 18:51:48.356588 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.356446    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm" podUID="35429f65-863d-45be-a5ad-1bee106b1ba5" containerName="payload-processing" containerID="cri-o://26db26eaeed416cb7704cb05b6bec09bdfcaca5ff937b32726dce1c2aaac2e13" gracePeriod=30
Aug 04 18:51:48.360088 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.360066    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w"]
Aug 04 18:51:48.360438 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.360382    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w" podUID="ec7e9cc7-74e3-4eb1-9035-f2186991f9f3" containerName="payload-pre-processing" containerID="cri-o://83361c7573cc9164a730e877e7a8ad5a28ff51af44ff63d4f6b71efbecd9cd9a" gracePeriod=30
Aug 04 18:51:48.628804 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.628773    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p"
Aug 04 18:51:48.634269 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.634207    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm"
Aug 04 18:51:48.659073 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.659043    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w"
Aug 04 18:51:48.751164 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.751134    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96zdx\" (UniqueName: \"kubernetes.io/projected/ec7e9cc7-74e3-4eb1-9035-f2186991f9f3-kube-api-access-96zdx\") pod \"ec7e9cc7-74e3-4eb1-9035-f2186991f9f3\" (UID: \"ec7e9cc7-74e3-4eb1-9035-f2186991f9f3\") "
Aug 04 18:51:48.751312 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.751202    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20d4d278-4eae-4298-8f17-878684d2b21c-maas-api-tls\") pod \"20d4d278-4eae-4298-8f17-878684d2b21c\" (UID: \"20d4d278-4eae-4298-8f17-878684d2b21c\") "
Aug 04 18:51:48.751312 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.751234    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35429f65-863d-45be-a5ad-1bee106b1ba5-plugins-config-volume\") pod \"35429f65-863d-45be-a5ad-1bee106b1ba5\" (UID: \"35429f65-863d-45be-a5ad-1bee106b1ba5\") "
Aug 04 18:51:48.751312 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.751269    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmp9d\" (UniqueName: \"kubernetes.io/projected/35429f65-863d-45be-a5ad-1bee106b1ba5-kube-api-access-xmp9d\") pod \"35429f65-863d-45be-a5ad-1bee106b1ba5\" (UID: \"35429f65-863d-45be-a5ad-1bee106b1ba5\") "
Aug 04 18:51:48.751477 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.751322    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdgg9\" (UniqueName: \"kubernetes.io/projected/20d4d278-4eae-4298-8f17-878684d2b21c-kube-api-access-sdgg9\") pod \"20d4d278-4eae-4298-8f17-878684d2b21c\" (UID: \"20d4d278-4eae-4298-8f17-878684d2b21c\") "
Aug 04 18:51:48.751477 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.751376    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20d4d278-4eae-4298-8f17-878684d2b21c-openshift-service-ca\") pod \"20d4d278-4eae-4298-8f17-878684d2b21c\" (UID: \"20d4d278-4eae-4298-8f17-878684d2b21c\") "
Aug 04 18:51:48.751583 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.751506    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec7e9cc7-74e3-4eb1-9035-f2186991f9f3-plugins-config-volume\") pod \"ec7e9cc7-74e3-4eb1-9035-f2186991f9f3\" (UID: \"ec7e9cc7-74e3-4eb1-9035-f2186991f9f3\") "
Aug 04 18:51:48.751786 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.751676    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35429f65-863d-45be-a5ad-1bee106b1ba5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "35429f65-863d-45be-a5ad-1bee106b1ba5" (UID: "35429f65-863d-45be-a5ad-1bee106b1ba5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:51:48.751872 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.751779    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20d4d278-4eae-4298-8f17-878684d2b21c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "20d4d278-4eae-4298-8f17-878684d2b21c" (UID: "20d4d278-4eae-4298-8f17-878684d2b21c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:51:48.751872 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.751846    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec7e9cc7-74e3-4eb1-9035-f2186991f9f3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ec7e9cc7-74e3-4eb1-9035-f2186991f9f3" (UID: "ec7e9cc7-74e3-4eb1-9035-f2186991f9f3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:51:48.753474 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.753443    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20d4d278-4eae-4298-8f17-878684d2b21c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "20d4d278-4eae-4298-8f17-878684d2b21c" (UID: "20d4d278-4eae-4298-8f17-878684d2b21c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:51:48.753677 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.753657    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35429f65-863d-45be-a5ad-1bee106b1ba5-kube-api-access-xmp9d" (OuterVolumeSpecName: "kube-api-access-xmp9d") pod "35429f65-863d-45be-a5ad-1bee106b1ba5" (UID: "35429f65-863d-45be-a5ad-1bee106b1ba5"). InnerVolumeSpecName "kube-api-access-xmp9d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:51:48.754079 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.754064    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20d4d278-4eae-4298-8f17-878684d2b21c-kube-api-access-sdgg9" (OuterVolumeSpecName: "kube-api-access-sdgg9") pod "20d4d278-4eae-4298-8f17-878684d2b21c" (UID: "20d4d278-4eae-4298-8f17-878684d2b21c"). InnerVolumeSpecName "kube-api-access-sdgg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:51:48.754126 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.754074    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec7e9cc7-74e3-4eb1-9035-f2186991f9f3-kube-api-access-96zdx" (OuterVolumeSpecName: "kube-api-access-96zdx") pod "ec7e9cc7-74e3-4eb1-9035-f2186991f9f3" (UID: "ec7e9cc7-74e3-4eb1-9035-f2186991f9f3"). InnerVolumeSpecName "kube-api-access-96zdx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:51:48.832390 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.832333    2560 generic.go:356] "Generic (PLEG): container finished" podID="ec7e9cc7-74e3-4eb1-9035-f2186991f9f3" containerID="83361c7573cc9164a730e877e7a8ad5a28ff51af44ff63d4f6b71efbecd9cd9a" exitCode=0
Aug 04 18:51:48.832390 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.832382    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w"
Aug 04 18:51:48.832544 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.832414    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w" event={"ID":"ec7e9cc7-74e3-4eb1-9035-f2186991f9f3","Type":"ContainerDied","Data":"83361c7573cc9164a730e877e7a8ad5a28ff51af44ff63d4f6b71efbecd9cd9a"}
Aug 04 18:51:48.832544 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.832463    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w" event={"ID":"ec7e9cc7-74e3-4eb1-9035-f2186991f9f3","Type":"ContainerDied","Data":"0bd7352c1fc14540809cfd8669ad0e5499a9ae3cb674509e63461399fb7a094f"}
Aug 04 18:51:48.832544 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.832489    2560 scope.go:117] "RemoveContainer" containerID="83361c7573cc9164a730e877e7a8ad5a28ff51af44ff63d4f6b71efbecd9cd9a"
Aug 04 18:51:48.833649 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.833628    2560 generic.go:356] "Generic (PLEG): container finished" podID="35429f65-863d-45be-a5ad-1bee106b1ba5" containerID="26db26eaeed416cb7704cb05b6bec09bdfcaca5ff937b32726dce1c2aaac2e13" exitCode=0
Aug 04 18:51:48.833732 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.833663    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm"
Aug 04 18:51:48.833732 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.833710    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm" event={"ID":"35429f65-863d-45be-a5ad-1bee106b1ba5","Type":"ContainerDied","Data":"26db26eaeed416cb7704cb05b6bec09bdfcaca5ff937b32726dce1c2aaac2e13"}
Aug 04 18:51:48.833892 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.833741    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm" event={"ID":"35429f65-863d-45be-a5ad-1bee106b1ba5","Type":"ContainerDied","Data":"08dcec4beb60cc0f0d5bcfbe3b53a0f9576d0aea65abea7fb79c29256ca40094"}
Aug 04 18:51:48.835142 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.835119    2560 generic.go:356] "Generic (PLEG): container finished" podID="20d4d278-4eae-4298-8f17-878684d2b21c" containerID="37ed8ec9ffd75706087e509570e634aff1f17014fc6b806bd45ed14482d24b19" exitCode=0
Aug 04 18:51:48.835256 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.835151    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p" event={"ID":"20d4d278-4eae-4298-8f17-878684d2b21c","Type":"ContainerDied","Data":"37ed8ec9ffd75706087e509570e634aff1f17014fc6b806bd45ed14482d24b19"}
Aug 04 18:51:48.835256 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.835163    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p"
Aug 04 18:51:48.835256 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.835235    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p" event={"ID":"20d4d278-4eae-4298-8f17-878684d2b21c","Type":"ContainerDied","Data":"642f13c308ec0338b58fe6fbf794ec1091fbfd4ff1cf4c1c0066d5e11dae7cf6"}
Aug 04 18:51:48.843639 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.843604    2560 scope.go:117] "RemoveContainer" containerID="83361c7573cc9164a730e877e7a8ad5a28ff51af44ff63d4f6b71efbecd9cd9a"
Aug 04 18:51:48.843922 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:51:48.843903    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83361c7573cc9164a730e877e7a8ad5a28ff51af44ff63d4f6b71efbecd9cd9a\": container with ID starting with 83361c7573cc9164a730e877e7a8ad5a28ff51af44ff63d4f6b71efbecd9cd9a not found: ID does not exist" containerID="83361c7573cc9164a730e877e7a8ad5a28ff51af44ff63d4f6b71efbecd9cd9a"
Aug 04 18:51:48.844035 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.843934    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83361c7573cc9164a730e877e7a8ad5a28ff51af44ff63d4f6b71efbecd9cd9a"} err="failed to get container status \"83361c7573cc9164a730e877e7a8ad5a28ff51af44ff63d4f6b71efbecd9cd9a\": rpc error: code = NotFound desc = could not find container \"83361c7573cc9164a730e877e7a8ad5a28ff51af44ff63d4f6b71efbecd9cd9a\": container with ID starting with 83361c7573cc9164a730e877e7a8ad5a28ff51af44ff63d4f6b71efbecd9cd9a not found: ID does not exist"
Aug 04 18:51:48.844035 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.843957    2560 scope.go:117] "RemoveContainer" containerID="26db26eaeed416cb7704cb05b6bec09bdfcaca5ff937b32726dce1c2aaac2e13"
Aug 04 18:51:48.852087 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.852068    2560 scope.go:117] "RemoveContainer" containerID="26db26eaeed416cb7704cb05b6bec09bdfcaca5ff937b32726dce1c2aaac2e13"
Aug 04 18:51:48.852340 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:51:48.852324    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26db26eaeed416cb7704cb05b6bec09bdfcaca5ff937b32726dce1c2aaac2e13\": container with ID starting with 26db26eaeed416cb7704cb05b6bec09bdfcaca5ff937b32726dce1c2aaac2e13 not found: ID does not exist" containerID="26db26eaeed416cb7704cb05b6bec09bdfcaca5ff937b32726dce1c2aaac2e13"
Aug 04 18:51:48.852414 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.852345    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26db26eaeed416cb7704cb05b6bec09bdfcaca5ff937b32726dce1c2aaac2e13"} err="failed to get container status \"26db26eaeed416cb7704cb05b6bec09bdfcaca5ff937b32726dce1c2aaac2e13\": rpc error: code = NotFound desc = could not find container \"26db26eaeed416cb7704cb05b6bec09bdfcaca5ff937b32726dce1c2aaac2e13\": container with ID starting with 26db26eaeed416cb7704cb05b6bec09bdfcaca5ff937b32726dce1c2aaac2e13 not found: ID does not exist"
Aug 04 18:51:48.852414 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.852360    2560 scope.go:117] "RemoveContainer" containerID="37ed8ec9ffd75706087e509570e634aff1f17014fc6b806bd45ed14482d24b19"
Aug 04 18:51:48.852414 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.852366    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20d4d278-4eae-4298-8f17-878684d2b21c-maas-api-tls\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:51:48.852414 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.852382    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35429f65-863d-45be-a5ad-1bee106b1ba5-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:51:48.852414 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.852392    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmp9d\" (UniqueName: \"kubernetes.io/projected/35429f65-863d-45be-a5ad-1bee106b1ba5-kube-api-access-xmp9d\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:51:48.852414 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.852401    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdgg9\" (UniqueName: \"kubernetes.io/projected/20d4d278-4eae-4298-8f17-878684d2b21c-kube-api-access-sdgg9\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:51:48.852414 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.852409    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20d4d278-4eae-4298-8f17-878684d2b21c-openshift-service-ca\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:51:48.852414 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.852416    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec7e9cc7-74e3-4eb1-9035-f2186991f9f3-plugins-config-volume\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:51:48.852740 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.852425    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96zdx\" (UniqueName: \"kubernetes.io/projected/ec7e9cc7-74e3-4eb1-9035-f2186991f9f3-kube-api-access-96zdx\") on node \"ip-10-0-128-172.ec2.internal\" DevicePath \"\""
Aug 04 18:51:48.858545 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.858528    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w"]
Aug 04 18:51:48.860523 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.860497    2560 scope.go:117] "RemoveContainer" containerID="37ed8ec9ffd75706087e509570e634aff1f17014fc6b806bd45ed14482d24b19"
Aug 04 18:51:48.860834 ip-10-0-128-172 kubenswrapper[2560]: E0804 18:51:48.860808    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37ed8ec9ffd75706087e509570e634aff1f17014fc6b806bd45ed14482d24b19\": container with ID starting with 37ed8ec9ffd75706087e509570e634aff1f17014fc6b806bd45ed14482d24b19 not found: ID does not exist" containerID="37ed8ec9ffd75706087e509570e634aff1f17014fc6b806bd45ed14482d24b19"
Aug 04 18:51:48.860923 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.860837    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37ed8ec9ffd75706087e509570e634aff1f17014fc6b806bd45ed14482d24b19"} err="failed to get container status \"37ed8ec9ffd75706087e509570e634aff1f17014fc6b806bd45ed14482d24b19\": rpc error: code = NotFound desc = could not find container \"37ed8ec9ffd75706087e509570e634aff1f17014fc6b806bd45ed14482d24b19\": container with ID starting with 37ed8ec9ffd75706087e509570e634aff1f17014fc6b806bd45ed14482d24b19 not found: ID does not exist"
Aug 04 18:51:48.864185 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.864167    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-bc18f1-68c5685b5b-g5j5w"]
Aug 04 18:51:48.875886 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.875863    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm"]
Aug 04 18:51:48.880790 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.880771    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-bc18f1-6757fbfbdb-2xhgm"]
Aug 04 18:51:48.892082 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.892054    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p"]
Aug 04 18:51:48.899912 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:48.899895    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bc18f1-6f785f64df-g9t5p"]
Aug 04 18:51:50.353491 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:50.353455    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20d4d278-4eae-4298-8f17-878684d2b21c" path="/var/lib/kubelet/pods/20d4d278-4eae-4298-8f17-878684d2b21c/volumes"
Aug 04 18:51:50.353922 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:50.353906    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35429f65-863d-45be-a5ad-1bee106b1ba5" path="/var/lib/kubelet/pods/35429f65-863d-45be-a5ad-1bee106b1ba5/volumes"
Aug 04 18:51:50.354282 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:50.354269    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec7e9cc7-74e3-4eb1-9035-f2186991f9f3" path="/var/lib/kubelet/pods/ec7e9cc7-74e3-4eb1-9035-f2186991f9f3/volumes"
Aug 04 18:51:56.286981 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:56.286951    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-54dcfb4cb7-5lwnq_4cda36d4-0d07-4bf9-a53d-64d18263c5fd/authorino/0.log"
Aug 04 18:51:58.611729 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:58.611628    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/2.log"
Aug 04 18:51:58.641798 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:58.616951    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 18:51:58.641798 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:58.622205    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/2.log"
Aug 04 18:51:58.641798 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:51:58.627710    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wj5g_d5c319a9-83f7-4237-be47-bc28a033babe/ovn-acl-logging/0.log"
Aug 04 18:52:02.016818 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:02.016781    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-srjp7_58d7dbcb-b524-4c25-8112-ac3cb702e76c/manager/2.log"
Aug 04 18:52:02.283485 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:02.283408    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-57c8d79cc7-97k8r_76b5a999-8b81-45a0-9663-9daabaca87fa/manager/1.log"
Aug 04 18:52:02.650463 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:02.650388    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fd6db6bcb-98q8z_521e0e75-e94e-4541-a553-6ef6b6330eb4/copy-manifests/0.log"
Aug 04 18:52:02.656940 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:02.656903    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fd6db6bcb-98q8z_521e0e75-e94e-4541-a553-6ef6b6330eb4/manager/0.log"
Aug 04 18:52:03.301569 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:03.301537    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-67c8968587-5cqm2_96d4bdeb-9a3c-41e1-9568-3fab780c0469/maas-api/0.log"
Aug 04 18:52:03.790860 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:03.790828    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-5q8m5_be680e73-4411-45f1-9849-3ad182348acf/postgres/0.log"
Aug 04 18:52:04.628438 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:04.628406    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-54dcfb4cb7-5lwnq_4cda36d4-0d07-4bf9-a53d-64d18263c5fd/authorino/0.log"
Aug 04 18:52:06.370204 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:06.370174    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-4kddb_7fff5555-9d9f-475d-82af-84fcf3e5a7b6/payload-pre-processing/0.log"
Aug 04 18:52:06.496366 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:06.496319    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-6wqzc_12873b28-6874-43e7-b589-16b8f7f7fef4/payload-processing/0.log"
Aug 04 18:52:06.633025 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:06.632915    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5955597dc4-jbsv8_b35d3198-f4e3-44a5-a0b7-08c987b5805f/router/0.log"
Aug 04 18:52:07.007327 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:07.007300    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-8bzbg_997763e6-8ea2-4603-be9f-123e78c2b87e/main/0.log"
Aug 04 18:52:07.130853 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:07.130815    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-zclgp_336edf09-c429-4f98-a313-edd0af9ace0f/main/0.log"
Aug 04 18:52:07.248757 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:07.248719    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-4rt8w_d0d794f7-bbdb-4dba-b945-61b1f8535542/main/0.log"
Aug 04 18:52:16.469074 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:16.469040    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jgm5s_5ee91876-14a5-4691-9f54-0d3d11cf70e3/global-pull-secret-syncer/0.log"
Aug 04 18:52:16.687053 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:16.687026    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qrkjb_ac303603-7175-4c58-8989-bafad506bc15/konnectivity-agent/0.log"
Aug 04 18:52:16.785494 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:16.785420    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-172.ec2.internal_3428ebcd2d197fe8677d9367742cfcaa/haproxy/0.log"
Aug 04 18:52:21.357454 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:21.357433    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-54dcfb4cb7-5lwnq_4cda36d4-0d07-4bf9-a53d-64d18263c5fd/authorino/0.log"
Aug 04 18:52:23.424824 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:23.424791    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6f55c4f58d-kxp92_165249a4-0e3d-4f20-9719-c91a6d7ce78f/cluster-monitoring-operator/0.log"
Aug 04 18:52:23.730194 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:23.730119    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gb6pp_32e2da66-4bc5-4f64-9ff7-0d202f34d48e/node-exporter/0.log"
Aug 04 18:52:23.787371 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:23.787346    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gb6pp_32e2da66-4bc5-4f64-9ff7-0d202f34d48e/kube-rbac-proxy/0.log"
Aug 04 18:52:23.889739 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:23.889714    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gb6pp_32e2da66-4bc5-4f64-9ff7-0d202f34d48e/init-textfile/0.log"
Aug 04 18:52:26.322304 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.322276    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-f77f48c6d-stb8t_f583466f-47c3-4403-a048-4c48550ac4fc/networking-console-plugin/0.log"
Aug 04 18:52:26.396475 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.396447    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-v4pcl/perf-node-gather-daemonset-vpqhb"]
Aug 04 18:52:26.396806 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.396794    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ff9e4ac-14b4-400d-855b-4804b76c43e2" containerName="payload-processing"
Aug 04 18:52:26.396852 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.396809    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ff9e4ac-14b4-400d-855b-4804b76c43e2" containerName="payload-processing"
Aug 04 18:52:26.396852 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.396819    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35429f65-863d-45be-a5ad-1bee106b1ba5" containerName="payload-processing"
Aug 04 18:52:26.396852 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.396825    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="35429f65-863d-45be-a5ad-1bee106b1ba5" containerName="payload-processing"
Aug 04 18:52:26.396852 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.396833    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bb7a6fe-b7c7-4676-b0ce-87c1137cae07" containerName="payload-pre-processing"
Aug 04 18:52:26.396852 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.396838    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bb7a6fe-b7c7-4676-b0ce-87c1137cae07" containerName="payload-pre-processing"
Aug 04 18:52:26.396852 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.396843    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69ab0c0c-3753-4181-9085-bed884dac850" containerName="payload-processing"
Aug 04 18:52:26.396852 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.396848    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="69ab0c0c-3753-4181-9085-bed884dac850" containerName="payload-processing"
Aug 04 18:52:26.397092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.396864    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed1e3367-3fc6-454c-bf49-4a66dd82e41b" containerName="payload-processing"
Aug 04 18:52:26.397092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.396869    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed1e3367-3fc6-454c-bf49-4a66dd82e41b" containerName="payload-processing"
Aug 04 18:52:26.397092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.396878    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d095b18a-feca-4015-a4c4-689e24253d8d" containerName="payload-pre-processing"
Aug 04 18:52:26.397092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.396883    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d095b18a-feca-4015-a4c4-689e24253d8d" containerName="payload-pre-processing"
Aug 04 18:52:26.397092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.396892    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d547860-b313-4977-a2d2-c05253219539" containerName="payload-processing"
Aug 04 18:52:26.397092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.396897    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d547860-b313-4977-a2d2-c05253219539" containerName="payload-processing"
Aug 04 18:52:26.397092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.396904    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfb66d6b-b813-4b58-ae53-326f08bd3878" containerName="maas-api"
Aug 04 18:52:26.397092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.396910    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfb66d6b-b813-4b58-ae53-326f08bd3878" containerName="maas-api"
Aug 04 18:52:26.397092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.396920    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20d4d278-4eae-4298-8f17-878684d2b21c" containerName="maas-api"
Aug 04 18:52:26.397092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.396925    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="20d4d278-4eae-4298-8f17-878684d2b21c" containerName="maas-api"
Aug 04 18:52:26.397092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.396932    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d085af0-a8cb-48ed-86ce-d6e190d7afd9" containerName="payload-pre-processing"
Aug 04 18:52:26.397092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.396936    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d085af0-a8cb-48ed-86ce-d6e190d7afd9" containerName="payload-pre-processing"
Aug 04 18:52:26.397092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.396941    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9d6f224-ceb0-4f3f-8377-6467e9b99d14" containerName="maas-api"
Aug 04 18:52:26.397092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.396946    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9d6f224-ceb0-4f3f-8377-6467e9b99d14" containerName="maas-api"
Aug 04 18:52:26.397092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.396954    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec7e9cc7-74e3-4eb1-9035-f2186991f9f3" containerName="payload-pre-processing"
Aug 04 18:52:26.397092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.396959    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec7e9cc7-74e3-4eb1-9035-f2186991f9f3" containerName="payload-pre-processing"
Aug 04 18:52:26.397092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.396965    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19ad9e9e-84ef-4724-8492-1bbd185b9eab" containerName="payload-pre-processing"
Aug 04 18:52:26.397092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.396970    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="19ad9e9e-84ef-4724-8492-1bbd185b9eab" containerName="payload-pre-processing"
Aug 04 18:52:26.397092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.397047    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="20d4d278-4eae-4298-8f17-878684d2b21c" containerName="maas-api"
Aug 04 18:52:26.397092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.397057    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfb66d6b-b813-4b58-ae53-326f08bd3878" containerName="maas-api"
Aug 04 18:52:26.397092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.397064    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="69ab0c0c-3753-4181-9085-bed884dac850" containerName="payload-processing"
Aug 04 18:52:26.397092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.397070    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="35429f65-863d-45be-a5ad-1bee106b1ba5" containerName="payload-processing"
Aug 04 18:52:26.397092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.397076    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="19ad9e9e-84ef-4724-8492-1bbd185b9eab" containerName="payload-pre-processing"
Aug 04 18:52:26.397092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.397081    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec7e9cc7-74e3-4eb1-9035-f2186991f9f3" containerName="payload-pre-processing"
Aug 04 18:52:26.397092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.397086    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bb7a6fe-b7c7-4676-b0ce-87c1137cae07" containerName="payload-pre-processing"
Aug 04 18:52:26.397092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.397091    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d547860-b313-4977-a2d2-c05253219539" containerName="payload-processing"
Aug 04 18:52:26.397092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.397096    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9d6f224-ceb0-4f3f-8377-6467e9b99d14" containerName="maas-api"
Aug 04 18:52:26.397092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.397103    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed1e3367-3fc6-454c-bf49-4a66dd82e41b" containerName="payload-processing"
Aug 04 18:52:26.397944 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.397110    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d085af0-a8cb-48ed-86ce-d6e190d7afd9" containerName="payload-pre-processing"
Aug 04 18:52:26.397944 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.397115    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d095b18a-feca-4015-a4c4-689e24253d8d" containerName="payload-pre-processing"
Aug 04 18:52:26.397944 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.397121    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ff9e4ac-14b4-400d-855b-4804b76c43e2" containerName="payload-processing"
Aug 04 18:52:26.400085 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.400065    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v4pcl/perf-node-gather-daemonset-vpqhb"
Aug 04 18:52:26.403449 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.403419    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-v4pcl\"/\"kube-root-ca.crt\""
Aug 04 18:52:26.403575 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.403534    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-v4pcl\"/\"default-dockercfg-hmk57\""
Aug 04 18:52:26.403575 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.403547    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-v4pcl\"/\"openshift-service-ca.crt\""
Aug 04 18:52:26.415009 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.414975    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v4pcl/perf-node-gather-daemonset-vpqhb"]
Aug 04 18:52:26.466732 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.466705    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w677d\" (UniqueName: \"kubernetes.io/projected/dda05f1c-e583-4ef8-90aa-15f809436460-kube-api-access-w677d\") pod \"perf-node-gather-daemonset-vpqhb\" (UID: \"dda05f1c-e583-4ef8-90aa-15f809436460\") " pod="openshift-must-gather-v4pcl/perf-node-gather-daemonset-vpqhb"
Aug 04 18:52:26.466856 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.466763    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dda05f1c-e583-4ef8-90aa-15f809436460-proc\") pod \"perf-node-gather-daemonset-vpqhb\" (UID: \"dda05f1c-e583-4ef8-90aa-15f809436460\") " pod="openshift-must-gather-v4pcl/perf-node-gather-daemonset-vpqhb"
Aug 04 18:52:26.466856 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.466794    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dda05f1c-e583-4ef8-90aa-15f809436460-lib-modules\") pod \"perf-node-gather-daemonset-vpqhb\" (UID: \"dda05f1c-e583-4ef8-90aa-15f809436460\") " pod="openshift-must-gather-v4pcl/perf-node-gather-daemonset-vpqhb"
Aug 04 18:52:26.466962 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.466872    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dda05f1c-e583-4ef8-90aa-15f809436460-sys\") pod \"perf-node-gather-daemonset-vpqhb\" (UID: \"dda05f1c-e583-4ef8-90aa-15f809436460\") " pod="openshift-must-gather-v4pcl/perf-node-gather-daemonset-vpqhb"
Aug 04 18:52:26.466962 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.466926    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dda05f1c-e583-4ef8-90aa-15f809436460-podres\") pod \"perf-node-gather-daemonset-vpqhb\" (UID: \"dda05f1c-e583-4ef8-90aa-15f809436460\") " pod="openshift-must-gather-v4pcl/perf-node-gather-daemonset-vpqhb"
Aug 04 18:52:26.568011 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.567957    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dda05f1c-e583-4ef8-90aa-15f809436460-lib-modules\") pod \"perf-node-gather-daemonset-vpqhb\" (UID: \"dda05f1c-e583-4ef8-90aa-15f809436460\") " pod="openshift-must-gather-v4pcl/perf-node-gather-daemonset-vpqhb"
Aug 04 18:52:26.568130 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.568034    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dda05f1c-e583-4ef8-90aa-15f809436460-sys\") pod \"perf-node-gather-daemonset-vpqhb\" (UID: \"dda05f1c-e583-4ef8-90aa-15f809436460\") " pod="openshift-must-gather-v4pcl/perf-node-gather-daemonset-vpqhb"
Aug 04 18:52:26.568130 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.568096    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dda05f1c-e583-4ef8-90aa-15f809436460-podres\") pod \"perf-node-gather-daemonset-vpqhb\" (UID: \"dda05f1c-e583-4ef8-90aa-15f809436460\") " pod="openshift-must-gather-v4pcl/perf-node-gather-daemonset-vpqhb"
Aug 04 18:52:26.568130 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.568116    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dda05f1c-e583-4ef8-90aa-15f809436460-lib-modules\") pod \"perf-node-gather-daemonset-vpqhb\" (UID: \"dda05f1c-e583-4ef8-90aa-15f809436460\") " pod="openshift-must-gather-v4pcl/perf-node-gather-daemonset-vpqhb"
Aug 04 18:52:26.568261 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.568140    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dda05f1c-e583-4ef8-90aa-15f809436460-sys\") pod \"perf-node-gather-daemonset-vpqhb\" (UID: \"dda05f1c-e583-4ef8-90aa-15f809436460\") " pod="openshift-must-gather-v4pcl/perf-node-gather-daemonset-vpqhb"
Aug 04 18:52:26.568261 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.568145    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w677d\" (UniqueName: \"kubernetes.io/projected/dda05f1c-e583-4ef8-90aa-15f809436460-kube-api-access-w677d\") pod \"perf-node-gather-daemonset-vpqhb\" (UID: \"dda05f1c-e583-4ef8-90aa-15f809436460\") " pod="openshift-must-gather-v4pcl/perf-node-gather-daemonset-vpqhb"
Aug 04 18:52:26.568261 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.568224    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dda05f1c-e583-4ef8-90aa-15f809436460-proc\") pod \"perf-node-gather-daemonset-vpqhb\" (UID: \"dda05f1c-e583-4ef8-90aa-15f809436460\") " pod="openshift-must-gather-v4pcl/perf-node-gather-daemonset-vpqhb"
Aug 04 18:52:26.568261 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.568230    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dda05f1c-e583-4ef8-90aa-15f809436460-podres\") pod \"perf-node-gather-daemonset-vpqhb\" (UID: \"dda05f1c-e583-4ef8-90aa-15f809436460\") " pod="openshift-must-gather-v4pcl/perf-node-gather-daemonset-vpqhb"
Aug 04 18:52:26.568396 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.568298    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dda05f1c-e583-4ef8-90aa-15f809436460-proc\") pod \"perf-node-gather-daemonset-vpqhb\" (UID: \"dda05f1c-e583-4ef8-90aa-15f809436460\") " pod="openshift-must-gather-v4pcl/perf-node-gather-daemonset-vpqhb"
Aug 04 18:52:26.581376 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.581313    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w677d\" (UniqueName: \"kubernetes.io/projected/dda05f1c-e583-4ef8-90aa-15f809436460-kube-api-access-w677d\") pod \"perf-node-gather-daemonset-vpqhb\" (UID: \"dda05f1c-e583-4ef8-90aa-15f809436460\") " pod="openshift-must-gather-v4pcl/perf-node-gather-daemonset-vpqhb"
Aug 04 18:52:26.710400 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.710372    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v4pcl/perf-node-gather-daemonset-vpqhb"
Aug 04 18:52:26.845032 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:26.845009    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v4pcl/perf-node-gather-daemonset-vpqhb"]
Aug 04 18:52:26.846740 ip-10-0-128-172 kubenswrapper[2560]: W0804 18:52:26.846716    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poddda05f1c_e583_4ef8_90aa_15f809436460.slice/crio-dea09f54ea58c9788d269a096b9c7d6ae08b69c7273dab2b0a56b078b8ef2dde WatchSource:0}: Error finding container dea09f54ea58c9788d269a096b9c7d6ae08b69c7273dab2b0a56b078b8ef2dde: Status 404 returned error can't find the container with id dea09f54ea58c9788d269a096b9c7d6ae08b69c7273dab2b0a56b078b8ef2dde
Aug 04 18:52:27.024037 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:27.024008    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v4pcl/perf-node-gather-daemonset-vpqhb" event={"ID":"dda05f1c-e583-4ef8-90aa-15f809436460","Type":"ContainerStarted","Data":"3f32b0302f730c182918ec5947f969ed974ff81407dcdb9ad1cec4fb34f4c96b"}
Aug 04 18:52:27.024173 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:27.024058    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v4pcl/perf-node-gather-daemonset-vpqhb" event={"ID":"dda05f1c-e583-4ef8-90aa-15f809436460","Type":"ContainerStarted","Data":"dea09f54ea58c9788d269a096b9c7d6ae08b69c7273dab2b0a56b078b8ef2dde"}
Aug 04 18:52:27.024173 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:27.024080    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-v4pcl/perf-node-gather-daemonset-vpqhb"
Aug 04 18:52:27.043903 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:27.043860    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-v4pcl/perf-node-gather-daemonset-vpqhb" podStartSLOduration=1.043845794 podStartE2EDuration="1.043845794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 18:52:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:52:27.04259093 +0000 UTC m=+3629.225751107" watchObservedRunningTime="2026-08-04 18:52:27.043845794 +0000 UTC m=+3629.227005969"
Aug 04 18:52:27.078911 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:27.078882    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/2.log"
Aug 04 18:52:27.083022 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:27.083004    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-gtwkf_8eaf7533-2fc5-4da2-bec2-282dce644049/console-operator/3.log"
Aug 04 18:52:28.370808 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:28.370776    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5b8776669-jkhcl_90aff035-abe4-4fcf-bcd6-76f25a1061dc/volume-data-source-validator/0.log"
Aug 04 18:52:29.636531 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:29.636503    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-d58jd_0a01b635-81f5-4c48-9906-dbf914e08592/dns/0.log"
Aug 04 18:52:29.660466 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:29.660447    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-d58jd_0a01b635-81f5-4c48-9906-dbf914e08592/kube-rbac-proxy/0.log"
Aug 04 18:52:29.718765 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:29.718730    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lzkvj_0b696fd5-c22f-480a-a9bd-a2f8a19784ae/dns-node-resolver/0.log"
Aug 04 18:52:30.407015 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:30.406967    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mdtdv_a4441318-a230-45e7-8ab5-bf4049533e35/node-ca/0.log"
Aug 04 18:52:31.754076 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:31.754051    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-4kddb_7fff5555-9d9f-475d-82af-84fcf3e5a7b6/payload-pre-processing/0.log"
Aug 04 18:52:31.791092 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:31.791071    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-6wqzc_12873b28-6874-43e7-b589-16b8f7f7fef4/payload-processing/0.log"
Aug 04 18:52:31.819960 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:31.819937    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5955597dc4-jbsv8_b35d3198-f4e3-44a5-a0b7-08c987b5805f/router/0.log"
Aug 04 18:52:32.704463 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:32.704438    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5s8jf_76442945-00cd-4bbd-8dc1-a65a9573b031/serve-healthcheck-canary/0.log"
Aug 04 18:52:33.039297 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:33.039224    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-v4pcl/perf-node-gather-daemonset-vpqhb"
Aug 04 18:52:33.295282 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:33.295209    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-ks7s5_6b29597f-59c4-4f28-a164-7d2e97248d83/insights-operator/0.log"
Aug 04 18:52:33.296290 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:33.296275    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-ks7s5_6b29597f-59c4-4f28-a164-7d2e97248d83/insights-operator/1.log"
Aug 04 18:52:33.319767 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:33.319745    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-c94lk_568af22c-f1cf-44ba-abe1-508565a26ebd/kube-rbac-proxy/0.log"
Aug 04 18:52:33.344877 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:33.344853    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-c94lk_568af22c-f1cf-44ba-abe1-508565a26ebd/exporter/0.log"
Aug 04 18:52:33.374616 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:33.374594    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-c94lk_568af22c-f1cf-44ba-abe1-508565a26ebd/extractor/0.log"
Aug 04 18:52:35.875240 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:35.875214    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-srjp7_58d7dbcb-b524-4c25-8112-ac3cb702e76c/manager/1.log"
Aug 04 18:52:35.875852 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:35.875835    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-srjp7_58d7dbcb-b524-4c25-8112-ac3cb702e76c/manager/2.log"
Aug 04 18:52:36.256084 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:36.256053    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-57c8d79cc7-97k8r_76b5a999-8b81-45a0-9663-9daabaca87fa/manager/1.log"
Aug 04 18:52:36.343101 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:36.343070    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-57c8d79cc7-97k8r_76b5a999-8b81-45a0-9663-9daabaca87fa/manager/0.log"
Aug 04 18:52:36.501071 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:36.501046    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fd6db6bcb-98q8z_521e0e75-e94e-4541-a553-6ef6b6330eb4/manager/0.log"
Aug 04 18:52:36.538355 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:36.538329    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fd6db6bcb-98q8z_521e0e75-e94e-4541-a553-6ef6b6330eb4/copy-manifests/0.log"
Aug 04 18:52:37.478222 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:37.478190    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-4kddb_7fff5555-9d9f-475d-82af-84fcf3e5a7b6/payload-pre-processing/0.log"
Aug 04 18:52:37.497353 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:37.497307    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-6wqzc_12873b28-6874-43e7-b589-16b8f7f7fef4/payload-processing/0.log"
Aug 04 18:52:37.519592 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:37.519569    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5955597dc4-jbsv8_b35d3198-f4e3-44a5-a0b7-08c987b5805f/router/0.log"
Aug 04 18:52:38.101082 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:38.101053    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-55bf5c9f58-7gdxs_b748fb50-d006-4dc4-900d-bbebd806fc17/manager/1.log"
Aug 04 18:52:38.101412 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:38.101395    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-55bf5c9f58-7gdxs_b748fb50-d006-4dc4-900d-bbebd806fc17/manager/0.log"
Aug 04 18:52:39.251584 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:39.251554    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-4kddb_7fff5555-9d9f-475d-82af-84fcf3e5a7b6/payload-pre-processing/0.log"
Aug 04 18:52:39.273124 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:39.273098    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-6wqzc_12873b28-6874-43e7-b589-16b8f7f7fef4/payload-processing/0.log"
Aug 04 18:52:39.290265 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:39.290242    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5955597dc4-jbsv8_b35d3198-f4e3-44a5-a0b7-08c987b5805f/router/0.log"
Aug 04 18:52:43.230951 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:43.230919    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-68bc745d87-wtfkt_d3f38436-e32b-4010-81ee-bf8d1f811a17/migrator/0.log"
Aug 04 18:52:43.253124 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:43.253100    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-68bc745d87-wtfkt_d3f38436-e32b-4010-81ee-bf8d1f811a17/graceful-termination/0.log"
Aug 04 18:52:43.595096 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:43.595003    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-47zwf_3f8891ae-cf02-49a7-9144-ef455be4cdee/kube-storage-version-migrator-operator/1.log"
Aug 04 18:52:43.595775 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:43.595757    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-47zwf_3f8891ae-cf02-49a7-9144-ef455be4cdee/kube-storage-version-migrator-operator/0.log"
Aug 04 18:52:44.646820 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:44.646793    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fjnbn_8941bb3a-0305-4d69-8a95-bee3e685acbb/kube-multus-additional-cni-plugins/0.log"
Aug 04 18:52:44.676226 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:44.676200    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fjnbn_8941bb3a-0305-4d69-8a95-bee3e685acbb/egress-router-binary-copy/0.log"
Aug 04 18:52:44.701702 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:44.701681    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fjnbn_8941bb3a-0305-4d69-8a95-bee3e685acbb/cni-plugins/0.log"
Aug 04 18:52:44.729964 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:44.729942    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fjnbn_8941bb3a-0305-4d69-8a95-bee3e685acbb/bond-cni-plugin/0.log"
Aug 04 18:52:44.755523 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:44.755501    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fjnbn_8941bb3a-0305-4d69-8a95-bee3e685acbb/routeoverride-cni/0.log"
Aug 04 18:52:44.778166 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:44.778146    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fjnbn_8941bb3a-0305-4d69-8a95-bee3e685acbb/whereabouts-cni-bincopy/0.log"
Aug 04 18:52:44.800818 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:44.800785    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fjnbn_8941bb3a-0305-4d69-8a95-bee3e685acbb/whereabouts-cni/0.log"
Aug 04 18:52:45.212767 ip-10-0-128-172 kubenswrapper[2560]: I0804 18:52:45.212737    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-f2zqg_cb7cece1-6af0-4189-b5e0-3b5db5c0ca56/kube-multus/0.log"