Jul 24 17:15:27.776740 ip-10-0-131-159 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 24 17:15:27.776751 ip-10-0-131-159 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 24 17:15:27.776758 ip-10-0-131-159 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 24 17:15:27.777018 ip-10-0-131-159 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 24 17:15:37.888170 ip-10-0-131-159 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 24 17:15:37.888191 ip-10-0-131-159 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot df22cb44516f4dab971f131367231c64 --
Jul 24 17:17:16.009299 ip-10-0-131-159 systemd[1]: Starting Kubernetes Kubelet...
Jul 24 17:17:16.407038 ip-10-0-131-159 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 17:17:16.407038 ip-10-0-131-159 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 24 17:17:16.407038 ip-10-0-131-159 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 17:17:16.407038 ip-10-0-131-159 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 24 17:17:16.407038 ip-10-0-131-159 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 17:17:16.407765 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.407101    2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 24 17:17:16.414851 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414828    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:17:16.414851 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414847    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:17:16.414851 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414851    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:17:16.414958 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414873    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:17:16.414958 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414879    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:17:16.414958 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414883    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:17:16.414958 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414886    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:17:16.414958 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414889    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:17:16.414958 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414892    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:17:16.414958 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414895    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:17:16.414958 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414899    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:17:16.414958 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414901    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:17:16.414958 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414904    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:17:16.414958 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414909    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:17:16.414958 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414912    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:17:16.414958 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414915    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:17:16.414958 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414918    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:17:16.414958 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414920    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:17:16.414958 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414924    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:17:16.414958 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414927    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:17:16.414958 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414930    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:17:16.414958 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414934    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:17:16.415440 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414937    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:17:16.415440 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414940    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:17:16.415440 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414942    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:17:16.415440 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414945    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:17:16.415440 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414948    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:17:16.415440 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414950    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:17:16.415440 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414953    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:17:16.415440 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414957    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:17:16.415440 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414960    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:17:16.415440 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414962    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:17:16.415440 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414965    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:17:16.415440 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414968    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:17:16.415440 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414971    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:17:16.415440 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414974    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:17:16.415440 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414991    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:17:16.415440 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414995    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:17:16.415440 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.414999    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:17:16.415440 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415003    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:17:16.415440 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415007    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:17:16.415440 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415011    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:17:16.415959 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415014    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:17:16.415959 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415018    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:17:16.415959 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415020    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:17:16.415959 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415023    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:17:16.415959 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415026    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:17:16.415959 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415029    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:17:16.415959 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415031    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:17:16.415959 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415036    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:17:16.415959 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415039    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:17:16.415959 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415042    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:17:16.415959 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415045    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:17:16.415959 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415048    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:17:16.415959 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415051    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:17:16.415959 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415057    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:17:16.415959 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415060    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:17:16.415959 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415062    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:17:16.415959 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415066    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:17:16.415959 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415069    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:17:16.415959 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415072    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:17:16.415959 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415074    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:17:16.416500 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415077    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:17:16.416500 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415080    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:17:16.416500 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415083    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:17:16.416500 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415086    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:17:16.416500 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415089    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:17:16.416500 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415091    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:17:16.416500 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415094    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:17:16.416500 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415096    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:17:16.416500 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415099    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:17:16.416500 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415102    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:17:16.416500 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415104    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:17:16.416500 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415107    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:17:16.416500 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415110    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:17:16.416500 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415112    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:17:16.416500 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415115    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:17:16.416500 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415117    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:17:16.416500 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415120    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:17:16.416500 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415122    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:17:16.416500 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415125    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:17:16.416500 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415127    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:17:16.417001 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415130    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:17:16.417001 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415133    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:17:16.417001 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415136    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:17:16.417001 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415139    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:17:16.417001 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415141    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:17:16.417001 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415149    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:17:16.417001 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415152    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:17:16.417001 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415155    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:17:16.417001 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415159    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:17:16.417001 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415162    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:17:16.417001 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415165    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:17:16.417001 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415168    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:17:16.417001 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415171    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:17:16.417001 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415174    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:17:16.417001 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415177    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:17:16.417001 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415180    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:17:16.417001 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415183    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:17:16.417001 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415895    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:17:16.417001 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415902    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:17:16.417462 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415905    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:17:16.417462 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415908    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:17:16.417462 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415911    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:17:16.417462 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415914    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:17:16.417462 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415917    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:17:16.417462 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415919    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:17:16.417462 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415922    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:17:16.417462 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415925    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:17:16.417462 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415928    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:17:16.417462 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415930    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:17:16.417462 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415933    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:17:16.417462 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415936    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:17:16.417462 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415938    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:17:16.417462 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415941    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:17:16.417462 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415944    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:17:16.417462 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415946    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:17:16.417462 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415949    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:17:16.417462 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415951    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:17:16.417462 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415955    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:17:16.417462 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415959    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:17:16.417964 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415962    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:17:16.417964 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415965    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:17:16.417964 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415967    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:17:16.417964 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415970    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:17:16.417964 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415973    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:17:16.417964 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415992    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:17:16.417964 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415995    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:17:16.417964 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.415998    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:17:16.417964 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416001    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:17:16.417964 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416004    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:17:16.417964 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416006    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:17:16.417964 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416009    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:17:16.417964 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416012    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:17:16.417964 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416015    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:17:16.417964 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416018    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:17:16.417964 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416020    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:17:16.417964 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416023    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:17:16.417964 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416026    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:17:16.417964 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416029    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:17:16.417964 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416032    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:17:16.418481 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416035    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:17:16.418481 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416037    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:17:16.418481 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416040    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:17:16.418481 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416043    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:17:16.418481 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416045    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:17:16.418481 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416048    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:17:16.418481 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416051    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:17:16.418481 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416054    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:17:16.418481 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416056    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:17:16.418481 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416059    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:17:16.418481 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416062    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:17:16.418481 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416067    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:17:16.418481 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416071    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:17:16.418481 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416074    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:17:16.418481 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416077    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:17:16.418481 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416079    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:17:16.418481 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416081    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:17:16.418481 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416084    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:17:16.418481 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416087    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:17:16.418481 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416089    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:17:16.418974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416092    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:17:16.418974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416095    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:17:16.418974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416097    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:17:16.418974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416100    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:17:16.418974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416102    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:17:16.418974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416105    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:17:16.418974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416109    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:17:16.418974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416112    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:17:16.418974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416115    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:17:16.418974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416119    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:17:16.418974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416122    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:17:16.418974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416125    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:17:16.418974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416128    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:17:16.418974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416131    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:17:16.418974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416133    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:17:16.418974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416136    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:17:16.418974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416140    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:17:16.418974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416143    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:17:16.418974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416147    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:17:16.418974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416149    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:17:16.419493 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416152    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:17:16.419493 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416155    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:17:16.419493 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416157    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:17:16.419493 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416160    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:17:16.419493 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416162    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:17:16.419493 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416166    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:17:16.419493 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416168    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:17:16.419493 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416171    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:17:16.419493 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416174    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:17:16.419493 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416176    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:17:16.419493 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416179    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:17:16.419493 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416182    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:17:16.419493 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416184    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:17:16.419493 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416187    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:17:16.419493 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416189    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:17:16.419493 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416192    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:17:16.419493 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416195    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:17:16.419493 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416579    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:17:16.419493 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416584    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:17:16.419493 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416589    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:17:16.420022 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416592    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:17:16.420022 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416594    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:17:16.420022 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416597    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:17:16.420022 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416600    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:17:16.420022 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416602    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:17:16.420022 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416605    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:17:16.420022 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416607    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:17:16.420022 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416610    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:17:16.420022 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416613    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:17:16.420022 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416617    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:17:16.420022 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416620    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:17:16.420022 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416622    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:17:16.420022 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416626    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:17:16.420022 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416629    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:17:16.420022 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416632    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:17:16.420022 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416634    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:17:16.420022 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416637    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:17:16.420022 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416640    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:17:16.420022 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416643    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:17:16.420022 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416646    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:17:16.420503 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416649    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:17:16.420503 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416653    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:17:16.420503 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416656    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:17:16.420503 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416659    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:17:16.420503 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416661    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:17:16.420503 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416664    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:17:16.420503 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416666    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:17:16.420503 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416669    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:17:16.420503 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416672    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:17:16.420503 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416674    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:17:16.420503 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416678    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:17:16.420503 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416682    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:17:16.420503 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416684    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:17:16.420503 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416687    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:17:16.420503 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416690    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:17:16.420503 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416693    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:17:16.420503 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416695    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:17:16.420503 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416698    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:17:16.420503 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416701    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:17:16.420974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416703    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:17:16.420974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416706    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:17:16.420974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416710    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:17:16.420974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416712    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:17:16.420974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416715    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:17:16.420974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416717    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:17:16.420974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416720    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:17:16.420974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416730    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:17:16.420974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416734    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:17:16.420974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416737    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:17:16.420974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416740    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:17:16.420974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416746    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:17:16.420974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416749    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:17:16.420974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416751    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:17:16.420974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416754    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:17:16.420974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416756    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:17:16.420974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416759    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:17:16.420974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416761    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:17:16.420974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416764    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:17:16.420974 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416767    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:17:16.421514 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416769    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:17:16.421514 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416772    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:17:16.421514 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416774    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:17:16.421514 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416777    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:17:16.421514 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416782    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:17:16.421514 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416785    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:17:16.421514 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416788    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:17:16.421514 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416790    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:17:16.421514 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416794    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:17:16.421514 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416798    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:17:16.421514 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416803    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:17:16.421514 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416806    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:17:16.421514 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416809    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:17:16.421514 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416812    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:17:16.421514 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416816    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:17:16.421514 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416819    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:17:16.421514 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416822    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:17:16.421514 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416825    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:17:16.421514 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416828    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:17:16.421973 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416830    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:17:16.421973 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416833    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:17:16.421973 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416836    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:17:16.421973 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416838    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:17:16.421973 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416841    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:17:16.421973 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416844    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:17:16.421973 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416846    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:17:16.421973 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416849    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:17:16.421973 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416852    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:17:16.421973 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416854    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:17:16.421973 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416856    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:17:16.421973 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416859    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:17:16.421973 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416862    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:17:16.421973 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416864    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:17:16.421973 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416867    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:17:16.421973 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416869    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:17:16.421973 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416872    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:17:16.421973 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.416876    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:17:16.421973 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.416953    2562 flags.go:64] FLAG: --address="0.0.0.0"
Jul 24 17:17:16.421973 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.416961    2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 24 17:17:16.422503 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.416967    2562 flags.go:64] FLAG: --anonymous-auth="true"
Jul 24 17:17:16.422503 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.416971    2562 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 24 17:17:16.422503 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.416988    2562 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 24 17:17:16.422503 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.416992    2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 24 17:17:16.422503 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.416997    2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 24 17:17:16.422503 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417002    2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 24 17:17:16.422503 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417006    2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 24 17:17:16.422503 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417010    2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 24 17:17:16.422503 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417014    2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 24 17:17:16.422503 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417018    2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 24 17:17:16.422503 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417021    2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 24 17:17:16.422503 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417024    2562 flags.go:64] FLAG: --cgroup-root=""
Jul 24 17:17:16.422503 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417027    2562 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 24 17:17:16.422503 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417030    2562 flags.go:64] FLAG: --client-ca-file=""
Jul 24 17:17:16.422503 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417033    2562 flags.go:64] FLAG: --cloud-provider="external"
Jul 24 17:17:16.422503 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417036    2562 flags.go:64] FLAG: --cluster-dns="[]"
Jul 24 17:17:16.422503 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417041    2562 flags.go:64] FLAG: --cluster-domain=""
Jul 24 17:17:16.422503 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417044    2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 24 17:17:16.422503 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417048    2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 24 17:17:16.422503 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417051    2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 24 17:17:16.422503 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417055    2562 flags.go:64] FLAG: --container-log-max-files="5"
Jul 24 17:17:16.422503 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417059    2562 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 24 17:17:16.422503 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417062    2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 24 17:17:16.423085 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417065    2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 24 17:17:16.423085 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417068    2562 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 24 17:17:16.423085 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417071    2562 flags.go:64] FLAG: --contention-profiling="false"
Jul 24 17:17:16.423085 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417074    2562 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 24 17:17:16.423085 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417077    2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 24 17:17:16.423085 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417080    2562 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 24 17:17:16.423085 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417085    2562 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 24 17:17:16.423085 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417089    2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 24 17:17:16.423085 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417092    2562 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 24 17:17:16.423085 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417095    2562 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 24 17:17:16.423085 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417098    2562 flags.go:64] FLAG: --enable-load-reader="false"
Jul 24 17:17:16.423085 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417101    2562 flags.go:64] FLAG: --enable-server="true"
Jul 24 17:17:16.423085 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417104    2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 24 17:17:16.423085 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417109    2562 flags.go:64] FLAG: --event-burst="100"
Jul 24 17:17:16.423085 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417112    2562 flags.go:64] FLAG: --event-qps="50"
Jul 24 17:17:16.423085 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417115    2562 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 24 17:17:16.423085 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417118    2562 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 24 17:17:16.423085 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417121    2562 flags.go:64] FLAG: --eviction-hard=""
Jul 24 17:17:16.423085 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417125    2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 24 17:17:16.423085 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417128    2562 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 24 17:17:16.423085 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417131    2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 24 17:17:16.423085 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417135    2562 flags.go:64] FLAG: --eviction-soft=""
Jul 24 17:17:16.423085 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417138    2562 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 24 17:17:16.423085 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417140    2562 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 24 17:17:16.423085 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417144    2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 24 17:17:16.423713 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417147    2562 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 24 17:17:16.423713 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417149    2562 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 24 17:17:16.423713 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417152    2562 flags.go:64] FLAG: --fail-swap-on="true"
Jul 24 17:17:16.423713 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417156    2562 flags.go:64] FLAG: --feature-gates=""
Jul 24 17:17:16.423713 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417160    2562 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 24 17:17:16.423713 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417163    2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 24 17:17:16.423713 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417166    2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 24 17:17:16.423713 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417169    2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 24 17:17:16.423713 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417172    2562 flags.go:64] FLAG: --healthz-port="10248"
Jul 24 17:17:16.423713 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417175    2562 flags.go:64] FLAG: --help="false"
Jul 24 17:17:16.423713 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417178    2562 flags.go:64] FLAG: --hostname-override="ip-10-0-131-159.ec2.internal"
Jul 24 17:17:16.423713 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417181    2562 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 24 17:17:16.423713 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417184    2562 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 24 17:17:16.423713 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417188    2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 24 17:17:16.423713 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417191    2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 24 17:17:16.423713 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417195    2562 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 24 17:17:16.423713 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417198    2562 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 24 17:17:16.423713 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417201    2562 flags.go:64] FLAG: --image-service-endpoint=""
Jul 24 17:17:16.423713 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417204    2562 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 24 17:17:16.423713 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417207    2562 flags.go:64] FLAG: --kube-api-burst="100"
Jul 24 17:17:16.423713 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417210    2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 24 17:17:16.423713 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417214    2562 flags.go:64] FLAG: --kube-api-qps="50"
Jul 24 17:17:16.423713 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417217    2562 flags.go:64] FLAG: --kube-reserved=""
Jul 24 17:17:16.423713 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417220    2562 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 24 17:17:16.424455 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417223    2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 24 17:17:16.424455 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417226    2562 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 24 17:17:16.424455 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417229    2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 24 17:17:16.424455 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417232    2562 flags.go:64] FLAG: --lock-file=""
Jul 24 17:17:16.424455 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417235    2562 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 24 17:17:16.424455 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417238    2562 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 24 17:17:16.424455 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417241    2562 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 24 17:17:16.424455 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417247    2562 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 24 17:17:16.424455 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417250    2562 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 24 17:17:16.424455 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417253    2562 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 24 17:17:16.424455 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417256    2562 flags.go:64] FLAG: --logging-format="text"
Jul 24 17:17:16.424455 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417259    2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 24 17:17:16.424455 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417263    2562 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 24 17:17:16.424455 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417266    2562 flags.go:64] FLAG: --manifest-url=""
Jul 24 17:17:16.424455 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417269    2562 flags.go:64] FLAG: --manifest-url-header=""
Jul 24 17:17:16.424455 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417273    2562 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 24 17:17:16.424455 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417276    2562 flags.go:64] FLAG: --max-open-files="1000000"
Jul 24 17:17:16.424455 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417281    2562 flags.go:64] FLAG: --max-pods="110"
Jul 24 17:17:16.424455 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417284    2562 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 24 17:17:16.424455 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417287    2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 24 17:17:16.424455 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417290    2562 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 24 17:17:16.424455 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417296    2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 24 17:17:16.424455 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417299    2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 24 17:17:16.424455 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417302    2562 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 24 17:17:16.424455 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417306    2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 24 17:17:16.425501 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417313    2562 flags.go:64] FLAG: --node-status-max-images="50"
Jul 24 17:17:16.425501 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417316    2562 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 24 17:17:16.425501 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417319    2562 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 24 17:17:16.425501 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417323    2562 flags.go:64] FLAG: --pod-cidr=""
Jul 24 17:17:16.425501 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417326    2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 24 17:17:16.425501 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417331    2562 flags.go:64] FLAG: --pod-manifest-path=""
Jul 24 17:17:16.425501 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417334    2562 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 24 17:17:16.425501 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417337    2562 flags.go:64] FLAG: --pods-per-core="0"
Jul 24 17:17:16.425501 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417340    2562 flags.go:64] FLAG: --port="10250"
Jul 24 17:17:16.425501 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417343    2562 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 24 17:17:16.425501 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417347    2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0172a249111e87613"
Jul 24 17:17:16.425501 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417350    2562 flags.go:64] FLAG: --qos-reserved=""
Jul 24 17:17:16.425501 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417353    2562 flags.go:64] FLAG: --read-only-port="10255"
Jul 24 17:17:16.425501 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417356    2562 flags.go:64] FLAG: --register-node="true"
Jul 24 17:17:16.425501 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417359    2562 flags.go:64] FLAG: --register-with-taints=""
Jul 24 17:17:16.425501 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417362    2562 flags.go:64] FLAG: --registry-burst="10"
Jul 24 17:17:16.425501 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417365    2562 flags.go:64] FLAG: --registry-qps="5"
Jul 24 17:17:16.425501 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417368    2562 flags.go:64] FLAG: --reserved-cpus=""
Jul 24 17:17:16.425501 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417371    2562 flags.go:64] FLAG: --reserved-memory=""
Jul 24 17:17:16.425501 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417375    2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 24 17:17:16.425501 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417378    2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 24 17:17:16.425501 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417382    2562 flags.go:64] FLAG: --rotate-certificates="false"
Jul 24 17:17:16.425501 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417385    2562 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 24 17:17:16.425501 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417387    2562 flags.go:64] FLAG: --runonce="false"
Jul 24 17:17:16.425501 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417390    2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 24 17:17:16.426286 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417393    2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 24 17:17:16.426286 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417396    2562 flags.go:64] FLAG: --seccomp-default="false"
Jul 24 17:17:16.426286 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417399    2562 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 24 17:17:16.426286 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417402    2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 24 17:17:16.426286 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417406    2562 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 24 17:17:16.426286 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417410    2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 24 17:17:16.426286 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417413    2562 flags.go:64] FLAG: --storage-driver-password="root"
Jul 24 17:17:16.426286 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417416    2562 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 24 17:17:16.426286 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417419    2562 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 24 17:17:16.426286 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417422    2562 flags.go:64] FLAG: --storage-driver-user="root"
Jul 24 17:17:16.426286 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417424    2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 24 17:17:16.426286 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417427    2562 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 24 17:17:16.426286 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417430    2562 flags.go:64] FLAG: --system-cgroups=""
Jul 24 17:17:16.426286 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417434    2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 24 17:17:16.426286 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417440    2562 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 24 17:17:16.426286 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417443    2562 flags.go:64] FLAG: --tls-cert-file=""
Jul 24 17:17:16.426286 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417446    2562 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 24 17:17:16.426286 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417450    2562 flags.go:64] FLAG: --tls-min-version=""
Jul 24 17:17:16.426286 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417453    2562 flags.go:64] FLAG: --tls-private-key-file=""
Jul 24 17:17:16.426286 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417456    2562 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 24 17:17:16.426286 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417459    2562 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 24 17:17:16.426286 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417462    2562 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 24 17:17:16.426286 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417466    2562 flags.go:64] FLAG: --v="2"
Jul 24 17:17:16.426286 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417470    2562 flags.go:64] FLAG: --version="false"
Jul 24 17:17:16.426286 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417474    2562 flags.go:64] FLAG: --vmodule=""
Jul 24 17:17:16.426937 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417478    2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 24 17:17:16.426937 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.417482    2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 24 17:17:16.426937 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417598    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:17:16.426937 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417603    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:17:16.426937 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417607    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:17:16.426937 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417610    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:17:16.426937 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417613    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:17:16.426937 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417617    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:17:16.426937 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417620    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:17:16.426937 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417623    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:17:16.426937 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417626    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:17:16.426937 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417629    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:17:16.426937 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417632    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:17:16.426937 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417635    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:17:16.426937 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417638    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:17:16.426937 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417641    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:17:16.426937 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417645    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:17:16.426937 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417648    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:17:16.426937 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417651    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:17:16.427475 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417653    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:17:16.427475 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417657    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:17:16.427475 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417660    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:17:16.427475 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417663    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:17:16.427475 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417665    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:17:16.427475 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417668    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:17:16.427475 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417670    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:17:16.427475 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417673    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:17:16.427475 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417676    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:17:16.427475 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417679    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:17:16.427475 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417681    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:17:16.427475 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417684    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:17:16.427475 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417687    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:17:16.427475 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417689    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:17:16.427475 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417692    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:17:16.427475 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417695    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:17:16.427475 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417697    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:17:16.427475 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417701    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:17:16.427475 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417704    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:17:16.427475 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417706    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:17:16.427955 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417709    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:17:16.427955 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417711    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:17:16.427955 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417714    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:17:16.427955 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417717    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:17:16.427955 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417720    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:17:16.427955 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417722    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:17:16.427955 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417725    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:17:16.427955 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417727    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:17:16.427955 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417730    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:17:16.427955 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417732    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:17:16.427955 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417735    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:17:16.427955 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417738    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:17:16.427955 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417740    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:17:16.427955 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417744    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:17:16.427955 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417747    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:17:16.427955 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417749    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:17:16.427955 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417752    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:17:16.427955 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417754    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:17:16.427955 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417757    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:17:16.428445 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417761    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:17:16.428445 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417764    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:17:16.428445 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417767    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:17:16.428445 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417770    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:17:16.428445 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417772    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:17:16.428445 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417775    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:17:16.428445 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417778    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:17:16.428445 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417781    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:17:16.428445 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417783    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:17:16.428445 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417786    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:17:16.428445 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417788    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:17:16.428445 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417791    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:17:16.428445 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417794    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:17:16.428445 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417797    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:17:16.428445 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417799    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:17:16.428445 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417802    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:17:16.428445 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417804    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:17:16.428445 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417807    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:17:16.428445 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417810    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:17:16.428445 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417812    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:17:16.428938 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417815    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:17:16.428938 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417817    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:17:16.428938 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417820    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:17:16.428938 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417823    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:17:16.428938 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417825    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:17:16.428938 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417828    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:17:16.428938 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417832    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:17:16.428938 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417835    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:17:16.428938 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417838    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:17:16.428938 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417840    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:17:16.428938 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417843    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:17:16.428938 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417846    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:17:16.428938 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417848    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:17:16.428938 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417851    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:17:16.428938 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417853    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:17:16.428938 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417856    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:17:16.428938 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417858    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:17:16.428938 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417861    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:17:16.428938 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417864    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:17:16.428938 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417866    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:17:16.429448 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417869    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:17:16.429448 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417871    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:17:16.429448 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.417874    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:17:16.429448 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.418459    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 17:17:16.429448 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.427014    2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 24 17:17:16.429448 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.427032    2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 24 17:17:16.429448 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.427046    2562 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 24 17:17:16.429448 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.427052    2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 24 17:17:16.429448 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427113    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:17:16.429448 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427118    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:17:16.429448 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427122    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:17:16.429448 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427125    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:17:16.429448 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427128    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:17:16.429448 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427132    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:17:16.429448 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427135    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:17:16.429811 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427139    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:17:16.429811 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427141    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:17:16.429811 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427144    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:17:16.429811 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427147    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:17:16.429811 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427152    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:17:16.429811 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427157    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:17:16.429811 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427160    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:17:16.429811 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427163    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:17:16.429811 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427166    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:17:16.429811 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427169    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:17:16.429811 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427173    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:17:16.429811 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427176    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:17:16.429811 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427179    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:17:16.429811 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427182    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:17:16.429811 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427185    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:17:16.429811 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427188    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:17:16.429811 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427191    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:17:16.429811 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427193    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:17:16.429811 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427196    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:17:16.430363 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427199    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:17:16.430363 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427201    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:17:16.430363 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427204    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:17:16.430363 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427207    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:17:16.430363 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427210    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:17:16.430363 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427213    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:17:16.430363 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427217    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:17:16.430363 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427221    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:17:16.430363 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427224    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:17:16.430363 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427226    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:17:16.430363 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427229    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:17:16.430363 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427232    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:17:16.430363 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427234    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:17:16.430363 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427237    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:17:16.430363 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427240    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:17:16.430363 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427243    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:17:16.430363 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427245    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:17:16.430363 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427248    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:17:16.430363 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427250    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:17:16.430828 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427253    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:17:16.430828 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427257    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:17:16.430828 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427259    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:17:16.430828 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427262    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:17:16.430828 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427265    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:17:16.430828 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427267    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:17:16.430828 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427270    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:17:16.430828 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427272    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:17:16.430828 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427275    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:17:16.430828 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427277    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:17:16.430828 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427280    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:17:16.430828 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427283    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:17:16.430828 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427286    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:17:16.430828 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427288    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:17:16.430828 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427291    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:17:16.430828 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427293    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:17:16.430828 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427297    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:17:16.430828 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427300    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:17:16.430828 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427303    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:17:16.430828 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427305    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:17:16.431341 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427308    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:17:16.431341 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427310    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:17:16.431341 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427313    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:17:16.431341 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427316    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:17:16.431341 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427319    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:17:16.431341 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427321    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:17:16.431341 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427324    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:17:16.431341 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427326    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:17:16.431341 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427329    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:17:16.431341 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427332    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:17:16.431341 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427334    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:17:16.431341 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427337    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:17:16.431341 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427339    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:17:16.431341 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427343    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:17:16.431341 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427346    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:17:16.431341 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427349    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:17:16.431341 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427352    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:17:16.431341 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427355    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:17:16.431341 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427357    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:17:16.431812 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427361    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:17:16.431812 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427365    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:17:16.431812 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427368    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:17:16.431812 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427371    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:17:16.431812 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427374    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:17:16.431812 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427376    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:17:16.431812 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427379    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:17:16.431812 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427381    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:17:16.431812 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427384    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:17:16.431812 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427387    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:17:16.431812 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427389    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:17:16.431812 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427392    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:17:16.431812 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427395    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:17:16.431812 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427397    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:17:16.431812 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427400    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:17:16.432186 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.427405    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 17:17:16.432186 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427530    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:17:16.432186 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427535    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:17:16.432186 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427538    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:17:16.432186 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427542    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:17:16.432186 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427545    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:17:16.432186 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427547    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:17:16.432186 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427551    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:17:16.432186 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427553    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:17:16.432186 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427556    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:17:16.432186 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427559    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:17:16.432186 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427562    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:17:16.432186 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427567    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:17:16.432186 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427572    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:17:16.432186 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427575    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:17:16.432558 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427578    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:17:16.432558 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427581    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:17:16.432558 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427584    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:17:16.432558 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427586    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:17:16.432558 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427589    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:17:16.432558 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427592    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:17:16.432558 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427594    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:17:16.432558 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427597    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:17:16.432558 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427599    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:17:16.432558 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427602    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:17:16.432558 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427605    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:17:16.432558 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427608    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:17:16.432558 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427610    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:17:16.432558 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427613    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:17:16.432558 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427616    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:17:16.432558 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427618    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:17:16.432558 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427621    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:17:16.432558 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427624    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:17:16.432558 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427626    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:17:16.432558 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427629    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:17:16.433088 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427631    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:17:16.433088 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427634    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:17:16.433088 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427637    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:17:16.433088 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427639    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:17:16.433088 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427642    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:17:16.433088 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427644    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:17:16.433088 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427647    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:17:16.433088 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427650    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:17:16.433088 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427652    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:17:16.433088 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427655    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:17:16.433088 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427658    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:17:16.433088 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427662    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:17:16.433088 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427665    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:17:16.433088 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427667    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:17:16.433088 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427670    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:17:16.433088 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427673    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:17:16.433088 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427676    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:17:16.433088 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427678    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:17:16.433088 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427681    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:17:16.433088 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427683    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:17:16.433583 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427686    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:17:16.433583 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427688    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:17:16.433583 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427691    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:17:16.433583 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427694    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:17:16.433583 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427698    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:17:16.433583 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427701    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:17:16.433583 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427705    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:17:16.433583 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427708    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:17:16.433583 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427711    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:17:16.433583 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427714    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:17:16.433583 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427716    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:17:16.433583 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427719    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:17:16.433583 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427722    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:17:16.433583 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427724    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:17:16.433583 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427727    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:17:16.433583 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427730    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:17:16.433583 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427732    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:17:16.433583 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427735    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:17:16.433583 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427738    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:17:16.434159 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427740    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:17:16.434159 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427743    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:17:16.434159 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427746    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:17:16.434159 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427749    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:17:16.434159 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427752    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:17:16.434159 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427756    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:17:16.434159 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427758    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:17:16.434159 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427761    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:17:16.434159 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427764    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:17:16.434159 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427766    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:17:16.434159 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427769    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:17:16.434159 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427771    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:17:16.434159 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427774    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:17:16.434159 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427776    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:17:16.434159 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427779    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:17:16.434159 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427781    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:17:16.434159 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427784    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:17:16.434159 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427787    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:17:16.434159 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427789    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:17:16.434159 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427792    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:17:16.434681 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427794    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:17:16.434681 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427797    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:17:16.434681 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427799    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:17:16.434681 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427802    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:17:16.434681 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427805    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:17:16.434681 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:16.427807    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:17:16.434681 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.427812    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 17:17:16.434681 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.427937    2562 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 24 17:17:16.434681 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.430005    2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 24 17:17:16.434681 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.431321    2562 server.go:1019] "Starting client certificate rotation"
Jul 24 17:17:16.434681 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.431426    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 17:17:16.434681 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.431460    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 17:17:16.434681 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.434470    2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 17:17:16.435699 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.435679    2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 17:17:16.445607 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.445587    2562 log.go:25] "Validated CRI v1 runtime API"
Jul 24 17:17:16.447919 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.447901    2562 log.go:25] "Validated CRI v1 image API"
Jul 24 17:17:16.450239 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.450212    2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 24 17:17:16.452556 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.452535    2562 fs.go:135] Filesystem UUIDs: map[6995c90a-9226-4432-88e9-5717f2cffaa2:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e3b0fae0-aba7-44ad-a32b-8664fa64dc9b:/dev/nvme0n1p3]
Jul 24 17:17:16.452651 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.452554    2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 24 17:17:16.458210 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.458188    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 17:17:16.458537 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.458416    2562 manager.go:216] Machine: {Timestamp:2026-07-24 17:17:16.456904562 +0000 UTC m=+0.334223765 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3081155 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20da118abf88c33a94b80b3abf650c SystemUUID:ec20da11-8abf-88c3-3a94-b80b3abf650c BootID:df22cb44-516f-4dab-971f-131367231c64 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:1f:2d:e8:dc:ff Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:1f:2d:e8:dc:ff Speed:0 Mtu:9001} {Name:ovs-system MacAddress:46:48:07:4d:fb:43 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 24 17:17:16.458537 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.458533    2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 24 17:17:16.458653 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.458627    2562 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 24 17:17:16.461646 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.461622    2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 24 17:17:16.461806 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.461646    2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-159.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 24 17:17:16.462415 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.462404    2562 topology_manager.go:138] "Creating topology manager with none policy"
Jul 24 17:17:16.462455 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.462418    2562 container_manager_linux.go:309] "Creating device plugin manager"
Jul 24 17:17:16.462455 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.462425    2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 17:17:16.463039 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.463029    2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 17:17:16.463905 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.463894    2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 24 17:17:16.466100 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.466064    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 17:17:16.466269 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.466257    2562 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 24 17:17:16.466324 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.466294    2562 kubelet.go:486] "Attempting to sync node with API server"
Jul 24 17:17:16.466324 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.466321    2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 24 17:17:16.466406 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.466341    2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 24 17:17:16.466406 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.466357    2562 kubelet.go:398] "Adding apiserver pod source"
Jul 24 17:17:16.466406 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.466378    2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 24 17:17:16.467022 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.467004    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 17:17:16.467106 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.467035    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 17:17:16.469842 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.469821    2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 24 17:17:16.471932 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.471916    2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 24 17:17:16.471989 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.471949    2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 24 17:17:16.473492 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.473479    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 24 17:17:16.473548 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.473498    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 24 17:17:16.473548 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.473505    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 24 17:17:16.473548 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.473514    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 24 17:17:16.473548 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.473527    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 24 17:17:16.473548 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.473533    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 24 17:17:16.473548 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.473541    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 24 17:17:16.473548 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.473546    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 24 17:17:16.473736 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.473553    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 24 17:17:16.473736 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.473559    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 24 17:17:16.473736 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.473568    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 24 17:17:16.473736 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.473573    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 24 17:17:16.474570 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.474560    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 24 17:17:16.474615 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.474572    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 24 17:17:16.478364 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.478349    2562 server.go:1268] "Started kubelet"
Jul 24 17:17:16.478473 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.478435    2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 24 17:17:16.478566 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.478469    2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 24 17:17:16.478669 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.478598    2562 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 24 17:17:16.479158 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.478856    2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-159.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 17:17:16.479158 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:16.478885    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-159.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 17:17:16.479158 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:16.478951    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 17:17:16.479310 ip-10-0-131-159 systemd[1]: Started Kubernetes Kubelet.
Jul 24 17:17:16.480894 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.480868    2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 24 17:17:16.481570 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.481546    2562 server.go:310] "Adding debug handlers to kubelet server"
Jul 24 17:17:16.484531 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:16.484507    2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 24 17:17:16.486702 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:16.485779    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-159.ec2.internal.18c54950481c65d2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-159.ec2.internal,UID:ip-10-0-131-159.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-159.ec2.internal,},FirstTimestamp:2026-07-24 17:17:16.47830165 +0000 UTC m=+0.355620853,LastTimestamp:2026-07-24 17:17:16.47830165 +0000 UTC m=+0.355620853,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-159.ec2.internal,}"
Jul 24 17:17:16.487529 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.487511    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 24 17:17:16.487590 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.487514    2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 24 17:17:16.487720 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:16.487699    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-159.ec2.internal\" not found"
Jul 24 17:17:16.488356 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.488333    2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 24 17:17:16.488356 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.488333    2562 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 24 17:17:16.488514 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.488365    2562 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 24 17:17:16.488514 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.488445    2562 reconstruct.go:97] "Volume reconstruction finished"
Jul 24 17:17:16.488514 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.488457    2562 reconciler.go:29] "Reconciler: start to sync state"
Jul 24 17:17:16.488856 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.488842    2562 factory.go:153] Registering CRI-O factory
Jul 24 17:17:16.488955 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.488938    2562 factory.go:223] Registration of the crio container factory successfully
Jul 24 17:17:16.489060 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.489014    2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 24 17:17:16.489060 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.489023    2562 factory.go:55] Registering systemd factory
Jul 24 17:17:16.489060 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.489029    2562 factory.go:223] Registration of the systemd container factory successfully
Jul 24 17:17:16.489060 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.489046    2562 factory.go:103] Registering Raw factory
Jul 24 17:17:16.489060 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.489056    2562 manager.go:1196] Started watching for new ooms in manager
Jul 24 17:17:16.489662 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.489648    2562 manager.go:349] Starting recovery of all containers
Jul 24 17:17:16.493933 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:16.493899    2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-159.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 24 17:17:16.494072 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:16.494043    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 17:17:16.504398 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.504367    2562 manager.go:354] Recovery completed
Jul 24 17:17:16.506014 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:16.505953    2562 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 24 17:17:16.509816 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.509799    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:17:16.511772 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.511746    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-159.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:17:16.511837 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.511783    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-159.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:17:16.511837 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.511794    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-159.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:17:16.512407 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.512391    2562 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 24 17:17:16.512407 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.512404    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:17:16.512506 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.512406    2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 24 17:17:16.512506 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.512436    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 17:17:16.513186 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.513172    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-159.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:17:16.513241 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.513196    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-159.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:17:16.513241 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.513206    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-159.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:17:16.513886 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:16.513809    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-159.ec2.internal.18c549504a1b082f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-159.ec2.internal,UID:ip-10-0-131-159.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-159.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-159.ec2.internal,},FirstTimestamp:2026-07-24 17:17:16.511766575 +0000 UTC m=+0.389085780,LastTimestamp:2026-07-24 17:17:16.511766575 +0000 UTC m=+0.389085780,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-159.ec2.internal,}"
Jul 24 17:17:16.514335 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.514322    2562 policy_none.go:49] "None policy: Start"
Jul 24 17:17:16.514410 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.514340    2562 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 24 17:17:16.514410 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.514354    2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 24 17:17:16.515457 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.515444    2562 policy_none.go:47] "Start"
Jul 24 17:17:16.517507 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.517488    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bq6gt"
Jul 24 17:17:16.523118 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:16.523018    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-159.ec2.internal.18c549504a1b5e26  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-159.ec2.internal,UID:ip-10-0-131-159.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-159.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-159.ec2.internal,},FirstTimestamp:2026-07-24 17:17:16.511788582 +0000 UTC m=+0.389107786,LastTimestamp:2026-07-24 17:17:16.511788582 +0000 UTC m=+0.389107786,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-159.ec2.internal,}"
Jul 24 17:17:16.525211 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:16.525142    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-159.ec2.internal.18c549504a1b8347  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-159.ec2.internal,UID:ip-10-0-131-159.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-131-159.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-131-159.ec2.internal,},FirstTimestamp:2026-07-24 17:17:16.511798087 +0000 UTC m=+0.389117290,LastTimestamp:2026-07-24 17:17:16.511798087 +0000 UTC m=+0.389117290,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-159.ec2.internal,}"
Jul 24 17:17:16.527277 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.527252    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bq6gt"
Jul 24 17:17:16.531605 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:16.531533    2562 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-131-159.ec2.internal.18c549504a1b082f\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-159.ec2.internal.18c549504a1b082f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-159.ec2.internal,UID:ip-10-0-131-159.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-159.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-159.ec2.internal,},FirstTimestamp:2026-07-24 17:17:16.511766575 +0000 UTC m=+0.389085780,LastTimestamp:2026-07-24 17:17:16.513184858 +0000 UTC m=+0.390504062,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-159.ec2.internal,}"
Jul 24 17:17:16.541672 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.541646    2562 manager.go:337] "Starting Device Plugin manager"
Jul 24 17:17:16.541808 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:16.541705    2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 24 17:17:16.541808 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.541718    2562 server.go:87] "Starting device plugin registration server"
Jul 24 17:17:16.542066 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.542051    2562 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 24 17:17:16.542142 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.542068    2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 24 17:17:16.542263 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.542249    2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 24 17:17:16.542343 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.542331    2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 24 17:17:16.542395 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.542343    2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 24 17:17:16.542946 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:16.542918    2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 24 17:17:16.543034 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:16.542969    2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-159.ec2.internal\" not found"
Jul 24 17:17:16.554478 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.554454    2562 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2595 into the system container \"/system.slice\": <nil>"
Jul 24 17:17:16.642555 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.642522    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:17:16.649917 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.649892    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-159.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:17:16.650075 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.649929    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-159.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:17:16.650075 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.649951    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-159.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:17:16.650075 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.649999    2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-159.ec2.internal"
Jul 24 17:17:16.658739 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.658678    2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-159.ec2.internal"
Jul 24 17:17:16.658739 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:16.658709    2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-159.ec2.internal\": node \"ip-10-0-131-159.ec2.internal\" not found"
Jul 24 17:17:16.666337 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.666287    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 24 17:17:16.667638 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.667609    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 24 17:17:16.667763 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.667649    2562 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 24 17:17:16.667763 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.667679    2562 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 24 17:17:16.667763 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:16.667725    2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 24 17:17:16.670041 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.670021    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:17:16.686884 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:16.686702    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-159.ec2.internal\" not found"
Jul 24 17:17:16.768623 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.768576    2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-159.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-159.ec2.internal"]
Jul 24 17:17:16.768731 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.768669    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:17:16.770515 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.770499    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-159.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:17:16.770582 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.770532    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-159.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:17:16.770582 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.770545    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-159.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:17:16.772124 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.772112    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:17:16.772254 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.772237    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-159.ec2.internal"
Jul 24 17:17:16.772317 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.772274    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:17:16.772992 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.772959    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-159.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:17:16.773068 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.773007    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-159.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:17:16.773068 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.773018    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-159.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:17:16.773068 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.772959    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-159.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:17:16.773208 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.773079    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-159.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:17:16.773208 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.773093    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-159.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:17:16.774540 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.774522    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-159.ec2.internal"
Jul 24 17:17:16.774635 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.774548    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:17:16.775403 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.775385    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-159.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:17:16.775491 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.775414    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-159.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:17:16.775491 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.775424    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-159.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:17:16.787501 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:16.787481    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-159.ec2.internal\" not found"
Jul 24 17:17:16.790773 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.790742    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e423ebe6f419e89ee2be056e50f4506e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-159.ec2.internal\" (UID: \"e423ebe6f419e89ee2be056e50f4506e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-159.ec2.internal"
Jul 24 17:17:16.790839 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.790781    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e423ebe6f419e89ee2be056e50f4506e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-159.ec2.internal\" (UID: \"e423ebe6f419e89ee2be056e50f4506e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-159.ec2.internal"
Jul 24 17:17:16.790839 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.790800    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d4dd242ec3a9586b4b7071e62b5c988d-config\") pod \"kube-apiserver-proxy-ip-10-0-131-159.ec2.internal\" (UID: \"d4dd242ec3a9586b4b7071e62b5c988d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-159.ec2.internal"
Jul 24 17:17:16.806739 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:16.806720    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-159.ec2.internal\" not found" node="ip-10-0-131-159.ec2.internal"
Jul 24 17:17:16.811010 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:16.810994    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-159.ec2.internal\" not found" node="ip-10-0-131-159.ec2.internal"
Jul 24 17:17:16.888523 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:16.888485    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-159.ec2.internal\" not found"
Jul 24 17:17:16.891703 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.891688    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d4dd242ec3a9586b4b7071e62b5c988d-config\") pod \"kube-apiserver-proxy-ip-10-0-131-159.ec2.internal\" (UID: \"d4dd242ec3a9586b4b7071e62b5c988d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-159.ec2.internal"
Jul 24 17:17:16.891783 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.891748    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e423ebe6f419e89ee2be056e50f4506e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-159.ec2.internal\" (UID: \"e423ebe6f419e89ee2be056e50f4506e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-159.ec2.internal"
Jul 24 17:17:16.891828 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.891778    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d4dd242ec3a9586b4b7071e62b5c988d-config\") pod \"kube-apiserver-proxy-ip-10-0-131-159.ec2.internal\" (UID: \"d4dd242ec3a9586b4b7071e62b5c988d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-159.ec2.internal"
Jul 24 17:17:16.891828 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.891712    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e423ebe6f419e89ee2be056e50f4506e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-159.ec2.internal\" (UID: \"e423ebe6f419e89ee2be056e50f4506e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-159.ec2.internal"
Jul 24 17:17:16.891899 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.891833    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e423ebe6f419e89ee2be056e50f4506e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-159.ec2.internal\" (UID: \"e423ebe6f419e89ee2be056e50f4506e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-159.ec2.internal"
Jul 24 17:17:16.891899 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:16.891853    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e423ebe6f419e89ee2be056e50f4506e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-159.ec2.internal\" (UID: \"e423ebe6f419e89ee2be056e50f4506e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-159.ec2.internal"
Jul 24 17:17:16.989372 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:16.989328    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-159.ec2.internal\" not found"
Jul 24 17:17:17.089776 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:17.089744    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-159.ec2.internal\" not found"
Jul 24 17:17:17.109094 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:17.109067    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-159.ec2.internal"
Jul 24 17:17:17.114141 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:17.114124    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-159.ec2.internal"
Jul 24 17:17:17.189873 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:17.189829    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-159.ec2.internal\" not found"
Jul 24 17:17:17.290508 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:17.290429    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-159.ec2.internal\" not found"
Jul 24 17:17:18.541237 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:18.541196    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-159.ec2.internal\" not found"
Jul 24 17:17:18.580514 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:18.580474    2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 24 17:17:18.581030 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:18.580677    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 17:17:18.641720 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:18.641683    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-159.ec2.internal\" not found"
Jul 24 17:17:18.681635 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:18.681587    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 17:12:17 +0000 UTC" deadline="2027-12-30 07:36:02.076066064 +0000 UTC"
Jul 24 17:17:18.681782 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:18.681644    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12566h18m43.39443629s"
Jul 24 17:17:18.742117 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:18.742084    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-159.ec2.internal\" not found"
Jul 24 17:17:18.827964 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:18.827922    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode423ebe6f419e89ee2be056e50f4506e.slice/crio-6eda86dee0cf40b59527bc7c6a7d9c424f2bbe8bd710ff655ffdde0fda0e2806 WatchSource:0}: Error finding container 6eda86dee0cf40b59527bc7c6a7d9c424f2bbe8bd710ff655ffdde0fda0e2806: Status 404 returned error can't find the container with id 6eda86dee0cf40b59527bc7c6a7d9c424f2bbe8bd710ff655ffdde0fda0e2806
Jul 24 17:17:18.828212 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:18.828195    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4dd242ec3a9586b4b7071e62b5c988d.slice/crio-66802fe9ec3d958a45e4563cc8f66d81eae2232282c21d86cccf3c217208107f WatchSource:0}: Error finding container 66802fe9ec3d958a45e4563cc8f66d81eae2232282c21d86cccf3c217208107f: Status 404 returned error can't find the container with id 66802fe9ec3d958a45e4563cc8f66d81eae2232282c21d86cccf3c217208107f
Jul 24 17:17:18.832967 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:18.832952    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 17:17:18.842429 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:18.842395    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-159.ec2.internal\" not found"
Jul 24 17:17:18.843156 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:18.843123    2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:17:18.846340 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:18.846318    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:17:18.847450 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:18.847430    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-159.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:17:18.847572 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:18.847467    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-159.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:17:18.847572 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:18.847482    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-159.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:17:18.942831 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:18.942798    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-159.ec2.internal\" not found"
Jul 24 17:17:19.043499 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:19.043408    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-159.ec2.internal\" not found"
Jul 24 17:17:19.072169 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.072134    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:17:19.143736 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:19.143700    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-159.ec2.internal\" not found"
Jul 24 17:17:19.157505 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.157482    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:17:19.238676 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.238645    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-159.ec2.internal"
Jul 24 17:17:19.256361 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.256331    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 17:17:19.257999 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.257982    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-159.ec2.internal"
Jul 24 17:17:19.267414 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.267390    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 17:17:19.414297 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.414263    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:17:19.618085 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.618044    2562 apiserver.go:52] "Watching apiserver"
Jul 24 17:17:19.624906 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.624873    2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 24 17:17:19.627666 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.627632    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-z2whv","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7","openshift-dns/node-resolver-h2fn4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-159.ec2.internal","openshift-multus/multus-additional-cni-plugins-45mn5","openshift-network-diagnostics/network-check-target-k9hdw","openshift-ovn-kubernetes/ovnkube-node-kwq9x","kube-system/global-pull-secret-syncer-ts5kp","kube-system/kube-apiserver-proxy-ip-10-0-131-159.ec2.internal","openshift-cluster-node-tuning-operator/tuned-bbzvr","openshift-image-registry/node-ca-tdnbn","openshift-multus/multus-m77tg","openshift-multus/network-metrics-daemon-pjjdg","openshift-network-operator/iptables-alerter-b2t2r"]
Jul 24 17:17:19.629288 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.629257    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:17:19.629737 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:19.629703    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9hdw" podUID="179dc084-0504-4fcf-9608-07d598c9091c"
Jul 24 17:17:19.634732 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.634702    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:19.634863 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:19.634801    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjjdg" podUID="b87310bf-1f99-43b7-9731-29966b07be99"
Jul 24 17:17:19.637208 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.637179    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.639479 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.638869    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.639479 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.639423    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 24 17:17:19.640788 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.640334    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 24 17:17:19.640788 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.640440    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vq2kj\""
Jul 24 17:17:19.640788 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.640334    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 24 17:17:19.640788 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.640566    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 24 17:17:19.640788 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.640662    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 24 17:17:19.640788 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.640751    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-45mn5"
Jul 24 17:17:19.641378 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.641349    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 24 17:17:19.641475 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.641451    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 24 17:17:19.641537 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.641357    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 24 17:17:19.641717 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.641701    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bcqhl\""
Jul 24 17:17:19.641784 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.641727    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 24 17:17:19.641977 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.641960    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 24 17:17:19.642236 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.642220    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.643898 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.642882    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 24 17:17:19.643898 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.643321    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mcqph\""
Jul 24 17:17:19.645211 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.644447    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:19.645211 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:19.644519    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ts5kp" podUID="602851eb-593f-4245-9365-97ef80e3eb51"
Jul 24 17:17:19.645454 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.645387    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-z2whv"
Jul 24 17:17:19.647374 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.646167    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 24 17:17:19.647374 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.646291    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-bx54w\""
Jul 24 17:17:19.647374 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.646602    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 24 17:17:19.647374 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.646620    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 24 17:17:19.647374 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.646828    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 24 17:17:19.647374 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.647058    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tdnbn"
Jul 24 17:17:19.648841 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.647950    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-h2fn4"
Jul 24 17:17:19.648841 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.648024    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-98grh\""
Jul 24 17:17:19.648841 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.648375    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 24 17:17:19.648841 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.648593    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 24 17:17:19.650131 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.649059    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.650131 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.649114    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87cdg\" (UniqueName: \"kubernetes.io/projected/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-kube-api-access-87cdg\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.650131 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.649173    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccz6p\" (UniqueName: \"kubernetes.io/projected/179dc084-0504-4fcf-9608-07d598c9091c-kube-api-access-ccz6p\") pod \"network-check-target-k9hdw\" (UID: \"179dc084-0504-4fcf-9608-07d598c9091c\") " pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:17:19.650131 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.649203    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-var-lib-openvswitch\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.650131 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.649227    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-multus-cni-dir\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.650131 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.649251    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-host-run-netns\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.650131 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.649274    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-hostroot\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.650131 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.649313    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5cvk\" (UniqueName: \"kubernetes.io/projected/11f4dbdb-7967-407a-8572-8ba5dd91e233-kube-api-access-d5cvk\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.650131 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.649351    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-host-kubelet\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.650131 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.649379    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-ovn-node-metrics-cert\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.650131 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.649422    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/15f25938-a5da-432f-91f3-b40dae0912d3-cnibin\") pod \"multus-additional-cni-plugins-45mn5\" (UID: \"15f25938-a5da-432f-91f3-b40dae0912d3\") " pod="openshift-multus/multus-additional-cni-plugins-45mn5"
Jul 24 17:17:19.650131 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.649455    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-run-systemd\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.650131 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.649484    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-os-release\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.650131 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.649563    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-multus-socket-dir-parent\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.650131 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.649603    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/11f4dbdb-7967-407a-8572-8ba5dd91e233-multus-daemon-config\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.650131 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.649645    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-host-slash\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.650969 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.649677    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-run-openvswitch\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.650969 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.649757    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-host-run-netns\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.650969 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.649797    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-system-cni-dir\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.650969 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.649828    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-systemd-units\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.650969 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.649866    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-log-socket\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.650969 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.649887    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-host-cni-netd\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.650969 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.649909    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-env-overrides\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.650969 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.649999    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-ovnkube-script-lib\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.650969 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.650029    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/15f25938-a5da-432f-91f3-b40dae0912d3-cni-binary-copy\") pod \"multus-additional-cni-plugins-45mn5\" (UID: \"15f25938-a5da-432f-91f3-b40dae0912d3\") " pod="openshift-multus/multus-additional-cni-plugins-45mn5"
Jul 24 17:17:19.650969 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.650060    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-host-cni-bin\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.650969 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.650092    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/15f25938-a5da-432f-91f3-b40dae0912d3-system-cni-dir\") pod \"multus-additional-cni-plugins-45mn5\" (UID: \"15f25938-a5da-432f-91f3-b40dae0912d3\") " pod="openshift-multus/multus-additional-cni-plugins-45mn5"
Jul 24 17:17:19.650969 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.650133    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/15f25938-a5da-432f-91f3-b40dae0912d3-os-release\") pod \"multus-additional-cni-plugins-45mn5\" (UID: \"15f25938-a5da-432f-91f3-b40dae0912d3\") " pod="openshift-multus/multus-additional-cni-plugins-45mn5"
Jul 24 17:17:19.650969 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.650183    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/15f25938-a5da-432f-91f3-b40dae0912d3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-45mn5\" (UID: \"15f25938-a5da-432f-91f3-b40dae0912d3\") " pod="openshift-multus/multus-additional-cni-plugins-45mn5"
Jul 24 17:17:19.650969 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.650235    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wz9q\" (UniqueName: \"kubernetes.io/projected/15f25938-a5da-432f-91f3-b40dae0912d3-kube-api-access-7wz9q\") pod \"multus-additional-cni-plugins-45mn5\" (UID: \"15f25938-a5da-432f-91f3-b40dae0912d3\") " pod="openshift-multus/multus-additional-cni-plugins-45mn5"
Jul 24 17:17:19.650969 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.650282    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/11f4dbdb-7967-407a-8572-8ba5dd91e233-cni-binary-copy\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.650969 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.650327    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-host-run-multus-certs\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.651815 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.650351    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-etc-openvswitch\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.651815 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.650379    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs\") pod \"network-metrics-daemon-pjjdg\" (UID: \"b87310bf-1f99-43b7-9731-29966b07be99\") " pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:19.651815 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.650401    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-cnibin\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.651815 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.650482    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-host-var-lib-cni-multus\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.651815 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.650508    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-run-ovn\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.651815 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.650584    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-node-log\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.651815 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.650617    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-host-run-ovn-kubernetes\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.651815 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.650644    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-ovnkube-config\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.651815 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.650669    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/15f25938-a5da-432f-91f3-b40dae0912d3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-45mn5\" (UID: \"15f25938-a5da-432f-91f3-b40dae0912d3\") " pod="openshift-multus/multus-additional-cni-plugins-45mn5"
Jul 24 17:17:19.651815 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.650752    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzhfg\" (UniqueName: \"kubernetes.io/projected/b87310bf-1f99-43b7-9731-29966b07be99-kube-api-access-nzhfg\") pod \"network-metrics-daemon-pjjdg\" (UID: \"b87310bf-1f99-43b7-9731-29966b07be99\") " pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:19.651815 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.650779    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-host-run-k8s-cni-cncf-io\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.651815 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.650804    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-host-var-lib-cni-bin\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.651815 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.650843    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 24 17:17:19.651815 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.650851    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-host-var-lib-kubelet\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.651815 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.650874    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-multus-conf-dir\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.651815 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.650910    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/15f25938-a5da-432f-91f3-b40dae0912d3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-45mn5\" (UID: \"15f25938-a5da-432f-91f3-b40dae0912d3\") " pod="openshift-multus/multus-additional-cni-plugins-45mn5"
Jul 24 17:17:19.651815 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.650938    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-etc-kubernetes\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.652628 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.650974    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 24 17:17:19.652628 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.651107    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 24 17:17:19.652628 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.651239    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-94ffq\""
Jul 24 17:17:19.652628 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.651302    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 24 17:17:19.652628 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.651304    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-j87t6\""
Jul 24 17:17:19.652628 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.651525    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 24 17:17:19.652628 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.651646    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7"
Jul 24 17:17:19.652628 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.652368    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b2t2r"
Jul 24 17:17:19.653987 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.653966    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 24 17:17:19.654088 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.654051    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 24 17:17:19.654357 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.654339    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 24 17:17:19.654705 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.654688    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 24 17:17:19.655156 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.655126    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 24 17:17:19.655286 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.655270    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-599f8\""
Jul 24 17:17:19.656162 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.655816    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8l7jr\""
Jul 24 17:17:19.656162 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.655869    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 24 17:17:19.658555 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.658538    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 17:17:19.681919 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.681887    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-47v2p"
Jul 24 17:17:19.691411 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.691383    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-47v2p"
Jul 24 17:17:19.739823 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.739748    2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 24 17:17:19.751980 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.751938    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-multus-conf-dir\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.752186 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.751991    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/41b7767d-e084-4b00-8168-2d5f2e59c70a-socket-dir\") pod \"aws-ebs-csi-driver-node-mc5g7\" (UID: \"41b7767d-e084-4b00-8168-2d5f2e59c70a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7"
Jul 24 17:17:19.752186 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.752021    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-etc-kubernetes\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.752186 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.752047    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.752186 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.752057    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-multus-conf-dir\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.752186 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.752073    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87cdg\" (UniqueName: \"kubernetes.io/projected/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-kube-api-access-87cdg\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.752186 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.752097    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-etc-systemd\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.752186 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.752105    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-etc-kubernetes\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.752186 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.752135    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.752186 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.752163    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-run\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.752642 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.752282    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-hostroot\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.752642 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.752327    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5cvk\" (UniqueName: \"kubernetes.io/projected/11f4dbdb-7967-407a-8572-8ba5dd91e233-kube-api-access-d5cvk\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.752642 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.752356    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-host-kubelet\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.752642 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.752434    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wz9q\" (UniqueName: \"kubernetes.io/projected/15f25938-a5da-432f-91f3-b40dae0912d3-kube-api-access-7wz9q\") pod \"multus-additional-cni-plugins-45mn5\" (UID: \"15f25938-a5da-432f-91f3-b40dae0912d3\") " pod="openshift-multus/multus-additional-cni-plugins-45mn5"
Jul 24 17:17:19.752642 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.752464    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-etc-modprobe-d\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.752642 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.752498    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab25ce71-a92b-4f56-b9df-2c2812e744ef-tmp-dir\") pod \"node-resolver-h2fn4\" (UID: \"ab25ce71-a92b-4f56-b9df-2c2812e744ef\") " pod="openshift-dns/node-resolver-h2fn4"
Jul 24 17:17:19.752642 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.752543    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-host-kubelet\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.752642 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.752578    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e729f7f5-5dac-4c69-a970-d6aaa51cf093-host-slash\") pod \"iptables-alerter-b2t2r\" (UID: \"e729f7f5-5dac-4c69-a970-d6aaa51cf093\") " pod="openshift-network-operator/iptables-alerter-b2t2r"
Jul 24 17:17:19.752642 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.752625    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-multus-socket-dir-parent\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.753246 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.752692    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-run-openvswitch\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.753246 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.752730    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-hostroot\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.753246 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.752775    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-etc-sysctl-d\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.753246 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.752808    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-run-openvswitch\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.753246 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.752810    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-multus-socket-dir-parent\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.753246 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.752868    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-sys\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.753246 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.752902    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-lib-modules\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.753246 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.752947    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-host-run-netns\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.753246 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.752977    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-system-cni-dir\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.753246 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753003    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-systemd-units\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.753246 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753035    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-host-cni-netd\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.753246 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753061    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhchq\" (UniqueName: \"kubernetes.io/projected/b26dc7a6-6666-4814-b461-c9edad3dc705-kube-api-access-bhchq\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.753246 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753088    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-system-cni-dir\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.753246 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753096    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/602851eb-593f-4245-9365-97ef80e3eb51-original-pull-secret\") pod \"global-pull-secret-syncer-ts5kp\" (UID: \"602851eb-593f-4245-9365-97ef80e3eb51\") " pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:19.753246 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753135    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/15f25938-a5da-432f-91f3-b40dae0912d3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-45mn5\" (UID: \"15f25938-a5da-432f-91f3-b40dae0912d3\") " pod="openshift-multus/multus-additional-cni-plugins-45mn5"
Jul 24 17:17:19.753246 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753166    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-host-run-netns\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.753246 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753179    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-etc-openvswitch\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.753743 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753205    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/44c623e2-f2fa-418d-986a-af9370891a35-host\") pod \"node-ca-tdnbn\" (UID: \"44c623e2-f2fa-418d-986a-af9370891a35\") " pod="openshift-image-registry/node-ca-tdnbn"
Jul 24 17:17:19.753743 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753203    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-systemd-units\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.753743 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753265    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-host-cni-netd\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.753743 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753268    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/41b7767d-e084-4b00-8168-2d5f2e59c70a-registration-dir\") pod \"aws-ebs-csi-driver-node-mc5g7\" (UID: \"41b7767d-e084-4b00-8168-2d5f2e59c70a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7"
Jul 24 17:17:19.753743 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753329    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-etc-openvswitch\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.753743 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753392    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e729f7f5-5dac-4c69-a970-d6aaa51cf093-iptables-alerter-script\") pod \"iptables-alerter-b2t2r\" (UID: \"e729f7f5-5dac-4c69-a970-d6aaa51cf093\") " pod="openshift-network-operator/iptables-alerter-b2t2r"
Jul 24 17:17:19.753743 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753424    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs\") pod \"network-metrics-daemon-pjjdg\" (UID: \"b87310bf-1f99-43b7-9731-29966b07be99\") " pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:19.753743 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753431    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/15f25938-a5da-432f-91f3-b40dae0912d3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-45mn5\" (UID: \"15f25938-a5da-432f-91f3-b40dae0912d3\") " pod="openshift-multus/multus-additional-cni-plugins-45mn5"
Jul 24 17:17:19.753743 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753451    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-run-ovn\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.753743 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753476    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-node-log\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.753743 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753501    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-host-run-ovn-kubernetes\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.753743 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753529    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzpp8\" (UniqueName: \"kubernetes.io/projected/44c623e2-f2fa-418d-986a-af9370891a35-kube-api-access-tzpp8\") pod \"node-ca-tdnbn\" (UID: \"44c623e2-f2fa-418d-986a-af9370891a35\") " pod="openshift-image-registry/node-ca-tdnbn"
Jul 24 17:17:19.753743 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:19.753534    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:17:19.753743 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753554    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-etc-sysctl-conf\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.753743 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753580    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1a0c97c1-3fee-45af-bc53-36991e313aa8-agent-certs\") pod \"konnectivity-agent-z2whv\" (UID: \"1a0c97c1-3fee-45af-bc53-36991e313aa8\") " pod="kube-system/konnectivity-agent-z2whv"
Jul 24 17:17:19.753743 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:19.753602    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs podName:b87310bf-1f99-43b7-9731-29966b07be99 nodeName:}" failed. No retries permitted until 2026-07-24 17:17:20.253580269 +0000 UTC m=+2.980736477 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs") pod "network-metrics-daemon-pjjdg" (UID: "b87310bf-1f99-43b7-9731-29966b07be99") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:17:19.753743 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753627    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-run-ovn\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.754479 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753668    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-node-log\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.754479 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753707    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-host-run-ovn-kubernetes\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.754479 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753761    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1a0c97c1-3fee-45af-bc53-36991e313aa8-konnectivity-ca\") pod \"konnectivity-agent-z2whv\" (UID: \"1a0c97c1-3fee-45af-bc53-36991e313aa8\") " pod="kube-system/konnectivity-agent-z2whv"
Jul 24 17:17:19.754479 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753789    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzhfg\" (UniqueName: \"kubernetes.io/projected/b87310bf-1f99-43b7-9731-29966b07be99-kube-api-access-nzhfg\") pod \"network-metrics-daemon-pjjdg\" (UID: \"b87310bf-1f99-43b7-9731-29966b07be99\") " pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:19.754479 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753826    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-host-var-lib-kubelet\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.754479 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753872    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-host-var-lib-kubelet\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.754479 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.753977    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/41b7767d-e084-4b00-8168-2d5f2e59c70a-sys-fs\") pod \"aws-ebs-csi-driver-node-mc5g7\" (UID: \"41b7767d-e084-4b00-8168-2d5f2e59c70a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7"
Jul 24 17:17:19.754479 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754028    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccz6p\" (UniqueName: \"kubernetes.io/projected/179dc084-0504-4fcf-9608-07d598c9091c-kube-api-access-ccz6p\") pod \"network-check-target-k9hdw\" (UID: \"179dc084-0504-4fcf-9608-07d598c9091c\") " pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:17:19.754479 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754058    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/602851eb-593f-4245-9365-97ef80e3eb51-kubelet-config\") pod \"global-pull-secret-syncer-ts5kp\" (UID: \"602851eb-593f-4245-9365-97ef80e3eb51\") " pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:19.754479 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754166    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-var-lib-openvswitch\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.754479 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754193    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-multus-cni-dir\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.754479 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754215    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-var-lib-openvswitch\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.754479 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754233    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-host-run-netns\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.754479 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754255    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-ovn-node-metrics-cert\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.754479 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754275    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/15f25938-a5da-432f-91f3-b40dae0912d3-cnibin\") pod \"multus-additional-cni-plugins-45mn5\" (UID: \"15f25938-a5da-432f-91f3-b40dae0912d3\") " pod="openshift-multus/multus-additional-cni-plugins-45mn5"
Jul 24 17:17:19.754479 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754348    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-host-run-netns\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.754479 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754364    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-multus-cni-dir\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.755219 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754400    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-var-lib-kubelet\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.755219 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754420    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/15f25938-a5da-432f-91f3-b40dae0912d3-cnibin\") pod \"multus-additional-cni-plugins-45mn5\" (UID: \"15f25938-a5da-432f-91f3-b40dae0912d3\") " pod="openshift-multus/multus-additional-cni-plugins-45mn5"
Jul 24 17:17:19.755219 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754430    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b26dc7a6-6666-4814-b461-c9edad3dc705-etc-tuned\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.755219 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754459    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-run-systemd\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.755219 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754484    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-os-release\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.755219 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754510    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/11f4dbdb-7967-407a-8572-8ba5dd91e233-multus-daemon-config\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.755219 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754535    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-host-slash\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.755219 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754557    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-log-socket\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.755219 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754586    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-os-release\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.755219 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754599    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-host-slash\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.755219 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754602    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-log-socket\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.755219 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754631    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/15f25938-a5da-432f-91f3-b40dae0912d3-cni-binary-copy\") pod \"multus-additional-cni-plugins-45mn5\" (UID: \"15f25938-a5da-432f-91f3-b40dae0912d3\") " pod="openshift-multus/multus-additional-cni-plugins-45mn5"
Jul 24 17:17:19.755219 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754622    2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 24 17:17:19.755219 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754664    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/44c623e2-f2fa-418d-986a-af9370891a35-serviceca\") pod \"node-ca-tdnbn\" (UID: \"44c623e2-f2fa-418d-986a-af9370891a35\") " pod="openshift-image-registry/node-ca-tdnbn"
Jul 24 17:17:19.755219 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754535    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-run-systemd\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.755219 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754694    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-env-overrides\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.755219 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754721    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-ovnkube-script-lib\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.755219 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754743    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-etc-sysconfig\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.755923 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754764    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-etc-kubernetes\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.755923 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754786    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-host\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.755923 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754809    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ab25ce71-a92b-4f56-b9df-2c2812e744ef-hosts-file\") pod \"node-resolver-h2fn4\" (UID: \"ab25ce71-a92b-4f56-b9df-2c2812e744ef\") " pod="openshift-dns/node-resolver-h2fn4"
Jul 24 17:17:19.755923 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754832    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/41b7767d-e084-4b00-8168-2d5f2e59c70a-etc-selinux\") pod \"aws-ebs-csi-driver-node-mc5g7\" (UID: \"41b7767d-e084-4b00-8168-2d5f2e59c70a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7"
Jul 24 17:17:19.755923 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754857    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-host-cni-bin\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.755923 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754881    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/15f25938-a5da-432f-91f3-b40dae0912d3-system-cni-dir\") pod \"multus-additional-cni-plugins-45mn5\" (UID: \"15f25938-a5da-432f-91f3-b40dae0912d3\") " pod="openshift-multus/multus-additional-cni-plugins-45mn5"
Jul 24 17:17:19.755923 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754914    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/15f25938-a5da-432f-91f3-b40dae0912d3-os-release\") pod \"multus-additional-cni-plugins-45mn5\" (UID: \"15f25938-a5da-432f-91f3-b40dae0912d3\") " pod="openshift-multus/multus-additional-cni-plugins-45mn5"
Jul 24 17:17:19.755923 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754937    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chhjb\" (UniqueName: \"kubernetes.io/projected/ab25ce71-a92b-4f56-b9df-2c2812e744ef-kube-api-access-chhjb\") pod \"node-resolver-h2fn4\" (UID: \"ab25ce71-a92b-4f56-b9df-2c2812e744ef\") " pod="openshift-dns/node-resolver-h2fn4"
Jul 24 17:17:19.755923 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754964    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/41b7767d-e084-4b00-8168-2d5f2e59c70a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mc5g7\" (UID: \"41b7767d-e084-4b00-8168-2d5f2e59c70a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7"
Jul 24 17:17:19.755923 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.754987    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/11f4dbdb-7967-407a-8572-8ba5dd91e233-cni-binary-copy\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.755923 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.755012    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-host-run-multus-certs\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.755923 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.755035    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b26dc7a6-6666-4814-b461-c9edad3dc705-tmp\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.755923 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.755083    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/41b7767d-e084-4b00-8168-2d5f2e59c70a-device-dir\") pod \"aws-ebs-csi-driver-node-mc5g7\" (UID: \"41b7767d-e084-4b00-8168-2d5f2e59c70a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7"
Jul 24 17:17:19.755923 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.755102    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/11f4dbdb-7967-407a-8572-8ba5dd91e233-multus-daemon-config\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.755923 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.755107    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7pkh\" (UniqueName: \"kubernetes.io/projected/41b7767d-e084-4b00-8168-2d5f2e59c70a-kube-api-access-q7pkh\") pod \"aws-ebs-csi-driver-node-mc5g7\" (UID: \"41b7767d-e084-4b00-8168-2d5f2e59c70a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7"
Jul 24 17:17:19.755923 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.755168    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-cnibin\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.755923 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.755198    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-host-var-lib-cni-multus\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.756715 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.755223    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-ovnkube-config\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.756715 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.755248    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/15f25938-a5da-432f-91f3-b40dae0912d3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-45mn5\" (UID: \"15f25938-a5da-432f-91f3-b40dae0912d3\") " pod="openshift-multus/multus-additional-cni-plugins-45mn5"
Jul 24 17:17:19.756715 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.755255    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-host-cni-bin\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.756715 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.755274    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/15f25938-a5da-432f-91f3-b40dae0912d3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-45mn5\" (UID: \"15f25938-a5da-432f-91f3-b40dae0912d3\") " pod="openshift-multus/multus-additional-cni-plugins-45mn5"
Jul 24 17:17:19.756715 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.755279    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/15f25938-a5da-432f-91f3-b40dae0912d3-cni-binary-copy\") pod \"multus-additional-cni-plugins-45mn5\" (UID: \"15f25938-a5da-432f-91f3-b40dae0912d3\") " pod="openshift-multus/multus-additional-cni-plugins-45mn5"
Jul 24 17:17:19.756715 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.755301    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/602851eb-593f-4245-9365-97ef80e3eb51-dbus\") pod \"global-pull-secret-syncer-ts5kp\" (UID: \"602851eb-593f-4245-9365-97ef80e3eb51\") " pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:19.756715 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.755305    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/15f25938-a5da-432f-91f3-b40dae0912d3-system-cni-dir\") pod \"multus-additional-cni-plugins-45mn5\" (UID: \"15f25938-a5da-432f-91f3-b40dae0912d3\") " pod="openshift-multus/multus-additional-cni-plugins-45mn5"
Jul 24 17:17:19.756715 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.755341    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g482b\" (UniqueName: \"kubernetes.io/projected/e729f7f5-5dac-4c69-a970-d6aaa51cf093-kube-api-access-g482b\") pod \"iptables-alerter-b2t2r\" (UID: \"e729f7f5-5dac-4c69-a970-d6aaa51cf093\") " pod="openshift-network-operator/iptables-alerter-b2t2r"
Jul 24 17:17:19.756715 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.755366    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-ovnkube-script-lib\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.756715 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.755413    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-host-var-lib-cni-multus\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.756715 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.755468    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/15f25938-a5da-432f-91f3-b40dae0912d3-os-release\") pod \"multus-additional-cni-plugins-45mn5\" (UID: \"15f25938-a5da-432f-91f3-b40dae0912d3\") " pod="openshift-multus/multus-additional-cni-plugins-45mn5"
Jul 24 17:17:19.756715 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.755674    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-env-overrides\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.756715 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.755876    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/15f25938-a5da-432f-91f3-b40dae0912d3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-45mn5\" (UID: \"15f25938-a5da-432f-91f3-b40dae0912d3\") " pod="openshift-multus/multus-additional-cni-plugins-45mn5"
Jul 24 17:17:19.756715 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.755879    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/11f4dbdb-7967-407a-8572-8ba5dd91e233-cni-binary-copy\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.756715 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.755911    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-ovnkube-config\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.756715 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.755924    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-host-run-multus-certs\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.756715 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.755966    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-cnibin\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.757631 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.755975    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-host-run-k8s-cni-cncf-io\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.757631 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.756029    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-host-var-lib-cni-bin\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.757631 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.756031    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-host-run-k8s-cni-cncf-io\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.757631 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.756069    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/11f4dbdb-7967-407a-8572-8ba5dd91e233-host-var-lib-cni-bin\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.757631 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.756350    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/15f25938-a5da-432f-91f3-b40dae0912d3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-45mn5\" (UID: \"15f25938-a5da-432f-91f3-b40dae0912d3\") " pod="openshift-multus/multus-additional-cni-plugins-45mn5"
Jul 24 17:17:19.760670 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.760634    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-ovn-node-metrics-cert\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.761626 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:19.761339    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:17:19.761626 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:19.761363    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:17:19.761626 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:19.761376    2562 projected.go:196] Error preparing data for projected volume kube-api-access-ccz6p for pod openshift-network-diagnostics/network-check-target-k9hdw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:17:19.761626 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:19.761445    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/179dc084-0504-4fcf-9608-07d598c9091c-kube-api-access-ccz6p podName:179dc084-0504-4fcf-9608-07d598c9091c nodeName:}" failed. No retries permitted until 2026-07-24 17:17:20.261425108 +0000 UTC m=+2.988581314 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ccz6p" (UniqueName: "kubernetes.io/projected/179dc084-0504-4fcf-9608-07d598c9091c-kube-api-access-ccz6p") pod "network-check-target-k9hdw" (UID: "179dc084-0504-4fcf-9608-07d598c9091c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:17:19.764608 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.764581    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wz9q\" (UniqueName: \"kubernetes.io/projected/15f25938-a5da-432f-91f3-b40dae0912d3-kube-api-access-7wz9q\") pod \"multus-additional-cni-plugins-45mn5\" (UID: \"15f25938-a5da-432f-91f3-b40dae0912d3\") " pod="openshift-multus/multus-additional-cni-plugins-45mn5"
Jul 24 17:17:19.764747 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.764588    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87cdg\" (UniqueName: \"kubernetes.io/projected/2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a-kube-api-access-87cdg\") pod \"ovnkube-node-kwq9x\" (UID: \"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.765366 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.765340    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzhfg\" (UniqueName: \"kubernetes.io/projected/b87310bf-1f99-43b7-9731-29966b07be99-kube-api-access-nzhfg\") pod \"network-metrics-daemon-pjjdg\" (UID: \"b87310bf-1f99-43b7-9731-29966b07be99\") " pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:19.766251 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.766228    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5cvk\" (UniqueName: \"kubernetes.io/projected/11f4dbdb-7967-407a-8572-8ba5dd91e233-kube-api-access-d5cvk\") pod \"multus-m77tg\" (UID: \"11f4dbdb-7967-407a-8572-8ba5dd91e233\") " pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.825288 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.825187    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-159.ec2.internal" event={"ID":"d4dd242ec3a9586b4b7071e62b5c988d","Type":"ContainerStarted","Data":"66802fe9ec3d958a45e4563cc8f66d81eae2232282c21d86cccf3c217208107f"}
Jul 24 17:17:19.826698 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.826657    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-159.ec2.internal" event={"ID":"e423ebe6f419e89ee2be056e50f4506e","Type":"ContainerStarted","Data":"6eda86dee0cf40b59527bc7c6a7d9c424f2bbe8bd710ff655ffdde0fda0e2806"}
Jul 24 17:17:19.857116 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857079    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/41b7767d-e084-4b00-8168-2d5f2e59c70a-socket-dir\") pod \"aws-ebs-csi-driver-node-mc5g7\" (UID: \"41b7767d-e084-4b00-8168-2d5f2e59c70a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7"
Jul 24 17:17:19.857116 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857124    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-etc-systemd\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.857372 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857155    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-run\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.857372 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857173    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-etc-modprobe-d\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.857372 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857188    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab25ce71-a92b-4f56-b9df-2c2812e744ef-tmp-dir\") pod \"node-resolver-h2fn4\" (UID: \"ab25ce71-a92b-4f56-b9df-2c2812e744ef\") " pod="openshift-dns/node-resolver-h2fn4"
Jul 24 17:17:19.857372 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857202    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e729f7f5-5dac-4c69-a970-d6aaa51cf093-host-slash\") pod \"iptables-alerter-b2t2r\" (UID: \"e729f7f5-5dac-4c69-a970-d6aaa51cf093\") " pod="openshift-network-operator/iptables-alerter-b2t2r"
Jul 24 17:17:19.857372 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857225    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-etc-sysctl-d\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.857372 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857247    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-sys\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.857372 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857263    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-lib-modules\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.857372 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857279    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhchq\" (UniqueName: \"kubernetes.io/projected/b26dc7a6-6666-4814-b461-c9edad3dc705-kube-api-access-bhchq\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.857372 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857294    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/602851eb-593f-4245-9365-97ef80e3eb51-original-pull-secret\") pod \"global-pull-secret-syncer-ts5kp\" (UID: \"602851eb-593f-4245-9365-97ef80e3eb51\") " pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:19.857372 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857311    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/44c623e2-f2fa-418d-986a-af9370891a35-host\") pod \"node-ca-tdnbn\" (UID: \"44c623e2-f2fa-418d-986a-af9370891a35\") " pod="openshift-image-registry/node-ca-tdnbn"
Jul 24 17:17:19.857372 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857328    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/41b7767d-e084-4b00-8168-2d5f2e59c70a-registration-dir\") pod \"aws-ebs-csi-driver-node-mc5g7\" (UID: \"41b7767d-e084-4b00-8168-2d5f2e59c70a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7"
Jul 24 17:17:19.857372 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857347    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e729f7f5-5dac-4c69-a970-d6aaa51cf093-iptables-alerter-script\") pod \"iptables-alerter-b2t2r\" (UID: \"e729f7f5-5dac-4c69-a970-d6aaa51cf093\") " pod="openshift-network-operator/iptables-alerter-b2t2r"
Jul 24 17:17:19.857936 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857383    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzpp8\" (UniqueName: \"kubernetes.io/projected/44c623e2-f2fa-418d-986a-af9370891a35-kube-api-access-tzpp8\") pod \"node-ca-tdnbn\" (UID: \"44c623e2-f2fa-418d-986a-af9370891a35\") " pod="openshift-image-registry/node-ca-tdnbn"
Jul 24 17:17:19.857936 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857404    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-etc-sysctl-conf\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.857936 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857422    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1a0c97c1-3fee-45af-bc53-36991e313aa8-agent-certs\") pod \"konnectivity-agent-z2whv\" (UID: \"1a0c97c1-3fee-45af-bc53-36991e313aa8\") " pod="kube-system/konnectivity-agent-z2whv"
Jul 24 17:17:19.857936 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857436    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1a0c97c1-3fee-45af-bc53-36991e313aa8-konnectivity-ca\") pod \"konnectivity-agent-z2whv\" (UID: \"1a0c97c1-3fee-45af-bc53-36991e313aa8\") " pod="kube-system/konnectivity-agent-z2whv"
Jul 24 17:17:19.857936 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857452    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/41b7767d-e084-4b00-8168-2d5f2e59c70a-sys-fs\") pod \"aws-ebs-csi-driver-node-mc5g7\" (UID: \"41b7767d-e084-4b00-8168-2d5f2e59c70a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7"
Jul 24 17:17:19.857936 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857486    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/602851eb-593f-4245-9365-97ef80e3eb51-kubelet-config\") pod \"global-pull-secret-syncer-ts5kp\" (UID: \"602851eb-593f-4245-9365-97ef80e3eb51\") " pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:19.857936 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857512    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-var-lib-kubelet\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.857936 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857526    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b26dc7a6-6666-4814-b461-c9edad3dc705-etc-tuned\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.857936 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857549    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/44c623e2-f2fa-418d-986a-af9370891a35-serviceca\") pod \"node-ca-tdnbn\" (UID: \"44c623e2-f2fa-418d-986a-af9370891a35\") " pod="openshift-image-registry/node-ca-tdnbn"
Jul 24 17:17:19.857936 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857571    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-etc-sysconfig\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.857936 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857588    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-etc-kubernetes\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.857936 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857603    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-host\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.857936 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857627    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ab25ce71-a92b-4f56-b9df-2c2812e744ef-hosts-file\") pod \"node-resolver-h2fn4\" (UID: \"ab25ce71-a92b-4f56-b9df-2c2812e744ef\") " pod="openshift-dns/node-resolver-h2fn4"
Jul 24 17:17:19.857936 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857647    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/41b7767d-e084-4b00-8168-2d5f2e59c70a-etc-selinux\") pod \"aws-ebs-csi-driver-node-mc5g7\" (UID: \"41b7767d-e084-4b00-8168-2d5f2e59c70a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7"
Jul 24 17:17:19.857936 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857666    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chhjb\" (UniqueName: \"kubernetes.io/projected/ab25ce71-a92b-4f56-b9df-2c2812e744ef-kube-api-access-chhjb\") pod \"node-resolver-h2fn4\" (UID: \"ab25ce71-a92b-4f56-b9df-2c2812e744ef\") " pod="openshift-dns/node-resolver-h2fn4"
Jul 24 17:17:19.857936 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857683    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/41b7767d-e084-4b00-8168-2d5f2e59c70a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mc5g7\" (UID: \"41b7767d-e084-4b00-8168-2d5f2e59c70a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7"
Jul 24 17:17:19.857936 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857711    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b26dc7a6-6666-4814-b461-c9edad3dc705-tmp\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.858740 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857733    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/41b7767d-e084-4b00-8168-2d5f2e59c70a-device-dir\") pod \"aws-ebs-csi-driver-node-mc5g7\" (UID: \"41b7767d-e084-4b00-8168-2d5f2e59c70a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7"
Jul 24 17:17:19.858740 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857753    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7pkh\" (UniqueName: \"kubernetes.io/projected/41b7767d-e084-4b00-8168-2d5f2e59c70a-kube-api-access-q7pkh\") pod \"aws-ebs-csi-driver-node-mc5g7\" (UID: \"41b7767d-e084-4b00-8168-2d5f2e59c70a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7"
Jul 24 17:17:19.858740 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857771    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/602851eb-593f-4245-9365-97ef80e3eb51-dbus\") pod \"global-pull-secret-syncer-ts5kp\" (UID: \"602851eb-593f-4245-9365-97ef80e3eb51\") " pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:19.858740 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.857797    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g482b\" (UniqueName: \"kubernetes.io/projected/e729f7f5-5dac-4c69-a970-d6aaa51cf093-kube-api-access-g482b\") pod \"iptables-alerter-b2t2r\" (UID: \"e729f7f5-5dac-4c69-a970-d6aaa51cf093\") " pod="openshift-network-operator/iptables-alerter-b2t2r"
Jul 24 17:17:19.858740 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.858189    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-sys\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.858740 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.858327    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/41b7767d-e084-4b00-8168-2d5f2e59c70a-socket-dir\") pod \"aws-ebs-csi-driver-node-mc5g7\" (UID: \"41b7767d-e084-4b00-8168-2d5f2e59c70a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7"
Jul 24 17:17:19.858740 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.858383    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-etc-systemd\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.858740 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.858429    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-run\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.858740 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.858540    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-etc-modprobe-d\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.858740 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.858651    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1a0c97c1-3fee-45af-bc53-36991e313aa8-konnectivity-ca\") pod \"konnectivity-agent-z2whv\" (UID: \"1a0c97c1-3fee-45af-bc53-36991e313aa8\") " pod="kube-system/konnectivity-agent-z2whv"
Jul 24 17:17:19.858740 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.858704    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e729f7f5-5dac-4c69-a970-d6aaa51cf093-host-slash\") pod \"iptables-alerter-b2t2r\" (UID: \"e729f7f5-5dac-4c69-a970-d6aaa51cf093\") " pod="openshift-network-operator/iptables-alerter-b2t2r"
Jul 24 17:17:19.858740 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.858721    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/41b7767d-e084-4b00-8168-2d5f2e59c70a-sys-fs\") pod \"aws-ebs-csi-driver-node-mc5g7\" (UID: \"41b7767d-e084-4b00-8168-2d5f2e59c70a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7"
Jul 24 17:17:19.859318 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.858772    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/41b7767d-e084-4b00-8168-2d5f2e59c70a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mc5g7\" (UID: \"41b7767d-e084-4b00-8168-2d5f2e59c70a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7"
Jul 24 17:17:19.859318 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.858783    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ab25ce71-a92b-4f56-b9df-2c2812e744ef-hosts-file\") pod \"node-resolver-h2fn4\" (UID: \"ab25ce71-a92b-4f56-b9df-2c2812e744ef\") " pod="openshift-dns/node-resolver-h2fn4"
Jul 24 17:17:19.859318 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.858804    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab25ce71-a92b-4f56-b9df-2c2812e744ef-tmp-dir\") pod \"node-resolver-h2fn4\" (UID: \"ab25ce71-a92b-4f56-b9df-2c2812e744ef\") " pod="openshift-dns/node-resolver-h2fn4"
Jul 24 17:17:19.859318 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.858948    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-etc-sysctl-d\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.859318 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.858992    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/44c623e2-f2fa-418d-986a-af9370891a35-host\") pod \"node-ca-tdnbn\" (UID: \"44c623e2-f2fa-418d-986a-af9370891a35\") " pod="openshift-image-registry/node-ca-tdnbn"
Jul 24 17:17:19.859318 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.859056    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/602851eb-593f-4245-9365-97ef80e3eb51-kubelet-config\") pod \"global-pull-secret-syncer-ts5kp\" (UID: \"602851eb-593f-4245-9365-97ef80e3eb51\") " pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:19.859318 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.859084    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-lib-modules\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.859318 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.859112    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-var-lib-kubelet\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.859318 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:19.859271    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:17:19.859718 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:19.859337    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/602851eb-593f-4245-9365-97ef80e3eb51-original-pull-secret podName:602851eb-593f-4245-9365-97ef80e3eb51 nodeName:}" failed. No retries permitted until 2026-07-24 17:17:20.359316045 +0000 UTC m=+3.086472252 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/602851eb-593f-4245-9365-97ef80e3eb51-original-pull-secret") pod "global-pull-secret-syncer-ts5kp" (UID: "602851eb-593f-4245-9365-97ef80e3eb51") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:17:19.859718 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.859468    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/41b7767d-e084-4b00-8168-2d5f2e59c70a-device-dir\") pod \"aws-ebs-csi-driver-node-mc5g7\" (UID: \"41b7767d-e084-4b00-8168-2d5f2e59c70a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7"
Jul 24 17:17:19.859718 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.859535    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-etc-kubernetes\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.859879 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.859759    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-etc-sysconfig\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.859879 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.859819    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/41b7767d-e084-4b00-8168-2d5f2e59c70a-registration-dir\") pod \"aws-ebs-csi-driver-node-mc5g7\" (UID: \"41b7767d-e084-4b00-8168-2d5f2e59c70a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7"
Jul 24 17:17:19.859879 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.859872    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/602851eb-593f-4245-9365-97ef80e3eb51-dbus\") pod \"global-pull-secret-syncer-ts5kp\" (UID: \"602851eb-593f-4245-9365-97ef80e3eb51\") " pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:19.860031 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.859919    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-host\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.860031 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.859932    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b26dc7a6-6666-4814-b461-c9edad3dc705-etc-sysctl-conf\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.860031 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.859969    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e729f7f5-5dac-4c69-a970-d6aaa51cf093-iptables-alerter-script\") pod \"iptables-alerter-b2t2r\" (UID: \"e729f7f5-5dac-4c69-a970-d6aaa51cf093\") " pod="openshift-network-operator/iptables-alerter-b2t2r"
Jul 24 17:17:19.860031 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.860007    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/41b7767d-e084-4b00-8168-2d5f2e59c70a-etc-selinux\") pod \"aws-ebs-csi-driver-node-mc5g7\" (UID: \"41b7767d-e084-4b00-8168-2d5f2e59c70a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7"
Jul 24 17:17:19.860554 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.860528    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/44c623e2-f2fa-418d-986a-af9370891a35-serviceca\") pod \"node-ca-tdnbn\" (UID: \"44c623e2-f2fa-418d-986a-af9370891a35\") " pod="openshift-image-registry/node-ca-tdnbn"
Jul 24 17:17:19.861840 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.861797    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b26dc7a6-6666-4814-b461-c9edad3dc705-etc-tuned\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.862032 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.862009    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1a0c97c1-3fee-45af-bc53-36991e313aa8-agent-certs\") pod \"konnectivity-agent-z2whv\" (UID: \"1a0c97c1-3fee-45af-bc53-36991e313aa8\") " pod="kube-system/konnectivity-agent-z2whv"
Jul 24 17:17:19.862436 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.862415    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b26dc7a6-6666-4814-b461-c9edad3dc705-tmp\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.869289 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.869260    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g482b\" (UniqueName: \"kubernetes.io/projected/e729f7f5-5dac-4c69-a970-d6aaa51cf093-kube-api-access-g482b\") pod \"iptables-alerter-b2t2r\" (UID: \"e729f7f5-5dac-4c69-a970-d6aaa51cf093\") " pod="openshift-network-operator/iptables-alerter-b2t2r"
Jul 24 17:17:19.870494 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.870446    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chhjb\" (UniqueName: \"kubernetes.io/projected/ab25ce71-a92b-4f56-b9df-2c2812e744ef-kube-api-access-chhjb\") pod \"node-resolver-h2fn4\" (UID: \"ab25ce71-a92b-4f56-b9df-2c2812e744ef\") " pod="openshift-dns/node-resolver-h2fn4"
Jul 24 17:17:19.870597 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.870537    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7pkh\" (UniqueName: \"kubernetes.io/projected/41b7767d-e084-4b00-8168-2d5f2e59c70a-kube-api-access-q7pkh\") pod \"aws-ebs-csi-driver-node-mc5g7\" (UID: \"41b7767d-e084-4b00-8168-2d5f2e59c70a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7"
Jul 24 17:17:19.871972 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.871945    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzpp8\" (UniqueName: \"kubernetes.io/projected/44c623e2-f2fa-418d-986a-af9370891a35-kube-api-access-tzpp8\") pod \"node-ca-tdnbn\" (UID: \"44c623e2-f2fa-418d-986a-af9370891a35\") " pod="openshift-image-registry/node-ca-tdnbn"
Jul 24 17:17:19.872098 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.872075    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhchq\" (UniqueName: \"kubernetes.io/projected/b26dc7a6-6666-4814-b461-c9edad3dc705-kube-api-access-bhchq\") pod \"tuned-bbzvr\" (UID: \"b26dc7a6-6666-4814-b461-c9edad3dc705\") " pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.954678 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.954586    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-m77tg"
Jul 24 17:17:19.962799 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.962775    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:19.978031 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.977994    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-45mn5"
Jul 24 17:17:19.982960 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.982931    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bbzvr"
Jul 24 17:17:19.991763 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:19.991740    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-z2whv"
Jul 24 17:17:20.001661 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:20.001631    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tdnbn"
Jul 24 17:17:20.009574 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:20.009545    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-h2fn4"
Jul 24 17:17:20.018818 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:20.018788    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7"
Jul 24 17:17:20.027716 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:20.027688    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b2t2r"
Jul 24 17:17:20.261512 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:20.261417    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs\") pod \"network-metrics-daemon-pjjdg\" (UID: \"b87310bf-1f99-43b7-9731-29966b07be99\") " pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:20.261512 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:20.261477    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccz6p\" (UniqueName: \"kubernetes.io/projected/179dc084-0504-4fcf-9608-07d598c9091c-kube-api-access-ccz6p\") pod \"network-check-target-k9hdw\" (UID: \"179dc084-0504-4fcf-9608-07d598c9091c\") " pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:17:20.261681 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:20.261581    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:17:20.261681 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:20.261657    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:17:20.261681 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:20.261675    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:17:20.261806 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:20.261685    2562 projected.go:196] Error preparing data for projected volume kube-api-access-ccz6p for pod openshift-network-diagnostics/network-check-target-k9hdw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:17:20.261806 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:20.261665    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs podName:b87310bf-1f99-43b7-9731-29966b07be99 nodeName:}" failed. No retries permitted until 2026-07-24 17:17:21.261648974 +0000 UTC m=+3.988805168 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs") pod "network-metrics-daemon-pjjdg" (UID: "b87310bf-1f99-43b7-9731-29966b07be99") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:17:20.261806 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:20.261738    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/179dc084-0504-4fcf-9608-07d598c9091c-kube-api-access-ccz6p podName:179dc084-0504-4fcf-9608-07d598c9091c nodeName:}" failed. No retries permitted until 2026-07-24 17:17:21.261725784 +0000 UTC m=+3.988881976 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ccz6p" (UniqueName: "kubernetes.io/projected/179dc084-0504-4fcf-9608-07d598c9091c-kube-api-access-ccz6p") pod "network-check-target-k9hdw" (UID: "179dc084-0504-4fcf-9608-07d598c9091c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:17:20.362477 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:20.362443    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/602851eb-593f-4245-9365-97ef80e3eb51-original-pull-secret\") pod \"global-pull-secret-syncer-ts5kp\" (UID: \"602851eb-593f-4245-9365-97ef80e3eb51\") " pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:20.362635 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:20.362606    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:17:20.362713 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:20.362695    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/602851eb-593f-4245-9365-97ef80e3eb51-original-pull-secret podName:602851eb-593f-4245-9365-97ef80e3eb51 nodeName:}" failed. No retries permitted until 2026-07-24 17:17:21.362672447 +0000 UTC m=+4.089828651 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/602851eb-593f-4245-9365-97ef80e3eb51-original-pull-secret") pod "global-pull-secret-syncer-ts5kp" (UID: "602851eb-593f-4245-9365-97ef80e3eb51") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:17:20.694398 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:20.694349    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 17:12:19 +0000 UTC" deadline="2028-04-03 07:41:26.532941284 +0000 UTC"
Jul 24 17:17:20.694398 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:20.694383    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14846h24m5.838561886s"
Jul 24 17:17:20.819000 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:20.818964    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:20.819202 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:20.819106    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjjdg" podUID="b87310bf-1f99-43b7-9731-29966b07be99"
Jul 24 17:17:20.921740 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:20.921712    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44c623e2_f2fa_418d_986a_af9370891a35.slice/crio-8c7e083be779112344e0697d81a7197ec81d5b0fe17ec7ae1dc5ee72931ac60a WatchSource:0}: Error finding container 8c7e083be779112344e0697d81a7197ec81d5b0fe17ec7ae1dc5ee72931ac60a: Status 404 returned error can't find the container with id 8c7e083be779112344e0697d81a7197ec81d5b0fe17ec7ae1dc5ee72931ac60a
Jul 24 17:17:20.923100 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:20.923073    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11f4dbdb_7967_407a_8572_8ba5dd91e233.slice/crio-40b648a5f5f560ebee092d9838e8405267d03d9f8df52c0dfa5a6a3808e3f5f4 WatchSource:0}: Error finding container 40b648a5f5f560ebee092d9838e8405267d03d9f8df52c0dfa5a6a3808e3f5f4: Status 404 returned error can't find the container with id 40b648a5f5f560ebee092d9838e8405267d03d9f8df52c0dfa5a6a3808e3f5f4
Jul 24 17:17:20.924534 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:20.924379    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode729f7f5_5dac_4c69_a970_d6aaa51cf093.slice/crio-b50565d88b5c993b033086dab1aa4554bbfe8451359168645ea51cf73de4ad19 WatchSource:0}: Error finding container b50565d88b5c993b033086dab1aa4554bbfe8451359168645ea51cf73de4ad19: Status 404 returned error can't find the container with id b50565d88b5c993b033086dab1aa4554bbfe8451359168645ea51cf73de4ad19
Jul 24 17:17:20.927780 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:20.927759    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cf14aab_67c7_4f7b_8ad8_eb4c637fa08a.slice/crio-9c43d7ebeaf75b65a9142a371a3d802f390d205b565864f6caa358be422c5a1f WatchSource:0}: Error finding container 9c43d7ebeaf75b65a9142a371a3d802f390d205b565864f6caa358be422c5a1f: Status 404 returned error can't find the container with id 9c43d7ebeaf75b65a9142a371a3d802f390d205b565864f6caa358be422c5a1f
Jul 24 17:17:20.929014 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:20.928993    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb26dc7a6_6666_4814_b461_c9edad3dc705.slice/crio-75996e2b9f0bb6d8c17475dfb7fcd97399e45eaa76cd0bce7393bb0bf8be1646 WatchSource:0}: Error finding container 75996e2b9f0bb6d8c17475dfb7fcd97399e45eaa76cd0bce7393bb0bf8be1646: Status 404 returned error can't find the container with id 75996e2b9f0bb6d8c17475dfb7fcd97399e45eaa76cd0bce7393bb0bf8be1646
Jul 24 17:17:20.930944 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:20.930230    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15f25938_a5da_432f_91f3_b40dae0912d3.slice/crio-c428966097fc608477fae7c13242280ef8032461a1508d549d559b2877bf3ac1 WatchSource:0}: Error finding container c428966097fc608477fae7c13242280ef8032461a1508d549d559b2877bf3ac1: Status 404 returned error can't find the container with id c428966097fc608477fae7c13242280ef8032461a1508d549d559b2877bf3ac1
Jul 24 17:17:20.932448 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:20.931604    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab25ce71_a92b_4f56_b9df_2c2812e744ef.slice/crio-8b4d4008f95299552f2b401d47570de2c26da94a59883226c27753b2309424a6 WatchSource:0}: Error finding container 8b4d4008f95299552f2b401d47570de2c26da94a59883226c27753b2309424a6: Status 404 returned error can't find the container with id 8b4d4008f95299552f2b401d47570de2c26da94a59883226c27753b2309424a6
Jul 24 17:17:20.932448 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:20.932322    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41b7767d_e084_4b00_8168_2d5f2e59c70a.slice/crio-48aeb95af1eb8c9bd19b64326e14ca899694a55b139219d2eba448229f3d020e WatchSource:0}: Error finding container 48aeb95af1eb8c9bd19b64326e14ca899694a55b139219d2eba448229f3d020e: Status 404 returned error can't find the container with id 48aeb95af1eb8c9bd19b64326e14ca899694a55b139219d2eba448229f3d020e
Jul 24 17:17:20.934037 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:20.934017    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a0c97c1_3fee_45af_bc53_36991e313aa8.slice/crio-dd1b803e03337954f04b57f826f214d16b6af90840e1e39065225ffdda0fb2d2 WatchSource:0}: Error finding container dd1b803e03337954f04b57f826f214d16b6af90840e1e39065225ffdda0fb2d2: Status 404 returned error can't find the container with id dd1b803e03337954f04b57f826f214d16b6af90840e1e39065225ffdda0fb2d2
Jul 24 17:17:21.269729 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:21.269430    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs\") pod \"network-metrics-daemon-pjjdg\" (UID: \"b87310bf-1f99-43b7-9731-29966b07be99\") " pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:21.269729 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:21.269654    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccz6p\" (UniqueName: \"kubernetes.io/projected/179dc084-0504-4fcf-9608-07d598c9091c-kube-api-access-ccz6p\") pod \"network-check-target-k9hdw\" (UID: \"179dc084-0504-4fcf-9608-07d598c9091c\") " pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:17:21.269729 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:21.269594    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:17:21.270015 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:21.269760    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:17:21.270015 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:21.269770    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs podName:b87310bf-1f99-43b7-9731-29966b07be99 nodeName:}" failed. No retries permitted until 2026-07-24 17:17:23.269751159 +0000 UTC m=+5.996907364 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs") pod "network-metrics-daemon-pjjdg" (UID: "b87310bf-1f99-43b7-9731-29966b07be99") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:17:21.270015 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:21.269776    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:17:21.270015 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:21.269799    2562 projected.go:196] Error preparing data for projected volume kube-api-access-ccz6p for pod openshift-network-diagnostics/network-check-target-k9hdw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:17:21.270015 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:21.269843    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/179dc084-0504-4fcf-9608-07d598c9091c-kube-api-access-ccz6p podName:179dc084-0504-4fcf-9608-07d598c9091c nodeName:}" failed. No retries permitted until 2026-07-24 17:17:23.269831695 +0000 UTC m=+5.996987906 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ccz6p" (UniqueName: "kubernetes.io/projected/179dc084-0504-4fcf-9608-07d598c9091c-kube-api-access-ccz6p") pod "network-check-target-k9hdw" (UID: "179dc084-0504-4fcf-9608-07d598c9091c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:17:21.370689 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:21.370654    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/602851eb-593f-4245-9365-97ef80e3eb51-original-pull-secret\") pod \"global-pull-secret-syncer-ts5kp\" (UID: \"602851eb-593f-4245-9365-97ef80e3eb51\") " pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:21.370857 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:21.370796    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:17:21.370939 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:21.370865    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/602851eb-593f-4245-9365-97ef80e3eb51-original-pull-secret podName:602851eb-593f-4245-9365-97ef80e3eb51 nodeName:}" failed. No retries permitted until 2026-07-24 17:17:23.3708481 +0000 UTC m=+6.098004298 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/602851eb-593f-4245-9365-97ef80e3eb51-original-pull-secret") pod "global-pull-secret-syncer-ts5kp" (UID: "602851eb-593f-4245-9365-97ef80e3eb51") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:17:21.694878 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:21.694809    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 17:12:19 +0000 UTC" deadline="2028-02-01 21:22:02.084894483 +0000 UTC"
Jul 24 17:17:21.694878 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:21.694868    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13372h4m40.390030663s"
Jul 24 17:17:21.819831 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:21.819286    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:21.819831 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:21.819418    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ts5kp" podUID="602851eb-593f-4245-9365-97ef80e3eb51"
Jul 24 17:17:21.820306 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:21.820173    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:17:21.820306 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:21.820268    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9hdw" podUID="179dc084-0504-4fcf-9608-07d598c9091c"
Jul 24 17:17:21.838383 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:21.838277    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7" event={"ID":"41b7767d-e084-4b00-8168-2d5f2e59c70a","Type":"ContainerStarted","Data":"48aeb95af1eb8c9bd19b64326e14ca899694a55b139219d2eba448229f3d020e"}
Jul 24 17:17:21.848825 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:21.848789    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-h2fn4" event={"ID":"ab25ce71-a92b-4f56-b9df-2c2812e744ef","Type":"ContainerStarted","Data":"8b4d4008f95299552f2b401d47570de2c26da94a59883226c27753b2309424a6"}
Jul 24 17:17:21.851828 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:21.851793    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bbzvr" event={"ID":"b26dc7a6-6666-4814-b461-c9edad3dc705","Type":"ContainerStarted","Data":"75996e2b9f0bb6d8c17475dfb7fcd97399e45eaa76cd0bce7393bb0bf8be1646"}
Jul 24 17:17:21.857764 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:21.857724    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-45mn5" event={"ID":"15f25938-a5da-432f-91f3-b40dae0912d3","Type":"ContainerStarted","Data":"c428966097fc608477fae7c13242280ef8032461a1508d549d559b2877bf3ac1"}
Jul 24 17:17:21.863528 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:21.863488    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x" event={"ID":"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a","Type":"ContainerStarted","Data":"9c43d7ebeaf75b65a9142a371a3d802f390d205b565864f6caa358be422c5a1f"}
Jul 24 17:17:21.868256 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:21.868222    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b2t2r" event={"ID":"e729f7f5-5dac-4c69-a970-d6aaa51cf093","Type":"ContainerStarted","Data":"b50565d88b5c993b033086dab1aa4554bbfe8451359168645ea51cf73de4ad19"}
Jul 24 17:17:21.872207 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:21.872174    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-m77tg" event={"ID":"11f4dbdb-7967-407a-8572-8ba5dd91e233","Type":"ContainerStarted","Data":"40b648a5f5f560ebee092d9838e8405267d03d9f8df52c0dfa5a6a3808e3f5f4"}
Jul 24 17:17:21.888272 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:21.888242    2562 generic.go:356] "Generic (PLEG): container finished" podID="e423ebe6f419e89ee2be056e50f4506e" containerID="f365263d447885a55712862a9f9d3dd75b0681b5505607fd754ab6e854efe713" exitCode=0
Jul 24 17:17:21.888427 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:21.888348    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-159.ec2.internal" event={"ID":"e423ebe6f419e89ee2be056e50f4506e","Type":"ContainerDied","Data":"f365263d447885a55712862a9f9d3dd75b0681b5505607fd754ab6e854efe713"}
Jul 24 17:17:21.900878 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:21.900847    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-z2whv" event={"ID":"1a0c97c1-3fee-45af-bc53-36991e313aa8","Type":"ContainerStarted","Data":"dd1b803e03337954f04b57f826f214d16b6af90840e1e39065225ffdda0fb2d2"}
Jul 24 17:17:21.911976 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:21.911905    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tdnbn" event={"ID":"44c623e2-f2fa-418d-986a-af9370891a35","Type":"ContainerStarted","Data":"8c7e083be779112344e0697d81a7197ec81d5b0fe17ec7ae1dc5ee72931ac60a"}
Jul 24 17:17:21.930015 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:21.929985    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-159.ec2.internal" event={"ID":"d4dd242ec3a9586b4b7071e62b5c988d","Type":"ContainerStarted","Data":"48c20291b4ba2b4479713770dc18bc32eb672bd14be52f9a200b8620f08d1af8"}
Jul 24 17:17:21.950256 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:21.950157    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-159.ec2.internal" podStartSLOduration=2.950119416 podStartE2EDuration="2.950119416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:17:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:17:21.949361241 +0000 UTC m=+4.676517456" watchObservedRunningTime="2026-07-24 17:17:21.950119416 +0000 UTC m=+4.677275631"
Jul 24 17:17:22.819366 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:22.818807    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:22.819366 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:22.818967    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjjdg" podUID="b87310bf-1f99-43b7-9731-29966b07be99"
Jul 24 17:17:22.954444 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:22.954196    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-159.ec2.internal" event={"ID":"e423ebe6f419e89ee2be056e50f4506e","Type":"ContainerStarted","Data":"dc02f26cec0fe6cf3583804eebb7eebd1f619ac73cd95247ae8768ad17ed7c14"}
Jul 24 17:17:22.972320 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:22.971724    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-159.ec2.internal" podStartSLOduration=3.971705158 podStartE2EDuration="3.971705158s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:17:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:17:22.970960274 +0000 UTC m=+5.698116539" watchObservedRunningTime="2026-07-24 17:17:22.971705158 +0000 UTC m=+5.698861374"
Jul 24 17:17:23.287041 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:23.286954    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs\") pod \"network-metrics-daemon-pjjdg\" (UID: \"b87310bf-1f99-43b7-9731-29966b07be99\") " pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:23.287041 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:23.287010    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccz6p\" (UniqueName: \"kubernetes.io/projected/179dc084-0504-4fcf-9608-07d598c9091c-kube-api-access-ccz6p\") pod \"network-check-target-k9hdw\" (UID: \"179dc084-0504-4fcf-9608-07d598c9091c\") " pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:17:23.287290 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:23.287187    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:17:23.287290 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:23.287207    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:17:23.287290 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:23.287221    2562 projected.go:196] Error preparing data for projected volume kube-api-access-ccz6p for pod openshift-network-diagnostics/network-check-target-k9hdw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:17:23.287290 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:23.287281    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/179dc084-0504-4fcf-9608-07d598c9091c-kube-api-access-ccz6p podName:179dc084-0504-4fcf-9608-07d598c9091c nodeName:}" failed. No retries permitted until 2026-07-24 17:17:27.287262158 +0000 UTC m=+10.014418355 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ccz6p" (UniqueName: "kubernetes.io/projected/179dc084-0504-4fcf-9608-07d598c9091c-kube-api-access-ccz6p") pod "network-check-target-k9hdw" (UID: "179dc084-0504-4fcf-9608-07d598c9091c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:17:23.287684 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:23.287666    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:17:23.287749 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:23.287719    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs podName:b87310bf-1f99-43b7-9731-29966b07be99 nodeName:}" failed. No retries permitted until 2026-07-24 17:17:27.287704667 +0000 UTC m=+10.014860867 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs") pod "network-metrics-daemon-pjjdg" (UID: "b87310bf-1f99-43b7-9731-29966b07be99") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:17:23.387513 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:23.387475    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/602851eb-593f-4245-9365-97ef80e3eb51-original-pull-secret\") pod \"global-pull-secret-syncer-ts5kp\" (UID: \"602851eb-593f-4245-9365-97ef80e3eb51\") " pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:23.387700 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:23.387651    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:17:23.387764 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:23.387715    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/602851eb-593f-4245-9365-97ef80e3eb51-original-pull-secret podName:602851eb-593f-4245-9365-97ef80e3eb51 nodeName:}" failed. No retries permitted until 2026-07-24 17:17:27.387696505 +0000 UTC m=+10.114852701 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/602851eb-593f-4245-9365-97ef80e3eb51-original-pull-secret") pod "global-pull-secret-syncer-ts5kp" (UID: "602851eb-593f-4245-9365-97ef80e3eb51") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:17:23.820634 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:23.820603    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:23.821056 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:23.820738    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ts5kp" podUID="602851eb-593f-4245-9365-97ef80e3eb51"
Jul 24 17:17:23.821056 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:23.820897    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:17:23.821056 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:23.820999    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9hdw" podUID="179dc084-0504-4fcf-9608-07d598c9091c"
Jul 24 17:17:24.818338 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:24.818304    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:24.818504 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:24.818445    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjjdg" podUID="b87310bf-1f99-43b7-9731-29966b07be99"
Jul 24 17:17:25.818422 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:25.818247    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:17:25.818422 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:25.818367    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:25.818422 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:25.818384    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9hdw" podUID="179dc084-0504-4fcf-9608-07d598c9091c"
Jul 24 17:17:25.819003 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:25.818480    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ts5kp" podUID="602851eb-593f-4245-9365-97ef80e3eb51"
Jul 24 17:17:26.818152 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:26.818053    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:26.818320 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:26.818223    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjjdg" podUID="b87310bf-1f99-43b7-9731-29966b07be99"
Jul 24 17:17:27.319396 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:27.319355    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs\") pod \"network-metrics-daemon-pjjdg\" (UID: \"b87310bf-1f99-43b7-9731-29966b07be99\") " pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:27.319841 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:27.319411    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccz6p\" (UniqueName: \"kubernetes.io/projected/179dc084-0504-4fcf-9608-07d598c9091c-kube-api-access-ccz6p\") pod \"network-check-target-k9hdw\" (UID: \"179dc084-0504-4fcf-9608-07d598c9091c\") " pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:17:27.319841 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:27.319551    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:17:27.319841 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:27.319569    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:17:27.319841 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:27.319582    2562 projected.go:196] Error preparing data for projected volume kube-api-access-ccz6p for pod openshift-network-diagnostics/network-check-target-k9hdw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:17:27.319841 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:27.319641    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/179dc084-0504-4fcf-9608-07d598c9091c-kube-api-access-ccz6p podName:179dc084-0504-4fcf-9608-07d598c9091c nodeName:}" failed. No retries permitted until 2026-07-24 17:17:35.319623635 +0000 UTC m=+18.046779832 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ccz6p" (UniqueName: "kubernetes.io/projected/179dc084-0504-4fcf-9608-07d598c9091c-kube-api-access-ccz6p") pod "network-check-target-k9hdw" (UID: "179dc084-0504-4fcf-9608-07d598c9091c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:17:27.320120 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:27.320037    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:17:27.320120 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:27.320080    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs podName:b87310bf-1f99-43b7-9731-29966b07be99 nodeName:}" failed. No retries permitted until 2026-07-24 17:17:35.320065813 +0000 UTC m=+18.047222011 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs") pod "network-metrics-daemon-pjjdg" (UID: "b87310bf-1f99-43b7-9731-29966b07be99") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:17:27.420180 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:27.419978    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/602851eb-593f-4245-9365-97ef80e3eb51-original-pull-secret\") pod \"global-pull-secret-syncer-ts5kp\" (UID: \"602851eb-593f-4245-9365-97ef80e3eb51\") " pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:27.420180 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:27.420169    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:17:27.420411 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:27.420238    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/602851eb-593f-4245-9365-97ef80e3eb51-original-pull-secret podName:602851eb-593f-4245-9365-97ef80e3eb51 nodeName:}" failed. No retries permitted until 2026-07-24 17:17:35.420218354 +0000 UTC m=+18.147374548 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/602851eb-593f-4245-9365-97ef80e3eb51-original-pull-secret") pod "global-pull-secret-syncer-ts5kp" (UID: "602851eb-593f-4245-9365-97ef80e3eb51") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:17:27.820729 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:27.820267    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:27.820729 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:27.820386    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ts5kp" podUID="602851eb-593f-4245-9365-97ef80e3eb51"
Jul 24 17:17:27.820729 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:27.820543    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:17:27.820729 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:27.820650    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9hdw" podUID="179dc084-0504-4fcf-9608-07d598c9091c"
Jul 24 17:17:28.818778 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:28.818690    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:28.819190 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:28.818829    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjjdg" podUID="b87310bf-1f99-43b7-9731-29966b07be99"
Jul 24 17:17:29.822069 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:29.822036    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:17:29.822069 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:29.822052    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:29.822665 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:29.822191    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9hdw" podUID="179dc084-0504-4fcf-9608-07d598c9091c"
Jul 24 17:17:29.822665 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:29.822317    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ts5kp" podUID="602851eb-593f-4245-9365-97ef80e3eb51"
Jul 24 17:17:30.819034 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:30.819004    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:30.819198 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:30.819119    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjjdg" podUID="b87310bf-1f99-43b7-9731-29966b07be99"
Jul 24 17:17:31.818504 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:31.818385    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:17:31.818928 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:31.818519    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9hdw" podUID="179dc084-0504-4fcf-9608-07d598c9091c"
Jul 24 17:17:31.818928 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:31.818557    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:31.818928 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:31.818639    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ts5kp" podUID="602851eb-593f-4245-9365-97ef80e3eb51"
Jul 24 17:17:32.818445 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:32.818410    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:32.818637 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:32.818553    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjjdg" podUID="b87310bf-1f99-43b7-9731-29966b07be99"
Jul 24 17:17:33.818671 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:33.818632    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:33.819116 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:33.818783    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ts5kp" podUID="602851eb-593f-4245-9365-97ef80e3eb51"
Jul 24 17:17:33.819116 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:33.818830    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:17:33.819116 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:33.818950    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9hdw" podUID="179dc084-0504-4fcf-9608-07d598c9091c"
Jul 24 17:17:34.818898 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:34.818843    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:34.819354 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:34.818980    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjjdg" podUID="b87310bf-1f99-43b7-9731-29966b07be99"
Jul 24 17:17:35.382297 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:35.382211    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs\") pod \"network-metrics-daemon-pjjdg\" (UID: \"b87310bf-1f99-43b7-9731-29966b07be99\") " pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:35.382297 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:35.382261    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccz6p\" (UniqueName: \"kubernetes.io/projected/179dc084-0504-4fcf-9608-07d598c9091c-kube-api-access-ccz6p\") pod \"network-check-target-k9hdw\" (UID: \"179dc084-0504-4fcf-9608-07d598c9091c\") " pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:17:35.382481 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:35.382375    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:17:35.382481 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:35.382417    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:17:35.382481 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:35.382435    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:17:35.382481 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:35.382448    2562 projected.go:196] Error preparing data for projected volume kube-api-access-ccz6p for pod openshift-network-diagnostics/network-check-target-k9hdw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:17:35.382481 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:35.382452    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs podName:b87310bf-1f99-43b7-9731-29966b07be99 nodeName:}" failed. No retries permitted until 2026-07-24 17:17:51.382436627 +0000 UTC m=+34.109592823 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs") pod "network-metrics-daemon-pjjdg" (UID: "b87310bf-1f99-43b7-9731-29966b07be99") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:17:35.382696 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:35.382498    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/179dc084-0504-4fcf-9608-07d598c9091c-kube-api-access-ccz6p podName:179dc084-0504-4fcf-9608-07d598c9091c nodeName:}" failed. No retries permitted until 2026-07-24 17:17:51.382482895 +0000 UTC m=+34.109639089 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ccz6p" (UniqueName: "kubernetes.io/projected/179dc084-0504-4fcf-9608-07d598c9091c-kube-api-access-ccz6p") pod "network-check-target-k9hdw" (UID: "179dc084-0504-4fcf-9608-07d598c9091c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:17:35.483417 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:35.483378    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/602851eb-593f-4245-9365-97ef80e3eb51-original-pull-secret\") pod \"global-pull-secret-syncer-ts5kp\" (UID: \"602851eb-593f-4245-9365-97ef80e3eb51\") " pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:35.483603 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:35.483529    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:17:35.483603 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:35.483594    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/602851eb-593f-4245-9365-97ef80e3eb51-original-pull-secret podName:602851eb-593f-4245-9365-97ef80e3eb51 nodeName:}" failed. No retries permitted until 2026-07-24 17:17:51.483576571 +0000 UTC m=+34.210732783 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/602851eb-593f-4245-9365-97ef80e3eb51-original-pull-secret") pod "global-pull-secret-syncer-ts5kp" (UID: "602851eb-593f-4245-9365-97ef80e3eb51") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:17:35.818878 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:35.818787    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:35.819043 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:35.818787    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:17:35.819043 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:35.818936    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ts5kp" podUID="602851eb-593f-4245-9365-97ef80e3eb51"
Jul 24 17:17:35.819043 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:35.818969    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9hdw" podUID="179dc084-0504-4fcf-9608-07d598c9091c"
Jul 24 17:17:36.818673 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:36.818641    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:36.818832 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:36.818774    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjjdg" podUID="b87310bf-1f99-43b7-9731-29966b07be99"
Jul 24 17:17:37.819375 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:37.819328    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:37.819845 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:37.819434    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ts5kp" podUID="602851eb-593f-4245-9365-97ef80e3eb51"
Jul 24 17:17:37.819845 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:37.819520    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:17:37.819845 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:37.819630    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9hdw" podUID="179dc084-0504-4fcf-9608-07d598c9091c"
Jul 24 17:17:38.818516 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:38.818483    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:38.818702 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:38.818597    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjjdg" podUID="b87310bf-1f99-43b7-9731-29966b07be99"
Jul 24 17:17:39.818716 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:39.818316    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:17:39.819690 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:39.818382    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:39.819690 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:39.818817    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9hdw" podUID="179dc084-0504-4fcf-9608-07d598c9091c"
Jul 24 17:17:39.819690 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:39.818890    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ts5kp" podUID="602851eb-593f-4245-9365-97ef80e3eb51"
Jul 24 17:17:39.992375 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:39.992289    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x" event={"ID":"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a","Type":"ContainerStarted","Data":"e7dec8739b52ef859fc2680f0da24bd5a6baccab8aeea8bab85038366687e959"}
Jul 24 17:17:39.992576 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:39.992426    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x" event={"ID":"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a","Type":"ContainerStarted","Data":"9e64634eabf79996cf4325877e85dec8f8c6275f480e8eb8a6dca49e8a628d71"}
Jul 24 17:17:39.992576 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:39.992480    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x" event={"ID":"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a","Type":"ContainerStarted","Data":"6dfadc82d2785edb71180b56eb4bbbf042f06de8825c76ff096376ae67492cf3"}
Jul 24 17:17:39.992576 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:39.992513    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x" event={"ID":"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a","Type":"ContainerStarted","Data":"28c3a8e9fde8e2e2f154ce9e0fb730395c7c663739879054fdffb1524c6a584b"}
Jul 24 17:17:39.992576 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:39.992554    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x" event={"ID":"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a","Type":"ContainerStarted","Data":"c357affb9b709cba465679b04faffe2fabcd51357d259929a99146b2aa8f8760"}
Jul 24 17:17:39.992763 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:39.992597    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x" event={"ID":"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a","Type":"ContainerStarted","Data":"cf4f304a016acc6e00e88f481d254b1e1feb5e016c59a24d976240fc15a85c0d"}
Jul 24 17:17:39.993829 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:39.993807    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-m77tg" event={"ID":"11f4dbdb-7967-407a-8572-8ba5dd91e233","Type":"ContainerStarted","Data":"69dc2afcbbdc1ba2fa960ce9b8c27d94b57f4a95d777d126c6024fd7cfba98ba"}
Jul 24 17:17:39.995496 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:39.995460    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-z2whv" event={"ID":"1a0c97c1-3fee-45af-bc53-36991e313aa8","Type":"ContainerStarted","Data":"d43f097ebba4b14518916a558e5a7a5464962f823b0ec3086571ba88135f44d3"}
Jul 24 17:17:39.996903 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:39.996869    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tdnbn" event={"ID":"44c623e2-f2fa-418d-986a-af9370891a35","Type":"ContainerStarted","Data":"0de9502542d213759b6657c2258b8fb6d5cf40983479d74a866917be943d3ae7"}
Jul 24 17:17:39.998548 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:39.998524    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7" event={"ID":"41b7767d-e084-4b00-8168-2d5f2e59c70a","Type":"ContainerStarted","Data":"c791ffe918dec6e011f8b82d592b52acc3d241d4af0951fde7a9b1b0e7a026e2"}
Jul 24 17:17:39.999963 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:39.999940    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-h2fn4" event={"ID":"ab25ce71-a92b-4f56-b9df-2c2812e744ef","Type":"ContainerStarted","Data":"038fa5ae1603439d83e6fb92cb7702aa13935e28504853995554c637340f3bb7"}
Jul 24 17:17:40.001530 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:40.001499    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bbzvr" event={"ID":"b26dc7a6-6666-4814-b461-c9edad3dc705","Type":"ContainerStarted","Data":"ac8868635696486cb59cc58d86a3230caaebe38f12baf49d69b4d5a769fb49fc"}
Jul 24 17:17:40.003117 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:40.003092    2562 generic.go:356] "Generic (PLEG): container finished" podID="15f25938-a5da-432f-91f3-b40dae0912d3" containerID="fcbc7727e5ed4d68f66fa30133620fd199772589320cbc42c35e479066591dd6" exitCode=0
Jul 24 17:17:40.003228 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:40.003123    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-45mn5" event={"ID":"15f25938-a5da-432f-91f3-b40dae0912d3","Type":"ContainerDied","Data":"fcbc7727e5ed4d68f66fa30133620fd199772589320cbc42c35e479066591dd6"}
Jul 24 17:17:40.049293 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:40.049247    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-m77tg" podStartSLOduration=4.89940909 podStartE2EDuration="23.049231404s" totalImagesPullingTime="18.149822314s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:17:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:17:40.014909338 +0000 UTC m=+22.742065554" watchObservedRunningTime="2026-07-24 17:17:40.049231404 +0000 UTC m=+22.776387655"
Jul 24 17:17:40.076488 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:40.076436    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bbzvr" podStartSLOduration=3.9853898169999997 podStartE2EDuration="22.076419813s" totalImagesPullingTime="18.091029996s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:17:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:17:40.076165815 +0000 UTC m=+22.803322026" watchObservedRunningTime="2026-07-24 17:17:40.076419813 +0000 UTC m=+22.803576027"
Jul 24 17:17:40.092963 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:40.092915    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-z2whv" podStartSLOduration=7.926907084 podStartE2EDuration="22.092897145s" totalImagesPullingTime="14.165990061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:17:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:17:40.092498814 +0000 UTC m=+22.819655029" watchObservedRunningTime="2026-07-24 17:17:40.092897145 +0000 UTC m=+22.820053361"
Jul 24 17:17:40.145017 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:40.144969    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-h2fn4" podStartSLOduration=4.055980819 podStartE2EDuration="22.144954279s" totalImagesPullingTime="18.08897346s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:17:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:17:40.144267391 +0000 UTC m=+22.871423604" watchObservedRunningTime="2026-07-24 17:17:40.144954279 +0000 UTC m=+22.872110493"
Jul 24 17:17:40.145406 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:40.145381    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-tdnbn" podStartSLOduration=4.048210152 podStartE2EDuration="22.145375165s" totalImagesPullingTime="18.097165013s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:17:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:17:40.125726273 +0000 UTC m=+22.852882487" watchObservedRunningTime="2026-07-24 17:17:40.145375165 +0000 UTC m=+22.872531379"
Jul 24 17:17:40.522233 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:40.522201    2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 24 17:17:40.703301 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:40.703198    2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T17:17:40.522233836Z","UUID":"e6708412-70e6-4328-b033-1d1cb8e9c602","Handler":null,"Name":"","Endpoint":""}
Jul 24 17:17:40.704667 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:40.704649    2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 24 17:17:40.704762 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:40.704684    2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 24 17:17:40.818531 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:40.818487    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:40.818708 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:40.818612    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjjdg" podUID="b87310bf-1f99-43b7-9731-29966b07be99"
Jul 24 17:17:41.007655 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:41.007574    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7" event={"ID":"41b7767d-e084-4b00-8168-2d5f2e59c70a","Type":"ContainerStarted","Data":"d38b56138267e2bfbf5bb8336e386fd946a67e1b2703dce3fcbeda8e49db7c0b"}
Jul 24 17:17:41.008917 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:41.008887    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b2t2r" event={"ID":"e729f7f5-5dac-4c69-a970-d6aaa51cf093","Type":"ContainerStarted","Data":"743734f3f7dbe86cbae9869a49662d5104cc09a78e3e830d1efaa76bb325bd38"}
Jul 24 17:17:41.025351 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:41.025304    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-b2t2r" podStartSLOduration=4.929951518 podStartE2EDuration="23.025289814s" totalImagesPullingTime="18.095338296s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:17:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:17:41.024846264 +0000 UTC m=+23.752002478" watchObservedRunningTime="2026-07-24 17:17:41.025289814 +0000 UTC m=+23.752446028"
Jul 24 17:17:41.818648 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:41.818613    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:41.818828 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:41.818763    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ts5kp" podUID="602851eb-593f-4245-9365-97ef80e3eb51"
Jul 24 17:17:41.818828 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:41.818816    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:17:41.818967 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:41.818942    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9hdw" podUID="179dc084-0504-4fcf-9608-07d598c9091c"
Jul 24 17:17:42.818815 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:42.818612    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:42.819218 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:42.818857    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjjdg" podUID="b87310bf-1f99-43b7-9731-29966b07be99"
Jul 24 17:17:42.928158 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:42.928114    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-z2whv"
Jul 24 17:17:42.928770 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:42.928751    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-z2whv"
Jul 24 17:17:43.015315 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:43.015275    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7" event={"ID":"41b7767d-e084-4b00-8168-2d5f2e59c70a","Type":"ContainerStarted","Data":"3aae5ac40326996a1c5cf8ebdef0bc770c868a47b35dd6f966ef0a396f7f06e2"}
Jul 24 17:17:43.018997 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:43.018965    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x" event={"ID":"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a","Type":"ContainerStarted","Data":"2f2ebe594fa5579ccc929a07013607a3ba4d4a24ca546303522fe5df77f8d5a7"}
Jul 24 17:17:43.019117 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:43.019099    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-z2whv"
Jul 24 17:17:43.019670 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:43.019650    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-z2whv"
Jul 24 17:17:43.046395 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:43.046352    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mc5g7" podStartSLOduration=3.934963785 podStartE2EDuration="25.046338131s" totalImagesPullingTime="21.111374346s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:17:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:17:43.046065362 +0000 UTC m=+25.773221577" watchObservedRunningTime="2026-07-24 17:17:43.046338131 +0000 UTC m=+25.773494368"
Jul 24 17:17:43.818201 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:43.818168    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:17:43.818394 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:43.818306    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9hdw" podUID="179dc084-0504-4fcf-9608-07d598c9091c"
Jul 24 17:17:43.818394 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:43.818347    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:43.818506 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:43.818446    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ts5kp" podUID="602851eb-593f-4245-9365-97ef80e3eb51"
Jul 24 17:17:44.818440 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:44.818409    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:44.818912 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:44.818527    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjjdg" podUID="b87310bf-1f99-43b7-9731-29966b07be99"
Jul 24 17:17:45.818546 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:45.818508    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:17:45.819127 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:45.818673    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9hdw" podUID="179dc084-0504-4fcf-9608-07d598c9091c"
Jul 24 17:17:45.819127 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:45.818744    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:45.819127 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:45.818884    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ts5kp" podUID="602851eb-593f-4245-9365-97ef80e3eb51"
Jul 24 17:17:46.028799 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:46.028512    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-45mn5" event={"ID":"15f25938-a5da-432f-91f3-b40dae0912d3","Type":"ContainerStarted","Data":"b99eaedfd78c846c35637bfe153874905ee7c3fa9ba9d651b97bfc71b0e210d1"}
Jul 24 17:17:46.032281 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:46.032246    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x" event={"ID":"2cf14aab-67c7-4f7b-8ad8-eb4c637fa08a","Type":"ContainerStarted","Data":"fe754484acbeb669ff64827ba3cec7b9748098433e02a7355295617ab8583a90"}
Jul 24 17:17:46.032444 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:46.032427    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:46.032514 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:46.032501    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:46.047214 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:46.047184    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:46.082900 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:46.082848    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x" podStartSLOduration=10.922120756 podStartE2EDuration="29.082833445s" totalImagesPullingTime="18.160712689s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:17:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:17:46.081797772 +0000 UTC m=+28.808953987" watchObservedRunningTime="2026-07-24 17:17:46.082833445 +0000 UTC m=+28.809989672"
Jul 24 17:17:46.818440 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:46.818345    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:46.818596 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:46.818488    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjjdg" podUID="b87310bf-1f99-43b7-9731-29966b07be99"
Jul 24 17:17:47.036516 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:47.036473    2562 generic.go:356] "Generic (PLEG): container finished" podID="15f25938-a5da-432f-91f3-b40dae0912d3" containerID="b99eaedfd78c846c35637bfe153874905ee7c3fa9ba9d651b97bfc71b0e210d1" exitCode=0
Jul 24 17:17:47.036714 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:47.036564    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-45mn5" event={"ID":"15f25938-a5da-432f-91f3-b40dae0912d3","Type":"ContainerDied","Data":"b99eaedfd78c846c35637bfe153874905ee7c3fa9ba9d651b97bfc71b0e210d1"}
Jul 24 17:17:47.037086 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:47.037051    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:47.051954 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:47.051924    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:17:47.821289 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:47.821263    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:47.821687 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:47.821357    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ts5kp" podUID="602851eb-593f-4245-9365-97ef80e3eb51"
Jul 24 17:17:47.821687 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:47.821382    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:17:47.821687 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:47.821427    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9hdw" podUID="179dc084-0504-4fcf-9608-07d598c9091c"
Jul 24 17:17:48.121400 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:48.121366    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pjjdg"]
Jul 24 17:17:48.121518 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:48.121484    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:48.121607 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:48.121586    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjjdg" podUID="b87310bf-1f99-43b7-9731-29966b07be99"
Jul 24 17:17:48.124227 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:48.124134    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k9hdw"]
Jul 24 17:17:48.124290 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:48.124235    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:17:48.124368 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:48.124337    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9hdw" podUID="179dc084-0504-4fcf-9608-07d598c9091c"
Jul 24 17:17:48.136511 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:48.136488    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ts5kp"]
Jul 24 17:17:48.136604 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:48.136552    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:48.136645 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:48.136614    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ts5kp" podUID="602851eb-593f-4245-9365-97ef80e3eb51"
Jul 24 17:17:49.042752 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:49.042669    2562 generic.go:356] "Generic (PLEG): container finished" podID="15f25938-a5da-432f-91f3-b40dae0912d3" containerID="981571bce6f96c7873fb261916b3ec9ffb556149a5ba54432bee9aa7b97948e7" exitCode=0
Jul 24 17:17:49.042752 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:49.042706    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-45mn5" event={"ID":"15f25938-a5da-432f-91f3-b40dae0912d3","Type":"ContainerDied","Data":"981571bce6f96c7873fb261916b3ec9ffb556149a5ba54432bee9aa7b97948e7"}
Jul 24 17:17:49.819227 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:49.819062    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:17:49.819314 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:49.819064    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:49.819356 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:49.819317    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9hdw" podUID="179dc084-0504-4fcf-9608-07d598c9091c"
Jul 24 17:17:49.819391 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:49.819112    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:49.819474 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:49.819377    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjjdg" podUID="b87310bf-1f99-43b7-9731-29966b07be99"
Jul 24 17:17:49.819516 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:49.819470    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ts5kp" podUID="602851eb-593f-4245-9365-97ef80e3eb51"
Jul 24 17:17:50.047196 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:50.047100    2562 generic.go:356] "Generic (PLEG): container finished" podID="15f25938-a5da-432f-91f3-b40dae0912d3" containerID="ad4b710f2c75d1da8a38bcb87a496a2afc22bae6db37dc8af1fbdfd0499817a4" exitCode=0
Jul 24 17:17:50.047196 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:50.047174    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-45mn5" event={"ID":"15f25938-a5da-432f-91f3-b40dae0912d3","Type":"ContainerDied","Data":"ad4b710f2c75d1da8a38bcb87a496a2afc22bae6db37dc8af1fbdfd0499817a4"}
Jul 24 17:17:51.399777 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.399742    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs\") pod \"network-metrics-daemon-pjjdg\" (UID: \"b87310bf-1f99-43b7-9731-29966b07be99\") " pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:51.400254 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.399784    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccz6p\" (UniqueName: \"kubernetes.io/projected/179dc084-0504-4fcf-9608-07d598c9091c-kube-api-access-ccz6p\") pod \"network-check-target-k9hdw\" (UID: \"179dc084-0504-4fcf-9608-07d598c9091c\") " pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:17:51.400254 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:51.399926    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:17:51.400254 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:51.400005    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs podName:b87310bf-1f99-43b7-9731-29966b07be99 nodeName:}" failed. No retries permitted until 2026-07-24 17:18:23.399985506 +0000 UTC m=+66.127141721 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs") pod "network-metrics-daemon-pjjdg" (UID: "b87310bf-1f99-43b7-9731-29966b07be99") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:17:51.400254 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:51.400091    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:17:51.400254 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:51.400111    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:17:51.400254 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:51.400126    2562 projected.go:196] Error preparing data for projected volume kube-api-access-ccz6p for pod openshift-network-diagnostics/network-check-target-k9hdw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:17:51.400254 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:51.400195    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/179dc084-0504-4fcf-9608-07d598c9091c-kube-api-access-ccz6p podName:179dc084-0504-4fcf-9608-07d598c9091c nodeName:}" failed. No retries permitted until 2026-07-24 17:18:23.400180318 +0000 UTC m=+66.127336510 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-ccz6p" (UniqueName: "kubernetes.io/projected/179dc084-0504-4fcf-9608-07d598c9091c-kube-api-access-ccz6p") pod "network-check-target-k9hdw" (UID: "179dc084-0504-4fcf-9608-07d598c9091c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:17:51.500864 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.500830    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/602851eb-593f-4245-9365-97ef80e3eb51-original-pull-secret\") pod \"global-pull-secret-syncer-ts5kp\" (UID: \"602851eb-593f-4245-9365-97ef80e3eb51\") " pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:51.501102 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:51.500984    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:17:51.501102 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:51.501067    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/602851eb-593f-4245-9365-97ef80e3eb51-original-pull-secret podName:602851eb-593f-4245-9365-97ef80e3eb51 nodeName:}" failed. No retries permitted until 2026-07-24 17:18:23.501044291 +0000 UTC m=+66.228200486 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/602851eb-593f-4245-9365-97ef80e3eb51-original-pull-secret") pod "global-pull-secret-syncer-ts5kp" (UID: "602851eb-593f-4245-9365-97ef80e3eb51") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:17:51.616253 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.616230    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-159.ec2.internal" event="NodeReady"
Jul 24 17:17:51.616390 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.616342    2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 24 17:17:51.654846 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.654766    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-75594549dc-q5krg"]
Jul 24 17:17:51.656933 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.656897    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:17:51.659427 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.659403    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 24 17:17:51.660541 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.660519    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-p7fvc\""
Jul 24 17:17:51.660641 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.660621    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 24 17:17:51.660703 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.660638    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 24 17:17:51.667455 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.667432    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 24 17:17:51.667730 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.667708    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-7c7jv"]
Jul 24 17:17:51.669515 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.669496    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-4cmjc"]
Jul 24 17:17:51.669651 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.669635    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-7c7jv"
Jul 24 17:17:51.671601 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.671584    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4cmjc"
Jul 24 17:17:51.671840 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.671818    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-qsk2d\""
Jul 24 17:17:51.672088 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.672071    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 17:17:51.672436 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.672419    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 17:17:51.673873 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.673854    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 24 17:17:51.674460 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.674443    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 24 17:17:51.674624 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.674607    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-tkvbh\""
Jul 24 17:17:51.682853 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.682832    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75594549dc-q5krg"]
Jul 24 17:17:51.683833 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.683749    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gwrrp"]
Jul 24 17:17:51.686684 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.686667    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bf5b8"]
Jul 24 17:17:51.686937 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.686765    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gwrrp"
Jul 24 17:17:51.689587 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.689572    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-4cmjc"]
Jul 24 17:17:51.689718 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.689691    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bf5b8"
Jul 24 17:17:51.689797 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.689779    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 24 17:17:51.692899 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.692879    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-7c7jv"]
Jul 24 17:17:51.693961 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.693942    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 24 17:17:51.694602 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.694584    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 24 17:17:51.694708 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.694671    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tlf5h\""
Jul 24 17:17:51.694796 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.694779    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 24 17:17:51.694930 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.694903    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 24 17:17:51.695045 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.695033    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-r2b86\""
Jul 24 17:17:51.697831 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.697811    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gwrrp"]
Jul 24 17:17:51.707249 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.707230    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bf5b8"]
Jul 24 17:17:51.802852 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.802811    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-bound-sa-token\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:17:51.803030 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.802871    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4438603e-bddf-4e8f-979b-9f7a58c7e698-tmp-dir\") pod \"dns-default-bf5b8\" (UID: \"4438603e-bddf-4e8f-979b-9f7a58c7e698\") " pod="openshift-dns/dns-default-bf5b8"
Jul 24 17:17:51.803030 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.802910    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/615228ee-d6a8-4566-b5e7-82d5a27bf812-image-registry-private-configuration\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:17:51.803030 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.802931    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:17:51.803030 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.802956    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/615228ee-d6a8-4566-b5e7-82d5a27bf812-trusted-ca\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:17:51.803030 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.803009    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/615228ee-d6a8-4566-b5e7-82d5a27bf812-installation-pull-secrets\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:17:51.803294 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.803072    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/63522b79-132b-45c7-92c5-3bfbfa4c7b3a-cert\") pod \"ingress-canary-gwrrp\" (UID: \"63522b79-132b-45c7-92c5-3bfbfa4c7b3a\") " pod="openshift-ingress-canary/ingress-canary-gwrrp"
Jul 24 17:17:51.803294 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.803094    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/615228ee-d6a8-4566-b5e7-82d5a27bf812-ca-trust-extracted\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:17:51.803294 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.803122    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbtgd\" (UniqueName: \"kubernetes.io/projected/a14d12ef-f2fc-45d2-9cd5-5e21b1b1f670-kube-api-access-gbtgd\") pod \"network-check-source-7d88cb55d8-7c7jv\" (UID: \"a14d12ef-f2fc-45d2-9cd5-5e21b1b1f670\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-7c7jv"
Jul 24 17:17:51.803294 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.803163    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4cmjc\" (UID: \"f3d1e52c-502d-496f-bafd-2fcee624d002\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4cmjc"
Jul 24 17:17:51.803294 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.803193    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtbn7\" (UniqueName: \"kubernetes.io/projected/63522b79-132b-45c7-92c5-3bfbfa4c7b3a-kube-api-access-vtbn7\") pod \"ingress-canary-gwrrp\" (UID: \"63522b79-132b-45c7-92c5-3bfbfa4c7b3a\") " pod="openshift-ingress-canary/ingress-canary-gwrrp"
Jul 24 17:17:51.803294 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.803230    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-certificates\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:17:51.803294 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.803270    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb6s8\" (UniqueName: \"kubernetes.io/projected/4438603e-bddf-4e8f-979b-9f7a58c7e698-kube-api-access-zb6s8\") pod \"dns-default-bf5b8\" (UID: \"4438603e-bddf-4e8f-979b-9f7a58c7e698\") " pod="openshift-dns/dns-default-bf5b8"
Jul 24 17:17:51.803598 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.803334    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4438603e-bddf-4e8f-979b-9f7a58c7e698-config-volume\") pod \"dns-default-bf5b8\" (UID: \"4438603e-bddf-4e8f-979b-9f7a58c7e698\") " pod="openshift-dns/dns-default-bf5b8"
Jul 24 17:17:51.803598 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.803369    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6nk4\" (UniqueName: \"kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-kube-api-access-f6nk4\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:17:51.803598 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.803410    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls\") pod \"dns-default-bf5b8\" (UID: \"4438603e-bddf-4e8f-979b-9f7a58c7e698\") " pod="openshift-dns/dns-default-bf5b8"
Jul 24 17:17:51.803598 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.803437    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f3d1e52c-502d-496f-bafd-2fcee624d002-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-4cmjc\" (UID: \"f3d1e52c-502d-496f-bafd-2fcee624d002\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4cmjc"
Jul 24 17:17:51.822922 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.822894    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:17:51.823079 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.823000    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:17:51.823079 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.823034    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:17:51.827012 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.826749    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 17:17:51.827012 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.826832    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ntfrz\""
Jul 24 17:17:51.827012 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.826884    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 17:17:51.827012 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.826926    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-t6c84\""
Jul 24 17:17:51.831174 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.831075    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-z2whv_1a0c97c1-3fee-45af-bc53-36991e313aa8/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:51.904399 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.904371    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/63522b79-132b-45c7-92c5-3bfbfa4c7b3a-cert\") pod \"ingress-canary-gwrrp\" (UID: \"63522b79-132b-45c7-92c5-3bfbfa4c7b3a\") " pod="openshift-ingress-canary/ingress-canary-gwrrp"
Jul 24 17:17:51.904399 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.904403    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/615228ee-d6a8-4566-b5e7-82d5a27bf812-ca-trust-extracted\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:17:51.904619 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.904421    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbtgd\" (UniqueName: \"kubernetes.io/projected/a14d12ef-f2fc-45d2-9cd5-5e21b1b1f670-kube-api-access-gbtgd\") pod \"network-check-source-7d88cb55d8-7c7jv\" (UID: \"a14d12ef-f2fc-45d2-9cd5-5e21b1b1f670\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-7c7jv"
Jul 24 17:17:51.904619 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.904445    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4cmjc\" (UID: \"f3d1e52c-502d-496f-bafd-2fcee624d002\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4cmjc"
Jul 24 17:17:51.904619 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.904464    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtbn7\" (UniqueName: \"kubernetes.io/projected/63522b79-132b-45c7-92c5-3bfbfa4c7b3a-kube-api-access-vtbn7\") pod \"ingress-canary-gwrrp\" (UID: \"63522b79-132b-45c7-92c5-3bfbfa4c7b3a\") " pod="openshift-ingress-canary/ingress-canary-gwrrp"
Jul 24 17:17:51.904619 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.904480    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-certificates\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:17:51.904619 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.904496    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb6s8\" (UniqueName: \"kubernetes.io/projected/4438603e-bddf-4e8f-979b-9f7a58c7e698-kube-api-access-zb6s8\") pod \"dns-default-bf5b8\" (UID: \"4438603e-bddf-4e8f-979b-9f7a58c7e698\") " pod="openshift-dns/dns-default-bf5b8"
Jul 24 17:17:51.904619 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.904554    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4438603e-bddf-4e8f-979b-9f7a58c7e698-config-volume\") pod \"dns-default-bf5b8\" (UID: \"4438603e-bddf-4e8f-979b-9f7a58c7e698\") " pod="openshift-dns/dns-default-bf5b8"
Jul 24 17:17:51.904619 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:51.904578    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 17:17:51.904619 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.904608    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6nk4\" (UniqueName: \"kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-kube-api-access-f6nk4\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:17:51.905118 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:51.904649    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert podName:f3d1e52c-502d-496f-bafd-2fcee624d002 nodeName:}" failed. No retries permitted until 2026-07-24 17:17:52.40462753 +0000 UTC m=+35.131783739 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4cmjc" (UID: "f3d1e52c-502d-496f-bafd-2fcee624d002") : secret "networking-console-plugin-cert" not found
Jul 24 17:17:51.905118 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.904701    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls\") pod \"dns-default-bf5b8\" (UID: \"4438603e-bddf-4e8f-979b-9f7a58c7e698\") " pod="openshift-dns/dns-default-bf5b8"
Jul 24 17:17:51.905118 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.904733    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f3d1e52c-502d-496f-bafd-2fcee624d002-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-4cmjc\" (UID: \"f3d1e52c-502d-496f-bafd-2fcee624d002\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4cmjc"
Jul 24 17:17:51.905118 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:51.904825    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:17:51.905118 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.904851    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-bound-sa-token\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:17:51.905118 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:51.904875    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls podName:4438603e-bddf-4e8f-979b-9f7a58c7e698 nodeName:}" failed. No retries permitted until 2026-07-24 17:17:52.40485734 +0000 UTC m=+35.132013540 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls") pod "dns-default-bf5b8" (UID: "4438603e-bddf-4e8f-979b-9f7a58c7e698") : secret "dns-default-metrics-tls" not found
Jul 24 17:17:51.905118 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.904904    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4438603e-bddf-4e8f-979b-9f7a58c7e698-tmp-dir\") pod \"dns-default-bf5b8\" (UID: \"4438603e-bddf-4e8f-979b-9f7a58c7e698\") " pod="openshift-dns/dns-default-bf5b8"
Jul 24 17:17:51.905118 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.904943    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/615228ee-d6a8-4566-b5e7-82d5a27bf812-image-registry-private-configuration\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:17:51.905118 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.904825    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/615228ee-d6a8-4566-b5e7-82d5a27bf812-ca-trust-extracted\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:17:51.905118 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.904970    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:17:51.905118 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:51.905028    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 17:17:51.905118 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.905031    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/615228ee-d6a8-4566-b5e7-82d5a27bf812-trusted-ca\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:17:51.905118 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.905067    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/615228ee-d6a8-4566-b5e7-82d5a27bf812-installation-pull-secrets\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:17:51.905663 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.905135    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-certificates\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:17:51.905663 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:51.905041    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75594549dc-q5krg: secret "image-registry-tls" not found
Jul 24 17:17:51.905663 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:51.905606    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls podName:615228ee-d6a8-4566-b5e7-82d5a27bf812 nodeName:}" failed. No retries permitted until 2026-07-24 17:17:52.405586871 +0000 UTC m=+35.132743083 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls") pod "image-registry-75594549dc-q5krg" (UID: "615228ee-d6a8-4566-b5e7-82d5a27bf812") : secret "image-registry-tls" not found
Jul 24 17:17:51.905663 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.905358    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4438603e-bddf-4e8f-979b-9f7a58c7e698-config-volume\") pod \"dns-default-bf5b8\" (UID: \"4438603e-bddf-4e8f-979b-9f7a58c7e698\") " pod="openshift-dns/dns-default-bf5b8"
Jul 24 17:17:51.905663 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.905502    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f3d1e52c-502d-496f-bafd-2fcee624d002-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-4cmjc\" (UID: \"f3d1e52c-502d-496f-bafd-2fcee624d002\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4cmjc"
Jul 24 17:17:51.905663 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.905339    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4438603e-bddf-4e8f-979b-9f7a58c7e698-tmp-dir\") pod \"dns-default-bf5b8\" (UID: \"4438603e-bddf-4e8f-979b-9f7a58c7e698\") " pod="openshift-dns/dns-default-bf5b8"
Jul 24 17:17:51.905987 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.905971    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/615228ee-d6a8-4566-b5e7-82d5a27bf812-trusted-ca\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:17:51.909075 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.909051    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/615228ee-d6a8-4566-b5e7-82d5a27bf812-image-registry-private-configuration\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:17:51.909186 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.909112    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/615228ee-d6a8-4566-b5e7-82d5a27bf812-installation-pull-secrets\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:17:51.909186 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.909122    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/63522b79-132b-45c7-92c5-3bfbfa4c7b3a-cert\") pod \"ingress-canary-gwrrp\" (UID: \"63522b79-132b-45c7-92c5-3bfbfa4c7b3a\") " pod="openshift-ingress-canary/ingress-canary-gwrrp"
Jul 24 17:17:51.913580 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.913558    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-bound-sa-token\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:17:51.913801 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.913779    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtbn7\" (UniqueName: \"kubernetes.io/projected/63522b79-132b-45c7-92c5-3bfbfa4c7b3a-kube-api-access-vtbn7\") pod \"ingress-canary-gwrrp\" (UID: \"63522b79-132b-45c7-92c5-3bfbfa4c7b3a\") " pod="openshift-ingress-canary/ingress-canary-gwrrp"
Jul 24 17:17:51.913860 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.913814    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb6s8\" (UniqueName: \"kubernetes.io/projected/4438603e-bddf-4e8f-979b-9f7a58c7e698-kube-api-access-zb6s8\") pod \"dns-default-bf5b8\" (UID: \"4438603e-bddf-4e8f-979b-9f7a58c7e698\") " pod="openshift-dns/dns-default-bf5b8"
Jul 24 17:17:51.914322 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.914297    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbtgd\" (UniqueName: \"kubernetes.io/projected/a14d12ef-f2fc-45d2-9cd5-5e21b1b1f670-kube-api-access-gbtgd\") pod \"network-check-source-7d88cb55d8-7c7jv\" (UID: \"a14d12ef-f2fc-45d2-9cd5-5e21b1b1f670\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-7c7jv"
Jul 24 17:17:51.915439 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.915421    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6nk4\" (UniqueName: \"kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-kube-api-access-f6nk4\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:17:51.917567 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.917542    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-z2whv_1a0c97c1-3fee-45af-bc53-36991e313aa8/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:51.980578 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.980555    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-7c7jv"
Jul 24 17:17:51.999695 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:51.999669    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gwrrp"
Jul 24 17:17:52.143394 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:52.143358    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-7c7jv"]
Jul 24 17:17:52.146323 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:52.146297    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gwrrp"]
Jul 24 17:17:52.148435 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:52.148396    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda14d12ef_f2fc_45d2_9cd5_5e21b1b1f670.slice/crio-6feb56d0159ca0104a43450f29058b2e77f59de1f20aa19eb7a93919cc499edb WatchSource:0}: Error finding container 6feb56d0159ca0104a43450f29058b2e77f59de1f20aa19eb7a93919cc499edb: Status 404 returned error can't find the container with id 6feb56d0159ca0104a43450f29058b2e77f59de1f20aa19eb7a93919cc499edb
Jul 24 17:17:52.150444 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:17:52.150363    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63522b79_132b_45c7_92c5_3bfbfa4c7b3a.slice/crio-9ccb399d7c6aaa97261fe2ac3c98bbabfbc8aac24ef3e742a12a83f02dedb8c0 WatchSource:0}: Error finding container 9ccb399d7c6aaa97261fe2ac3c98bbabfbc8aac24ef3e742a12a83f02dedb8c0: Status 404 returned error can't find the container with id 9ccb399d7c6aaa97261fe2ac3c98bbabfbc8aac24ef3e742a12a83f02dedb8c0
Jul 24 17:17:52.161016 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:52.160961    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-z2whv_1a0c97c1-3fee-45af-bc53-36991e313aa8/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:52.409516 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:52.409480    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls\") pod \"dns-default-bf5b8\" (UID: \"4438603e-bddf-4e8f-979b-9f7a58c7e698\") " pod="openshift-dns/dns-default-bf5b8"
Jul 24 17:17:52.410195 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:52.409535    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:17:52.410195 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:52.409565    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4cmjc\" (UID: \"f3d1e52c-502d-496f-bafd-2fcee624d002\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4cmjc"
Jul 24 17:17:52.410195 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:52.409630    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:17:52.410195 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:52.409659    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 17:17:52.410195 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:52.409686    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 17:17:52.410195 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:52.409702    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75594549dc-q5krg: secret "image-registry-tls" not found
Jul 24 17:17:52.410195 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:52.409694    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls podName:4438603e-bddf-4e8f-979b-9f7a58c7e698 nodeName:}" failed. No retries permitted until 2026-07-24 17:17:53.409678104 +0000 UTC m=+36.136834300 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls") pod "dns-default-bf5b8" (UID: "4438603e-bddf-4e8f-979b-9f7a58c7e698") : secret "dns-default-metrics-tls" not found
Jul 24 17:17:52.410195 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:52.409747    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert podName:f3d1e52c-502d-496f-bafd-2fcee624d002 nodeName:}" failed. No retries permitted until 2026-07-24 17:17:53.409734619 +0000 UTC m=+36.136890811 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4cmjc" (UID: "f3d1e52c-502d-496f-bafd-2fcee624d002") : secret "networking-console-plugin-cert" not found
Jul 24 17:17:52.410195 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:52.409758    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls podName:615228ee-d6a8-4566-b5e7-82d5a27bf812 nodeName:}" failed. No retries permitted until 2026-07-24 17:17:53.409751802 +0000 UTC m=+36.136907994 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls") pod "image-registry-75594549dc-q5krg" (UID: "615228ee-d6a8-4566-b5e7-82d5a27bf812") : secret "image-registry-tls" not found
Jul 24 17:17:53.058732 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:53.058688    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gwrrp" event={"ID":"63522b79-132b-45c7-92c5-3bfbfa4c7b3a","Type":"ContainerStarted","Data":"9ccb399d7c6aaa97261fe2ac3c98bbabfbc8aac24ef3e742a12a83f02dedb8c0"}
Jul 24 17:17:53.061471 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:53.061435    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-7c7jv" event={"ID":"a14d12ef-f2fc-45d2-9cd5-5e21b1b1f670","Type":"ContainerStarted","Data":"6feb56d0159ca0104a43450f29058b2e77f59de1f20aa19eb7a93919cc499edb"}
Jul 24 17:17:53.418250 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:53.418214    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls\") pod \"dns-default-bf5b8\" (UID: \"4438603e-bddf-4e8f-979b-9f7a58c7e698\") " pod="openshift-dns/dns-default-bf5b8"
Jul 24 17:17:53.418719 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:53.418293    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:17:53.418719 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:53.418345    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4cmjc\" (UID: \"f3d1e52c-502d-496f-bafd-2fcee624d002\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4cmjc"
Jul 24 17:17:53.418719 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:53.418404    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:17:53.418719 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:53.418453    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 17:17:53.418719 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:53.418466    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 17:17:53.418719 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:53.418485    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75594549dc-q5krg: secret "image-registry-tls" not found
Jul 24 17:17:53.418719 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:53.418485    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls podName:4438603e-bddf-4e8f-979b-9f7a58c7e698 nodeName:}" failed. No retries permitted until 2026-07-24 17:17:55.418463998 +0000 UTC m=+38.145620198 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls") pod "dns-default-bf5b8" (UID: "4438603e-bddf-4e8f-979b-9f7a58c7e698") : secret "dns-default-metrics-tls" not found
Jul 24 17:17:53.418719 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:53.418551    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert podName:f3d1e52c-502d-496f-bafd-2fcee624d002 nodeName:}" failed. No retries permitted until 2026-07-24 17:17:55.418531831 +0000 UTC m=+38.145688031 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4cmjc" (UID: "f3d1e52c-502d-496f-bafd-2fcee624d002") : secret "networking-console-plugin-cert" not found
Jul 24 17:17:53.418719 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:53.418568    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls podName:615228ee-d6a8-4566-b5e7-82d5a27bf812 nodeName:}" failed. No retries permitted until 2026-07-24 17:17:55.418558218 +0000 UTC m=+38.145714411 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls") pod "image-registry-75594549dc-q5krg" (UID: "615228ee-d6a8-4566-b5e7-82d5a27bf812") : secret "image-registry-tls" not found
Jul 24 17:17:55.436183 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:55.436120    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls\") pod \"dns-default-bf5b8\" (UID: \"4438603e-bddf-4e8f-979b-9f7a58c7e698\") " pod="openshift-dns/dns-default-bf5b8"
Jul 24 17:17:55.436622 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:55.436222    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:17:55.436622 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:55.436277    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4cmjc\" (UID: \"f3d1e52c-502d-496f-bafd-2fcee624d002\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4cmjc"
Jul 24 17:17:55.436790 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:55.436760    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:17:55.436917 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:55.436813    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 17:17:55.436917 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:55.436834    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75594549dc-q5krg: secret "image-registry-tls" not found
Jul 24 17:17:55.436917 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:55.436838    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls podName:4438603e-bddf-4e8f-979b-9f7a58c7e698 nodeName:}" failed. No retries permitted until 2026-07-24 17:17:59.436818727 +0000 UTC m=+42.163974944 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls") pod "dns-default-bf5b8" (UID: "4438603e-bddf-4e8f-979b-9f7a58c7e698") : secret "dns-default-metrics-tls" not found
Jul 24 17:17:55.436917 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:55.436772    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 17:17:55.436917 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:55.436904    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls podName:615228ee-d6a8-4566-b5e7-82d5a27bf812 nodeName:}" failed. No retries permitted until 2026-07-24 17:17:59.436875677 +0000 UTC m=+42.164031875 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls") pod "image-registry-75594549dc-q5krg" (UID: "615228ee-d6a8-4566-b5e7-82d5a27bf812") : secret "image-registry-tls" not found
Jul 24 17:17:55.437187 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:55.436922    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert podName:f3d1e52c-502d-496f-bafd-2fcee624d002 nodeName:}" failed. No retries permitted until 2026-07-24 17:17:59.436912256 +0000 UTC m=+42.164068449 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4cmjc" (UID: "f3d1e52c-502d-496f-bafd-2fcee624d002") : secret "networking-console-plugin-cert" not found
Jul 24 17:17:56.452126 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:56.452045    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-z2whv_1a0c97c1-3fee-45af-bc53-36991e313aa8/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:56.509184 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:56.509083    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-z2whv_1a0c97c1-3fee-45af-bc53-36991e313aa8/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:58.075707 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:58.075672    2562 generic.go:356] "Generic (PLEG): container finished" podID="15f25938-a5da-432f-91f3-b40dae0912d3" containerID="e3e50c3af46f106be8191d0665ee5c8120a776004fcd7f64eaa266f5fdb11f14" exitCode=0
Jul 24 17:17:58.076239 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:58.075762    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-45mn5" event={"ID":"15f25938-a5da-432f-91f3-b40dae0912d3","Type":"ContainerDied","Data":"e3e50c3af46f106be8191d0665ee5c8120a776004fcd7f64eaa266f5fdb11f14"}
Jul 24 17:17:58.077412 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:58.077384    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gwrrp" event={"ID":"63522b79-132b-45c7-92c5-3bfbfa4c7b3a","Type":"ContainerStarted","Data":"63a9200dd531aa8a4747f03e94d5d8b1187b2e894b32d8f52795e107a7a49803"}
Jul 24 17:17:58.078871 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:58.078841    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-7c7jv" event={"ID":"a14d12ef-f2fc-45d2-9cd5-5e21b1b1f670","Type":"ContainerStarted","Data":"ed9087ed200e603153e2fed1bdea6447eb04131334b31bd8bd90635adc8f6e8e"}
Jul 24 17:17:58.116164 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:58.116088    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-7c7jv" podStartSLOduration=31.91655829 podStartE2EDuration="37.11607141s" totalImagesPullingTime="5.19951312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:17:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:17:58.11548361 +0000 UTC m=+40.842639824" watchObservedRunningTime="2026-07-24 17:17:58.11607141 +0000 UTC m=+40.843227624"
Jul 24 17:17:58.133038 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:58.132981    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gwrrp" podStartSLOduration=1.9440017859999998 podStartE2EDuration="7.132959745s" totalImagesPullingTime="5.188957959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:17:58.131416332 +0000 UTC m=+40.858572548" watchObservedRunningTime="2026-07-24 17:17:58.132959745 +0000 UTC m=+40.860115961"
Jul 24 17:17:59.084657 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:59.084619    2562 generic.go:356] "Generic (PLEG): container finished" podID="15f25938-a5da-432f-91f3-b40dae0912d3" containerID="9e119309b97f950eddb25919e5056d2d0613b4acc647fd514e760a6699277b27" exitCode=0
Jul 24 17:17:59.085135 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:59.084709    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-45mn5" event={"ID":"15f25938-a5da-432f-91f3-b40dae0912d3","Type":"ContainerDied","Data":"9e119309b97f950eddb25919e5056d2d0613b4acc647fd514e760a6699277b27"}
Jul 24 17:17:59.316333 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:59.316296    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-z2whv_1a0c97c1-3fee-45af-bc53-36991e313aa8/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:59.374612 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:59.374520    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-z2whv_1a0c97c1-3fee-45af-bc53-36991e313aa8/konnectivity-agent/0.log" limit=1024
Jul 24 17:17:59.470741 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:59.470690    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls\") pod \"dns-default-bf5b8\" (UID: \"4438603e-bddf-4e8f-979b-9f7a58c7e698\") " pod="openshift-dns/dns-default-bf5b8"
Jul 24 17:17:59.470955 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:59.470755    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:17:59.470955 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:17:59.470794    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4cmjc\" (UID: \"f3d1e52c-502d-496f-bafd-2fcee624d002\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4cmjc"
Jul 24 17:17:59.470955 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:59.470877    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:17:59.470955 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:59.470897    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 17:17:59.470955 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:59.470900    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 17:17:59.470955 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:59.470916    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75594549dc-q5krg: secret "image-registry-tls" not found
Jul 24 17:17:59.471188 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:59.470963    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert podName:f3d1e52c-502d-496f-bafd-2fcee624d002 nodeName:}" failed. No retries permitted until 2026-07-24 17:18:07.470948568 +0000 UTC m=+50.198104761 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4cmjc" (UID: "f3d1e52c-502d-496f-bafd-2fcee624d002") : secret "networking-console-plugin-cert" not found
Jul 24 17:17:59.471188 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:59.470977    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls podName:615228ee-d6a8-4566-b5e7-82d5a27bf812 nodeName:}" failed. No retries permitted until 2026-07-24 17:18:07.470971537 +0000 UTC m=+50.198127729 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls") pod "image-registry-75594549dc-q5krg" (UID: "615228ee-d6a8-4566-b5e7-82d5a27bf812") : secret "image-registry-tls" not found
Jul 24 17:17:59.471188 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:17:59.470989    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls podName:4438603e-bddf-4e8f-979b-9f7a58c7e698 nodeName:}" failed. No retries permitted until 2026-07-24 17:18:07.470983695 +0000 UTC m=+50.198139886 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls") pod "dns-default-bf5b8" (UID: "4438603e-bddf-4e8f-979b-9f7a58c7e698") : secret "dns-default-metrics-tls" not found
Jul 24 17:18:00.090978 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:00.090944    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-45mn5" event={"ID":"15f25938-a5da-432f-91f3-b40dae0912d3","Type":"ContainerStarted","Data":"f6e16647b4eadb5e2b5756079b8081711c5cfaf59a9be89219e0741cf822c670"}
Jul 24 17:18:00.114649 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:00.114598    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-45mn5" podStartSLOduration=13.959439739 podStartE2EDuration="43.11457658s" totalImagesPullingTime="29.155136841s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:17:17 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:18:00.112807452 +0000 UTC m=+42.839963666" watchObservedRunningTime="2026-07-24 17:18:00.11457658 +0000 UTC m=+42.841732796"
Jul 24 17:18:00.238900 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:00.238838    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-z2whv_1a0c97c1-3fee-45af-bc53-36991e313aa8/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:00.336429 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:00.336390    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-z2whv_1a0c97c1-3fee-45af-bc53-36991e313aa8/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:03.289434 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:03.289398    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-z2whv_1a0c97c1-3fee-45af-bc53-36991e313aa8/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:03.343280 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:03.343240    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-z2whv_1a0c97c1-3fee-45af-bc53-36991e313aa8/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:04.360176 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:04.360117    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-z2whv_1a0c97c1-3fee-45af-bc53-36991e313aa8/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:04.429317 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:04.429280    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-z2whv_1a0c97c1-3fee-45af-bc53-36991e313aa8/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:04.552928 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:04.552890    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-z2whv_1a0c97c1-3fee-45af-bc53-36991e313aa8/konnectivity-agent/0.log" limit=1024
Jul 24 17:18:07.534355 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:07.534321    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls\") pod \"dns-default-bf5b8\" (UID: \"4438603e-bddf-4e8f-979b-9f7a58c7e698\") " pod="openshift-dns/dns-default-bf5b8"
Jul 24 17:18:07.534788 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:07.534375    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:18:07.534788 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:18:07.534469    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 17:18:07.534788 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:18:07.534481    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75594549dc-q5krg: secret "image-registry-tls" not found
Jul 24 17:18:07.534788 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:18:07.534485    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:18:07.534788 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:07.534501    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4cmjc\" (UID: \"f3d1e52c-502d-496f-bafd-2fcee624d002\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4cmjc"
Jul 24 17:18:07.534788 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:18:07.534529    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls podName:615228ee-d6a8-4566-b5e7-82d5a27bf812 nodeName:}" failed. No retries permitted until 2026-07-24 17:18:23.534516941 +0000 UTC m=+66.261673137 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls") pod "image-registry-75594549dc-q5krg" (UID: "615228ee-d6a8-4566-b5e7-82d5a27bf812") : secret "image-registry-tls" not found
Jul 24 17:18:07.534788 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:18:07.534573    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 17:18:07.534788 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:18:07.534573    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls podName:4438603e-bddf-4e8f-979b-9f7a58c7e698 nodeName:}" failed. No retries permitted until 2026-07-24 17:18:23.534558287 +0000 UTC m=+66.261714491 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls") pod "dns-default-bf5b8" (UID: "4438603e-bddf-4e8f-979b-9f7a58c7e698") : secret "dns-default-metrics-tls" not found
Jul 24 17:18:07.534788 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:18:07.534639    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert podName:f3d1e52c-502d-496f-bafd-2fcee624d002 nodeName:}" failed. No retries permitted until 2026-07-24 17:18:23.534616956 +0000 UTC m=+66.261773160 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4cmjc" (UID: "f3d1e52c-502d-496f-bafd-2fcee624d002") : secret "networking-console-plugin-cert" not found
Jul 24 17:18:19.052968 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:19.052939    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kwq9x"
Jul 24 17:18:23.445107 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:23.445065    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs\") pod \"network-metrics-daemon-pjjdg\" (UID: \"b87310bf-1f99-43b7-9731-29966b07be99\") " pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:18:23.445107 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:23.445110    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccz6p\" (UniqueName: \"kubernetes.io/projected/179dc084-0504-4fcf-9608-07d598c9091c-kube-api-access-ccz6p\") pod \"network-check-target-k9hdw\" (UID: \"179dc084-0504-4fcf-9608-07d598c9091c\") " pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:18:23.447804 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:23.447780    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 17:18:23.448468 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:23.448449    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccz6p\" (UniqueName: \"kubernetes.io/projected/179dc084-0504-4fcf-9608-07d598c9091c-kube-api-access-ccz6p\") pod \"network-check-target-k9hdw\" (UID: \"179dc084-0504-4fcf-9608-07d598c9091c\") " pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:18:23.456194 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:18:23.456174    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 24 17:18:23.456271 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:18:23.456230    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs podName:b87310bf-1f99-43b7-9731-29966b07be99 nodeName:}" failed. No retries permitted until 2026-07-24 17:19:27.456214488 +0000 UTC m=+130.183370680 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs") pod "network-metrics-daemon-pjjdg" (UID: "b87310bf-1f99-43b7-9731-29966b07be99") : secret "metrics-daemon-secret" not found
Jul 24 17:18:23.546230 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:23.546193    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:18:23.546402 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:23.546246    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4cmjc\" (UID: \"f3d1e52c-502d-496f-bafd-2fcee624d002\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4cmjc"
Jul 24 17:18:23.546402 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:23.546284    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/602851eb-593f-4245-9365-97ef80e3eb51-original-pull-secret\") pod \"global-pull-secret-syncer-ts5kp\" (UID: \"602851eb-593f-4245-9365-97ef80e3eb51\") " pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:18:23.546402 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:23.546301    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls\") pod \"dns-default-bf5b8\" (UID: \"4438603e-bddf-4e8f-979b-9f7a58c7e698\") " pod="openshift-dns/dns-default-bf5b8"
Jul 24 17:18:23.546402 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:18:23.546380    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 17:18:23.546402 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:18:23.546396    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:18:23.546402 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:18:23.546403    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75594549dc-q5krg: secret "image-registry-tls" not found
Jul 24 17:18:23.546633 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:18:23.546405    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 17:18:23.546633 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:18:23.546459    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls podName:4438603e-bddf-4e8f-979b-9f7a58c7e698 nodeName:}" failed. No retries permitted until 2026-07-24 17:18:55.546441422 +0000 UTC m=+98.273597617 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls") pod "dns-default-bf5b8" (UID: "4438603e-bddf-4e8f-979b-9f7a58c7e698") : secret "dns-default-metrics-tls" not found
Jul 24 17:18:23.546633 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:18:23.546472    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert podName:f3d1e52c-502d-496f-bafd-2fcee624d002 nodeName:}" failed. No retries permitted until 2026-07-24 17:18:55.546466512 +0000 UTC m=+98.273622704 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4cmjc" (UID: "f3d1e52c-502d-496f-bafd-2fcee624d002") : secret "networking-console-plugin-cert" not found
Jul 24 17:18:23.546633 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:18:23.546481    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls podName:615228ee-d6a8-4566-b5e7-82d5a27bf812 nodeName:}" failed. No retries permitted until 2026-07-24 17:18:55.54647662 +0000 UTC m=+98.273632812 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls") pod "image-registry-75594549dc-q5krg" (UID: "615228ee-d6a8-4566-b5e7-82d5a27bf812") : secret "image-registry-tls" not found
Jul 24 17:18:23.548874 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:23.548855    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 17:18:23.558541 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:23.558520    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/602851eb-593f-4245-9365-97ef80e3eb51-original-pull-secret\") pod \"global-pull-secret-syncer-ts5kp\" (UID: \"602851eb-593f-4245-9365-97ef80e3eb51\") " pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:18:23.642414 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:23.642381    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ts5kp"
Jul 24 17:18:23.652381 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:23.652357    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-t6c84\""
Jul 24 17:18:23.671367 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:23.671338    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:18:23.790983 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:23.790958    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ts5kp"]
Jul 24 17:18:23.793357 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:18:23.793330    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod602851eb_593f_4245_9365_97ef80e3eb51.slice/crio-5ab622cf874d0260fb495c3f7e98de0a0c8a64553efd9607ffbe7b51e2cd8e24 WatchSource:0}: Error finding container 5ab622cf874d0260fb495c3f7e98de0a0c8a64553efd9607ffbe7b51e2cd8e24: Status 404 returned error can't find the container with id 5ab622cf874d0260fb495c3f7e98de0a0c8a64553efd9607ffbe7b51e2cd8e24
Jul 24 17:18:23.814697 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:23.814670    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k9hdw"]
Jul 24 17:18:23.817838 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:18:23.817816    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod179dc084_0504_4fcf_9608_07d598c9091c.slice/crio-e6700f699d7ed82ac545550e26bab2735e0ab328724da537a4486cfa9fa1a1ea WatchSource:0}: Error finding container e6700f699d7ed82ac545550e26bab2735e0ab328724da537a4486cfa9fa1a1ea: Status 404 returned error can't find the container with id e6700f699d7ed82ac545550e26bab2735e0ab328724da537a4486cfa9fa1a1ea
Jul 24 17:18:24.147991 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:24.147962    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ts5kp" event={"ID":"602851eb-593f-4245-9365-97ef80e3eb51","Type":"ContainerStarted","Data":"5ab622cf874d0260fb495c3f7e98de0a0c8a64553efd9607ffbe7b51e2cd8e24"}
Jul 24 17:18:24.149739 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:24.149719    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k9hdw" event={"ID":"179dc084-0504-4fcf-9608-07d598c9091c","Type":"ContainerStarted","Data":"ce5c5ecc6366acaa6f258e6673b9400c8798f520da82b845afaf73be3aa6b903"}
Jul 24 17:18:24.149860 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:24.149759    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k9hdw" event={"ID":"179dc084-0504-4fcf-9608-07d598c9091c","Type":"ContainerStarted","Data":"e6700f699d7ed82ac545550e26bab2735e0ab328724da537a4486cfa9fa1a1ea"}
Jul 24 17:18:24.149860 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:24.149851    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:18:24.166372 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:24.166322    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-k9hdw" podStartSLOduration=67.166306918 podStartE2EDuration="1m7.166306918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:17:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:18:24.165485411 +0000 UTC m=+66.892641625" watchObservedRunningTime="2026-07-24 17:18:24.166306918 +0000 UTC m=+66.893463169"
Jul 24 17:18:29.164398 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:29.164364    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ts5kp" event={"ID":"602851eb-593f-4245-9365-97ef80e3eb51","Type":"ContainerStarted","Data":"b0ad811d0bc29846826faf30890c3e9e16782eac9351d2e90ee3c7df28c4f967"}
Jul 24 17:18:29.179684 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:29.179633    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ts5kp" podStartSLOduration=66.644108486 podStartE2EDuration="1m11.179619349s" totalImagesPullingTime="4.535510863s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:17:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:18:29.179183678 +0000 UTC m=+71.906339893" watchObservedRunningTime="2026-07-24 17:18:29.179619349 +0000 UTC m=+71.906775579"
Jul 24 17:18:55.154982 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:55.154860    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-k9hdw"
Jul 24 17:18:55.588409 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:55.588380    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:18:55.588601 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:55.588442    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4cmjc\" (UID: \"f3d1e52c-502d-496f-bafd-2fcee624d002\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4cmjc"
Jul 24 17:18:55.588601 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:18:55.588493    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls\") pod \"dns-default-bf5b8\" (UID: \"4438603e-bddf-4e8f-979b-9f7a58c7e698\") " pod="openshift-dns/dns-default-bf5b8"
Jul 24 17:18:55.588601 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:18:55.588544    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 17:18:55.588601 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:18:55.588567    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75594549dc-q5krg: secret "image-registry-tls" not found
Jul 24 17:18:55.588601 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:18:55.588599    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:18:55.588798 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:18:55.588627    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls podName:615228ee-d6a8-4566-b5e7-82d5a27bf812 nodeName:}" failed. No retries permitted until 2026-07-24 17:19:59.588610786 +0000 UTC m=+162.315766977 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls") pod "image-registry-75594549dc-q5krg" (UID: "615228ee-d6a8-4566-b5e7-82d5a27bf812") : secret "image-registry-tls" not found
Jul 24 17:18:55.588798 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:18:55.588659    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls podName:4438603e-bddf-4e8f-979b-9f7a58c7e698 nodeName:}" failed. No retries permitted until 2026-07-24 17:19:59.588643763 +0000 UTC m=+162.315799961 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls") pod "dns-default-bf5b8" (UID: "4438603e-bddf-4e8f-979b-9f7a58c7e698") : secret "dns-default-metrics-tls" not found
Jul 24 17:18:55.588798 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:18:55.588599    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 17:18:55.588798 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:18:55.588732    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert podName:f3d1e52c-502d-496f-bafd-2fcee624d002 nodeName:}" failed. No retries permitted until 2026-07-24 17:19:59.588716923 +0000 UTC m=+162.315873118 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4cmjc" (UID: "f3d1e52c-502d-496f-bafd-2fcee624d002") : secret "networking-console-plugin-cert" not found
Jul 24 17:19:05.765680 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:05.765640    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-86fc948d4-zv2vg"]
Jul 24 17:19:05.768815 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:05.768794    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86fc948d4-zv2vg"
Jul 24 17:19:05.772032 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:05.772011    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 24 17:19:05.772463 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:05.772441    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 24 17:19:05.772557 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:05.772443    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 24 17:19:05.772557 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:05.772490    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 24 17:19:05.780467 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:05.780434    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-86fc948d4-zv2vg"]
Jul 24 17:19:05.860490 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:05.860453    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d1494ac6-d7ea-46da-b918-d38133d03ed3-klusterlet-config\") pod \"klusterlet-addon-workmgr-86fc948d4-zv2vg\" (UID: \"d1494ac6-d7ea-46da-b918-d38133d03ed3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86fc948d4-zv2vg"
Jul 24 17:19:05.860673 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:05.860532    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1494ac6-d7ea-46da-b918-d38133d03ed3-tmp\") pod \"klusterlet-addon-workmgr-86fc948d4-zv2vg\" (UID: \"d1494ac6-d7ea-46da-b918-d38133d03ed3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86fc948d4-zv2vg"
Jul 24 17:19:05.860673 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:05.860601    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbjc2\" (UniqueName: \"kubernetes.io/projected/d1494ac6-d7ea-46da-b918-d38133d03ed3-kube-api-access-nbjc2\") pod \"klusterlet-addon-workmgr-86fc948d4-zv2vg\" (UID: \"d1494ac6-d7ea-46da-b918-d38133d03ed3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86fc948d4-zv2vg"
Jul 24 17:19:05.961867 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:05.961835    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1494ac6-d7ea-46da-b918-d38133d03ed3-tmp\") pod \"klusterlet-addon-workmgr-86fc948d4-zv2vg\" (UID: \"d1494ac6-d7ea-46da-b918-d38133d03ed3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86fc948d4-zv2vg"
Jul 24 17:19:05.962052 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:05.961907    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbjc2\" (UniqueName: \"kubernetes.io/projected/d1494ac6-d7ea-46da-b918-d38133d03ed3-kube-api-access-nbjc2\") pod \"klusterlet-addon-workmgr-86fc948d4-zv2vg\" (UID: \"d1494ac6-d7ea-46da-b918-d38133d03ed3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86fc948d4-zv2vg"
Jul 24 17:19:05.962052 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:05.961930    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d1494ac6-d7ea-46da-b918-d38133d03ed3-klusterlet-config\") pod \"klusterlet-addon-workmgr-86fc948d4-zv2vg\" (UID: \"d1494ac6-d7ea-46da-b918-d38133d03ed3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86fc948d4-zv2vg"
Jul 24 17:19:05.962313 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:05.962294    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1494ac6-d7ea-46da-b918-d38133d03ed3-tmp\") pod \"klusterlet-addon-workmgr-86fc948d4-zv2vg\" (UID: \"d1494ac6-d7ea-46da-b918-d38133d03ed3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86fc948d4-zv2vg"
Jul 24 17:19:05.964354 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:05.964334    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d1494ac6-d7ea-46da-b918-d38133d03ed3-klusterlet-config\") pod \"klusterlet-addon-workmgr-86fc948d4-zv2vg\" (UID: \"d1494ac6-d7ea-46da-b918-d38133d03ed3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86fc948d4-zv2vg"
Jul 24 17:19:05.969971 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:05.969947    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbjc2\" (UniqueName: \"kubernetes.io/projected/d1494ac6-d7ea-46da-b918-d38133d03ed3-kube-api-access-nbjc2\") pod \"klusterlet-addon-workmgr-86fc948d4-zv2vg\" (UID: \"d1494ac6-d7ea-46da-b918-d38133d03ed3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86fc948d4-zv2vg"
Jul 24 17:19:06.077073 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:06.077000    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86fc948d4-zv2vg"
Jul 24 17:19:06.193826 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:06.193795    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-86fc948d4-zv2vg"]
Jul 24 17:19:06.196918 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:19:06.196892    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1494ac6_d7ea_46da_b918_d38133d03ed3.slice/crio-71a9bc21389f3da8fcf01044eb7104034d52834d60c56b807c067fdc3aa1b470 WatchSource:0}: Error finding container 71a9bc21389f3da8fcf01044eb7104034d52834d60c56b807c067fdc3aa1b470: Status 404 returned error can't find the container with id 71a9bc21389f3da8fcf01044eb7104034d52834d60c56b807c067fdc3aa1b470
Jul 24 17:19:06.255874 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:06.255841    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86fc948d4-zv2vg" event={"ID":"d1494ac6-d7ea-46da-b918-d38133d03ed3","Type":"ContainerStarted","Data":"71a9bc21389f3da8fcf01044eb7104034d52834d60c56b807c067fdc3aa1b470"}
Jul 24 17:19:10.268814 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:10.268772    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86fc948d4-zv2vg" event={"ID":"d1494ac6-d7ea-46da-b918-d38133d03ed3","Type":"ContainerStarted","Data":"b9f8300cf602a65edeb5b58a66df66363e955db302770bae76c790827ec8b4e7"}
Jul 24 17:19:10.269290 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:10.268971    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86fc948d4-zv2vg"
Jul 24 17:19:10.270464 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:10.270443    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86fc948d4-zv2vg"
Jul 24 17:19:10.284379 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:10.284325    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86fc948d4-zv2vg" podStartSLOduration=1.710351058 podStartE2EDuration="5.284310921s" totalImagesPullingTime="3.573959863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:19:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:19:10.284229946 +0000 UTC m=+113.011386160" watchObservedRunningTime="2026-07-24 17:19:10.284310921 +0000 UTC m=+113.011467136"
Jul 24 17:19:27.529158 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:27.529105    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs\") pod \"network-metrics-daemon-pjjdg\" (UID: \"b87310bf-1f99-43b7-9731-29966b07be99\") " pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:19:27.529657 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:19:27.529256    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 24 17:19:27.529657 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:19:27.529320    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs podName:b87310bf-1f99-43b7-9731-29966b07be99 nodeName:}" failed. No retries permitted until 2026-07-24 17:21:29.529303641 +0000 UTC m=+252.256459832 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs") pod "network-metrics-daemon-pjjdg" (UID: "b87310bf-1f99-43b7-9731-29966b07be99") : secret "metrics-daemon-secret" not found
Jul 24 17:19:54.669699 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:19:54.669660    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-75594549dc-q5krg" podUID="615228ee-d6a8-4566-b5e7-82d5a27bf812"
Jul 24 17:19:54.688800 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:19:54.688771    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4cmjc" podUID="f3d1e52c-502d-496f-bafd-2fcee624d002"
Jul 24 17:19:54.707051 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:19:54.707018    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-bf5b8" podUID="4438603e-bddf-4e8f-979b-9f7a58c7e698"
Jul 24 17:19:54.834824 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:19:54.834784    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-pjjdg" podUID="b87310bf-1f99-43b7-9731-29966b07be99"
Jul 24 17:19:55.373067 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:55.373035    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bf5b8"
Jul 24 17:19:55.373278 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:55.373035    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4cmjc"
Jul 24 17:19:55.373278 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:55.373035    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:19:59.657860 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:59.657813    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4cmjc\" (UID: \"f3d1e52c-502d-496f-bafd-2fcee624d002\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4cmjc"
Jul 24 17:19:59.658382 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:59.657876    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls\") pod \"dns-default-bf5b8\" (UID: \"4438603e-bddf-4e8f-979b-9f7a58c7e698\") " pod="openshift-dns/dns-default-bf5b8"
Jul 24 17:19:59.658382 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:19:59.657918    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:19:59.658382 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:19:59.658015    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 17:19:59.658382 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:19:59.658036    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:19:59.658382 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:19:59.658026    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 17:19:59.658382 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:19:59.658101    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls podName:4438603e-bddf-4e8f-979b-9f7a58c7e698 nodeName:}" failed. No retries permitted until 2026-07-24 17:22:01.658086637 +0000 UTC m=+284.385242832 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls") pod "dns-default-bf5b8" (UID: "4438603e-bddf-4e8f-979b-9f7a58c7e698") : secret "dns-default-metrics-tls" not found
Jul 24 17:19:59.658382 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:19:59.658108    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75594549dc-q5krg: secret "image-registry-tls" not found
Jul 24 17:19:59.658382 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:19:59.658119    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert podName:f3d1e52c-502d-496f-bafd-2fcee624d002 nodeName:}" failed. No retries permitted until 2026-07-24 17:22:01.658109666 +0000 UTC m=+284.385265862 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4cmjc" (UID: "f3d1e52c-502d-496f-bafd-2fcee624d002") : secret "networking-console-plugin-cert" not found
Jul 24 17:19:59.658382 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:19:59.658176    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls podName:615228ee-d6a8-4566-b5e7-82d5a27bf812 nodeName:}" failed. No retries permitted until 2026-07-24 17:22:01.658136493 +0000 UTC m=+284.385292700 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls") pod "image-registry-75594549dc-q5krg" (UID: "615228ee-d6a8-4566-b5e7-82d5a27bf812") : secret "image-registry-tls" not found
Jul 24 17:20:06.818221 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:20:06.818169    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:21:10.269604 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:10.269490    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86fc948d4-zv2vg" podUID="d1494ac6-d7ea-46da-b918-d38133d03ed3" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.11:8000/readyz\": dial tcp 10.132.0.11:8000: connect: connection refused"
Jul 24 17:21:10.549877 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:10.549799    2562 generic.go:356] "Generic (PLEG): container finished" podID="d1494ac6-d7ea-46da-b918-d38133d03ed3" containerID="b9f8300cf602a65edeb5b58a66df66363e955db302770bae76c790827ec8b4e7" exitCode=1
Jul 24 17:21:10.549877 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:10.549843    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86fc948d4-zv2vg" event={"ID":"d1494ac6-d7ea-46da-b918-d38133d03ed3","Type":"ContainerDied","Data":"b9f8300cf602a65edeb5b58a66df66363e955db302770bae76c790827ec8b4e7"}
Jul 24 17:21:10.550179 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:10.550164    2562 scope.go:117] "RemoveContainer" containerID="b9f8300cf602a65edeb5b58a66df66363e955db302770bae76c790827ec8b4e7"
Jul 24 17:21:11.555378 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:11.555343    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86fc948d4-zv2vg" event={"ID":"d1494ac6-d7ea-46da-b918-d38133d03ed3","Type":"ContainerStarted","Data":"972610f215cb5c58e7609a5320273d7ce7b4c19726a92cb724f7532868dd5472"}
Jul 24 17:21:11.555753 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:11.555614    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86fc948d4-zv2vg"
Jul 24 17:21:11.556130 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:11.556114    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86fc948d4-zv2vg"
Jul 24 17:21:29.610059 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:29.610026    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs\") pod \"network-metrics-daemon-pjjdg\" (UID: \"b87310bf-1f99-43b7-9731-29966b07be99\") " pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:21:29.610562 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:29.610175    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 24 17:21:29.610562 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:29.610229    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs podName:b87310bf-1f99-43b7-9731-29966b07be99 nodeName:}" failed. No retries permitted until 2026-07-24 17:23:31.610214606 +0000 UTC m=+374.337370798 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs") pod "network-metrics-daemon-pjjdg" (UID: "b87310bf-1f99-43b7-9731-29966b07be99") : secret "metrics-daemon-secret" not found
Jul 24 17:21:38.437391 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:38.437353    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-f6n5n"]
Jul 24 17:21:38.439407 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:38.439388    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-f6n5n"
Jul 24 17:21:38.441615 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:38.441594    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 24 17:21:38.442025 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:38.442007    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 24 17:21:38.442430 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:38.442414    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 24 17:21:38.442515 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:38.442414    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-jstpx\""
Jul 24 17:21:38.456341 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:38.456317    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-f6n5n"]
Jul 24 17:21:38.570956 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:38.570920    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4ef149a7-ed57-4e91-aedc-58e5072a5086-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-f6n5n\" (UID: \"4ef149a7-ed57-4e91-aedc-58e5072a5086\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-f6n5n"
Jul 24 17:21:38.571117 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:38.571040    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr9t8\" (UniqueName: \"kubernetes.io/projected/4ef149a7-ed57-4e91-aedc-58e5072a5086-kube-api-access-hr9t8\") pod \"cluster-samples-operator-6748489656-f6n5n\" (UID: \"4ef149a7-ed57-4e91-aedc-58e5072a5086\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-f6n5n"
Jul 24 17:21:38.671836 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:38.671806    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr9t8\" (UniqueName: \"kubernetes.io/projected/4ef149a7-ed57-4e91-aedc-58e5072a5086-kube-api-access-hr9t8\") pod \"cluster-samples-operator-6748489656-f6n5n\" (UID: \"4ef149a7-ed57-4e91-aedc-58e5072a5086\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-f6n5n"
Jul 24 17:21:38.671990 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:38.671847    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4ef149a7-ed57-4e91-aedc-58e5072a5086-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-f6n5n\" (UID: \"4ef149a7-ed57-4e91-aedc-58e5072a5086\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-f6n5n"
Jul 24 17:21:38.671990 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:38.671946    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 17:21:38.672069 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:38.671999    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ef149a7-ed57-4e91-aedc-58e5072a5086-samples-operator-tls podName:4ef149a7-ed57-4e91-aedc-58e5072a5086 nodeName:}" failed. No retries permitted until 2026-07-24 17:21:39.171984024 +0000 UTC m=+261.899140219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4ef149a7-ed57-4e91-aedc-58e5072a5086-samples-operator-tls") pod "cluster-samples-operator-6748489656-f6n5n" (UID: "4ef149a7-ed57-4e91-aedc-58e5072a5086") : secret "samples-operator-tls" not found
Jul 24 17:21:38.680347 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:38.680323    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr9t8\" (UniqueName: \"kubernetes.io/projected/4ef149a7-ed57-4e91-aedc-58e5072a5086-kube-api-access-hr9t8\") pod \"cluster-samples-operator-6748489656-f6n5n\" (UID: \"4ef149a7-ed57-4e91-aedc-58e5072a5086\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-f6n5n"
Jul 24 17:21:39.175897 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:39.175861    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4ef149a7-ed57-4e91-aedc-58e5072a5086-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-f6n5n\" (UID: \"4ef149a7-ed57-4e91-aedc-58e5072a5086\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-f6n5n"
Jul 24 17:21:39.176084 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:39.175970    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 17:21:39.176084 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:39.176022    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ef149a7-ed57-4e91-aedc-58e5072a5086-samples-operator-tls podName:4ef149a7-ed57-4e91-aedc-58e5072a5086 nodeName:}" failed. No retries permitted until 2026-07-24 17:21:40.176008966 +0000 UTC m=+262.903165157 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4ef149a7-ed57-4e91-aedc-58e5072a5086-samples-operator-tls") pod "cluster-samples-operator-6748489656-f6n5n" (UID: "4ef149a7-ed57-4e91-aedc-58e5072a5086") : secret "samples-operator-tls" not found
Jul 24 17:21:40.183476 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:40.183389    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4ef149a7-ed57-4e91-aedc-58e5072a5086-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-f6n5n\" (UID: \"4ef149a7-ed57-4e91-aedc-58e5072a5086\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-f6n5n"
Jul 24 17:21:40.183835 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:40.183535    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 17:21:40.183835 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:40.183600    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ef149a7-ed57-4e91-aedc-58e5072a5086-samples-operator-tls podName:4ef149a7-ed57-4e91-aedc-58e5072a5086 nodeName:}" failed. No retries permitted until 2026-07-24 17:21:42.183584843 +0000 UTC m=+264.910741035 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4ef149a7-ed57-4e91-aedc-58e5072a5086-samples-operator-tls") pod "cluster-samples-operator-6748489656-f6n5n" (UID: "4ef149a7-ed57-4e91-aedc-58e5072a5086") : secret "samples-operator-tls" not found
Jul 24 17:21:42.198316 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:42.198279    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4ef149a7-ed57-4e91-aedc-58e5072a5086-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-f6n5n\" (UID: \"4ef149a7-ed57-4e91-aedc-58e5072a5086\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-f6n5n"
Jul 24 17:21:42.198713 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:42.198392    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 17:21:42.198713 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:42.198452    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ef149a7-ed57-4e91-aedc-58e5072a5086-samples-operator-tls podName:4ef149a7-ed57-4e91-aedc-58e5072a5086 nodeName:}" failed. No retries permitted until 2026-07-24 17:21:46.198437537 +0000 UTC m=+268.925593728 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4ef149a7-ed57-4e91-aedc-58e5072a5086-samples-operator-tls") pod "cluster-samples-operator-6748489656-f6n5n" (UID: "4ef149a7-ed57-4e91-aedc-58e5072a5086") : secret "samples-operator-tls" not found
Jul 24 17:21:43.382702 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:43.382663    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-hbbxj"]
Jul 24 17:21:43.384536 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:43.384521    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-hbbxj"
Jul 24 17:21:43.387506 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:43.387488    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-6hjm6\""
Jul 24 17:21:43.387744 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:43.387731    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 24 17:21:43.387834 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:43.387812    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 24 17:21:43.395618 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:43.395597    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-hbbxj"]
Jul 24 17:21:43.507702 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:43.507668    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7jhq\" (UniqueName: \"kubernetes.io/projected/37a1af80-b8c7-4056-ab27-34a4aef0e70e-kube-api-access-d7jhq\") pod \"migrator-6865c9967-hbbxj\" (UID: \"37a1af80-b8c7-4056-ab27-34a4aef0e70e\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-hbbxj"
Jul 24 17:21:43.608243 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:43.608201    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7jhq\" (UniqueName: \"kubernetes.io/projected/37a1af80-b8c7-4056-ab27-34a4aef0e70e-kube-api-access-d7jhq\") pod \"migrator-6865c9967-hbbxj\" (UID: \"37a1af80-b8c7-4056-ab27-34a4aef0e70e\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-hbbxj"
Jul 24 17:21:43.616414 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:43.616389    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7jhq\" (UniqueName: \"kubernetes.io/projected/37a1af80-b8c7-4056-ab27-34a4aef0e70e-kube-api-access-d7jhq\") pod \"migrator-6865c9967-hbbxj\" (UID: \"37a1af80-b8c7-4056-ab27-34a4aef0e70e\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-hbbxj"
Jul 24 17:21:43.693215 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:43.693117    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-hbbxj"
Jul 24 17:21:43.806520 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:43.806490    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-hbbxj"]
Jul 24 17:21:43.809272 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:21:43.809243    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37a1af80_b8c7_4056_ab27_34a4aef0e70e.slice/crio-95faddbc864ba1c05e0a684f7b1f2c35c505bede0600bfbdb3e5db10c43d57f3 WatchSource:0}: Error finding container 95faddbc864ba1c05e0a684f7b1f2c35c505bede0600bfbdb3e5db10c43d57f3: Status 404 returned error can't find the container with id 95faddbc864ba1c05e0a684f7b1f2c35c505bede0600bfbdb3e5db10c43d57f3
Jul 24 17:21:44.188096 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:44.188069    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kp5nk"]
Jul 24 17:21:44.190751 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:44.190734    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kp5nk"
Jul 24 17:21:44.192953 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:44.192931    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bstvq\""
Jul 24 17:21:44.193076 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:44.193059    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 24 17:21:44.193372 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:44.193359    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 24 17:21:44.193774 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:44.193758    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 24 17:21:44.194259 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:44.194244    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 24 17:21:44.209254 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:44.209233    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kp5nk"]
Jul 24 17:21:44.314518 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:44.314480    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/237e290b-e1f0-4685-b85f-a0cf55d124e5-crio-socket\") pod \"insights-runtime-extractor-kp5nk\" (UID: \"237e290b-e1f0-4685-b85f-a0cf55d124e5\") " pod="openshift-insights/insights-runtime-extractor-kp5nk"
Jul 24 17:21:44.314518 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:44.314526    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/237e290b-e1f0-4685-b85f-a0cf55d124e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kp5nk\" (UID: \"237e290b-e1f0-4685-b85f-a0cf55d124e5\") " pod="openshift-insights/insights-runtime-extractor-kp5nk"
Jul 24 17:21:44.314689 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:44.314545    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/237e290b-e1f0-4685-b85f-a0cf55d124e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kp5nk\" (UID: \"237e290b-e1f0-4685-b85f-a0cf55d124e5\") " pod="openshift-insights/insights-runtime-extractor-kp5nk"
Jul 24 17:21:44.314689 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:44.314613    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwvzb\" (UniqueName: \"kubernetes.io/projected/237e290b-e1f0-4685-b85f-a0cf55d124e5-kube-api-access-rwvzb\") pod \"insights-runtime-extractor-kp5nk\" (UID: \"237e290b-e1f0-4685-b85f-a0cf55d124e5\") " pod="openshift-insights/insights-runtime-extractor-kp5nk"
Jul 24 17:21:44.314769 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:44.314691    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/237e290b-e1f0-4685-b85f-a0cf55d124e5-data-volume\") pod \"insights-runtime-extractor-kp5nk\" (UID: \"237e290b-e1f0-4685-b85f-a0cf55d124e5\") " pod="openshift-insights/insights-runtime-extractor-kp5nk"
Jul 24 17:21:44.415942 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:44.415913    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/237e290b-e1f0-4685-b85f-a0cf55d124e5-crio-socket\") pod \"insights-runtime-extractor-kp5nk\" (UID: \"237e290b-e1f0-4685-b85f-a0cf55d124e5\") " pod="openshift-insights/insights-runtime-extractor-kp5nk"
Jul 24 17:21:44.416330 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:44.415955    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/237e290b-e1f0-4685-b85f-a0cf55d124e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kp5nk\" (UID: \"237e290b-e1f0-4685-b85f-a0cf55d124e5\") " pod="openshift-insights/insights-runtime-extractor-kp5nk"
Jul 24 17:21:44.416330 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:44.415971    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/237e290b-e1f0-4685-b85f-a0cf55d124e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kp5nk\" (UID: \"237e290b-e1f0-4685-b85f-a0cf55d124e5\") " pod="openshift-insights/insights-runtime-extractor-kp5nk"
Jul 24 17:21:44.416330 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:44.416029    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 17:21:44.416330 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:44.416039    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/237e290b-e1f0-4685-b85f-a0cf55d124e5-crio-socket\") pod \"insights-runtime-extractor-kp5nk\" (UID: \"237e290b-e1f0-4685-b85f-a0cf55d124e5\") " pod="openshift-insights/insights-runtime-extractor-kp5nk"
Jul 24 17:21:44.416330 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:44.416081    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/237e290b-e1f0-4685-b85f-a0cf55d124e5-kube-rbac-proxy-cm podName:237e290b-e1f0-4685-b85f-a0cf55d124e5 nodeName:}" failed. No retries permitted until 2026-07-24 17:21:44.916066566 +0000 UTC m=+267.643222762 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/237e290b-e1f0-4685-b85f-a0cf55d124e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kp5nk" (UID: "237e290b-e1f0-4685-b85f-a0cf55d124e5") : configmap "kube-rbac-proxy" not found
Jul 24 17:21:44.416330 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:44.416077    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwvzb\" (UniqueName: \"kubernetes.io/projected/237e290b-e1f0-4685-b85f-a0cf55d124e5-kube-api-access-rwvzb\") pod \"insights-runtime-extractor-kp5nk\" (UID: \"237e290b-e1f0-4685-b85f-a0cf55d124e5\") " pod="openshift-insights/insights-runtime-extractor-kp5nk"
Jul 24 17:21:44.416330 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:44.416137    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 17:21:44.416330 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:44.416195    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/237e290b-e1f0-4685-b85f-a0cf55d124e5-insights-runtime-extractor-tls podName:237e290b-e1f0-4685-b85f-a0cf55d124e5 nodeName:}" failed. No retries permitted until 2026-07-24 17:21:44.916183635 +0000 UTC m=+267.643339831 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/237e290b-e1f0-4685-b85f-a0cf55d124e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kp5nk" (UID: "237e290b-e1f0-4685-b85f-a0cf55d124e5") : secret "insights-runtime-extractor-tls" not found
Jul 24 17:21:44.416330 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:44.416195    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/237e290b-e1f0-4685-b85f-a0cf55d124e5-data-volume\") pod \"insights-runtime-extractor-kp5nk\" (UID: \"237e290b-e1f0-4685-b85f-a0cf55d124e5\") " pod="openshift-insights/insights-runtime-extractor-kp5nk"
Jul 24 17:21:44.416618 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:44.416469    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/237e290b-e1f0-4685-b85f-a0cf55d124e5-data-volume\") pod \"insights-runtime-extractor-kp5nk\" (UID: \"237e290b-e1f0-4685-b85f-a0cf55d124e5\") " pod="openshift-insights/insights-runtime-extractor-kp5nk"
Jul 24 17:21:44.427370 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:44.427342    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwvzb\" (UniqueName: \"kubernetes.io/projected/237e290b-e1f0-4685-b85f-a0cf55d124e5-kube-api-access-rwvzb\") pod \"insights-runtime-extractor-kp5nk\" (UID: \"237e290b-e1f0-4685-b85f-a0cf55d124e5\") " pod="openshift-insights/insights-runtime-extractor-kp5nk"
Jul 24 17:21:44.650572 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:44.650538    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-hbbxj" event={"ID":"37a1af80-b8c7-4056-ab27-34a4aef0e70e","Type":"ContainerStarted","Data":"95faddbc864ba1c05e0a684f7b1f2c35c505bede0600bfbdb3e5db10c43d57f3"}
Jul 24 17:21:44.919887 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:44.919799    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/237e290b-e1f0-4685-b85f-a0cf55d124e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kp5nk\" (UID: \"237e290b-e1f0-4685-b85f-a0cf55d124e5\") " pod="openshift-insights/insights-runtime-extractor-kp5nk"
Jul 24 17:21:44.919887 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:44.919852    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/237e290b-e1f0-4685-b85f-a0cf55d124e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kp5nk\" (UID: \"237e290b-e1f0-4685-b85f-a0cf55d124e5\") " pod="openshift-insights/insights-runtime-extractor-kp5nk"
Jul 24 17:21:44.920115 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:44.919933    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 17:21:44.920115 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:44.920008    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/237e290b-e1f0-4685-b85f-a0cf55d124e5-kube-rbac-proxy-cm podName:237e290b-e1f0-4685-b85f-a0cf55d124e5 nodeName:}" failed. No retries permitted until 2026-07-24 17:21:45.919986907 +0000 UTC m=+268.647143100 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/237e290b-e1f0-4685-b85f-a0cf55d124e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kp5nk" (UID: "237e290b-e1f0-4685-b85f-a0cf55d124e5") : configmap "kube-rbac-proxy" not found
Jul 24 17:21:44.920115 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:44.920028    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 17:21:44.920115 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:44.920066    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/237e290b-e1f0-4685-b85f-a0cf55d124e5-insights-runtime-extractor-tls podName:237e290b-e1f0-4685-b85f-a0cf55d124e5 nodeName:}" failed. No retries permitted until 2026-07-24 17:21:45.92005529 +0000 UTC m=+268.647211493 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/237e290b-e1f0-4685-b85f-a0cf55d124e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kp5nk" (UID: "237e290b-e1f0-4685-b85f-a0cf55d124e5") : secret "insights-runtime-extractor-tls" not found
Jul 24 17:21:45.654829 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:45.654790    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-hbbxj" event={"ID":"37a1af80-b8c7-4056-ab27-34a4aef0e70e","Type":"ContainerStarted","Data":"76915f42ec5c3d1f2f6f5f49eba370c38adb4d5271669bff3ffa8f79eb1362ed"}
Jul 24 17:21:45.655259 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:45.654886    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-hbbxj" event={"ID":"37a1af80-b8c7-4056-ab27-34a4aef0e70e","Type":"ContainerStarted","Data":"c64c9680a5eb43d0e0f35c6452f1c0c31af6230954bd333198331f6aef0bdaf5"}
Jul 24 17:21:45.671687 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:45.671648    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-hbbxj" podStartSLOduration=1.307125441 podStartE2EDuration="2.671635928s" totalImagesPullingTime="1.364510487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:21:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:21:45.670468112 +0000 UTC m=+268.397624325" watchObservedRunningTime="2026-07-24 17:21:45.671635928 +0000 UTC m=+268.398792141"
Jul 24 17:21:45.927162 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:45.927058    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/237e290b-e1f0-4685-b85f-a0cf55d124e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kp5nk\" (UID: \"237e290b-e1f0-4685-b85f-a0cf55d124e5\") " pod="openshift-insights/insights-runtime-extractor-kp5nk"
Jul 24 17:21:45.927162 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:45.927100    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/237e290b-e1f0-4685-b85f-a0cf55d124e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kp5nk\" (UID: \"237e290b-e1f0-4685-b85f-a0cf55d124e5\") " pod="openshift-insights/insights-runtime-extractor-kp5nk"
Jul 24 17:21:45.927379 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:45.927210    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 17:21:45.927379 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:45.927252    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 17:21:45.927379 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:45.927272    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/237e290b-e1f0-4685-b85f-a0cf55d124e5-kube-rbac-proxy-cm podName:237e290b-e1f0-4685-b85f-a0cf55d124e5 nodeName:}" failed. No retries permitted until 2026-07-24 17:21:47.927255976 +0000 UTC m=+270.654412168 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/237e290b-e1f0-4685-b85f-a0cf55d124e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kp5nk" (UID: "237e290b-e1f0-4685-b85f-a0cf55d124e5") : configmap "kube-rbac-proxy" not found
Jul 24 17:21:45.927379 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:45.927301    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/237e290b-e1f0-4685-b85f-a0cf55d124e5-insights-runtime-extractor-tls podName:237e290b-e1f0-4685-b85f-a0cf55d124e5 nodeName:}" failed. No retries permitted until 2026-07-24 17:21:47.927287644 +0000 UTC m=+270.654443849 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/237e290b-e1f0-4685-b85f-a0cf55d124e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kp5nk" (UID: "237e290b-e1f0-4685-b85f-a0cf55d124e5") : secret "insights-runtime-extractor-tls" not found
Jul 24 17:21:46.229277 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:46.229194    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4ef149a7-ed57-4e91-aedc-58e5072a5086-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-f6n5n\" (UID: \"4ef149a7-ed57-4e91-aedc-58e5072a5086\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-f6n5n"
Jul 24 17:21:46.229417 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:46.229321    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 17:21:46.229417 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:46.229378    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ef149a7-ed57-4e91-aedc-58e5072a5086-samples-operator-tls podName:4ef149a7-ed57-4e91-aedc-58e5072a5086 nodeName:}" failed. No retries permitted until 2026-07-24 17:21:54.229363965 +0000 UTC m=+276.956520158 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4ef149a7-ed57-4e91-aedc-58e5072a5086-samples-operator-tls") pod "cluster-samples-operator-6748489656-f6n5n" (UID: "4ef149a7-ed57-4e91-aedc-58e5072a5086") : secret "samples-operator-tls" not found
Jul 24 17:21:46.998320 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:46.998294    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-h2fn4_ab25ce71-a92b-4f56-b9df-2c2812e744ef/dns-node-resolver/0.log"
Jul 24 17:21:47.941521 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:47.941473    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/237e290b-e1f0-4685-b85f-a0cf55d124e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kp5nk\" (UID: \"237e290b-e1f0-4685-b85f-a0cf55d124e5\") " pod="openshift-insights/insights-runtime-extractor-kp5nk"
Jul 24 17:21:47.941521 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:47.941518    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/237e290b-e1f0-4685-b85f-a0cf55d124e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kp5nk\" (UID: \"237e290b-e1f0-4685-b85f-a0cf55d124e5\") " pod="openshift-insights/insights-runtime-extractor-kp5nk"
Jul 24 17:21:47.941778 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:47.941595    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 17:21:47.941778 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:47.941613    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 17:21:47.941778 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:47.941666    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/237e290b-e1f0-4685-b85f-a0cf55d124e5-kube-rbac-proxy-cm podName:237e290b-e1f0-4685-b85f-a0cf55d124e5 nodeName:}" failed. No retries permitted until 2026-07-24 17:21:51.941649263 +0000 UTC m=+274.668805458 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/237e290b-e1f0-4685-b85f-a0cf55d124e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kp5nk" (UID: "237e290b-e1f0-4685-b85f-a0cf55d124e5") : configmap "kube-rbac-proxy" not found
Jul 24 17:21:47.941778 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:47.941679    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/237e290b-e1f0-4685-b85f-a0cf55d124e5-insights-runtime-extractor-tls podName:237e290b-e1f0-4685-b85f-a0cf55d124e5 nodeName:}" failed. No retries permitted until 2026-07-24 17:21:51.941673214 +0000 UTC m=+274.668829406 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/237e290b-e1f0-4685-b85f-a0cf55d124e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kp5nk" (UID: "237e290b-e1f0-4685-b85f-a0cf55d124e5") : secret "insights-runtime-extractor-tls" not found
Jul 24 17:21:48.206589 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:48.206512    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tdnbn_44c623e2-f2fa-418d-986a-af9370891a35/node-ca/0.log"
Jul 24 17:21:48.598484 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:48.598458    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gwrrp_63522b79-132b-45c7-92c5-3bfbfa4c7b3a/serve-healthcheck-canary/0.log"
Jul 24 17:21:49.199550 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:49.199517    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-hbbxj_37a1af80-b8c7-4056-ab27-34a4aef0e70e/migrator/0.log"
Jul 24 17:21:49.398024 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:49.397993    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-hbbxj_37a1af80-b8c7-4056-ab27-34a4aef0e70e/graceful-termination/0.log"
Jul 24 17:21:51.976811 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:51.976778    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/237e290b-e1f0-4685-b85f-a0cf55d124e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kp5nk\" (UID: \"237e290b-e1f0-4685-b85f-a0cf55d124e5\") " pod="openshift-insights/insights-runtime-extractor-kp5nk"
Jul 24 17:21:51.976811 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:51.976814    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/237e290b-e1f0-4685-b85f-a0cf55d124e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kp5nk\" (UID: \"237e290b-e1f0-4685-b85f-a0cf55d124e5\") " pod="openshift-insights/insights-runtime-extractor-kp5nk"
Jul 24 17:21:51.977244 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:51.976900    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 17:21:51.977244 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:51.976923    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 17:21:51.977244 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:51.976973    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/237e290b-e1f0-4685-b85f-a0cf55d124e5-insights-runtime-extractor-tls podName:237e290b-e1f0-4685-b85f-a0cf55d124e5 nodeName:}" failed. No retries permitted until 2026-07-24 17:21:59.976959347 +0000 UTC m=+282.704115539 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/237e290b-e1f0-4685-b85f-a0cf55d124e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kp5nk" (UID: "237e290b-e1f0-4685-b85f-a0cf55d124e5") : secret "insights-runtime-extractor-tls" not found
Jul 24 17:21:51.977244 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:51.976991    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/237e290b-e1f0-4685-b85f-a0cf55d124e5-kube-rbac-proxy-cm podName:237e290b-e1f0-4685-b85f-a0cf55d124e5 nodeName:}" failed. No retries permitted until 2026-07-24 17:21:59.976985205 +0000 UTC m=+282.704141396 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/237e290b-e1f0-4685-b85f-a0cf55d124e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kp5nk" (UID: "237e290b-e1f0-4685-b85f-a0cf55d124e5") : configmap "kube-rbac-proxy" not found
Jul 24 17:21:54.295762 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:54.295726    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4ef149a7-ed57-4e91-aedc-58e5072a5086-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-f6n5n\" (UID: \"4ef149a7-ed57-4e91-aedc-58e5072a5086\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-f6n5n"
Jul 24 17:21:54.298030 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:54.298009    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4ef149a7-ed57-4e91-aedc-58e5072a5086-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-f6n5n\" (UID: \"4ef149a7-ed57-4e91-aedc-58e5072a5086\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-f6n5n"
Jul 24 17:21:54.348268 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:54.348230    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-f6n5n"
Jul 24 17:21:54.464404 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:54.464375    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-f6n5n"]
Jul 24 17:21:54.680427 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:54.680399    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-f6n5n" event={"ID":"4ef149a7-ed57-4e91-aedc-58e5072a5086","Type":"ContainerStarted","Data":"1586876695b3c02a32b6a47204d48c03dd55f5de702213b64082176be54947fe"}
Jul 24 17:21:56.689580 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:56.689545    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-f6n5n" event={"ID":"4ef149a7-ed57-4e91-aedc-58e5072a5086","Type":"ContainerStarted","Data":"172fa5466896984f94e3720e7930176a468b58c3d5139835afc844893927bcbd"}
Jul 24 17:21:56.690028 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:56.689606    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-f6n5n" event={"ID":"4ef149a7-ed57-4e91-aedc-58e5072a5086","Type":"ContainerStarted","Data":"8c9616d87e5336c3102a9cf51087115ee42747d2de0938a98e96e105331db72b"}
Jul 24 17:21:56.706603 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:56.706553    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-f6n5n" podStartSLOduration=16.917241215 podStartE2EDuration="18.706538918s" totalImagesPullingTime="1.789297703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:21:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:21:56.705346234 +0000 UTC m=+279.432502447" watchObservedRunningTime="2026-07-24 17:21:56.706538918 +0000 UTC m=+279.433695131"
Jul 24 17:21:58.374414 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:58.374350    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-75594549dc-q5krg" podUID="615228ee-d6a8-4566-b5e7-82d5a27bf812"
Jul 24 17:21:58.374414 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:58.374349    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4cmjc" podUID="f3d1e52c-502d-496f-bafd-2fcee624d002"
Jul 24 17:21:58.374841 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:21:58.374350    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-bf5b8" podUID="4438603e-bddf-4e8f-979b-9f7a58c7e698"
Jul 24 17:21:58.695250 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:58.695173    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4cmjc"
Jul 24 17:21:58.695416 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:58.695173    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:21:58.695416 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:21:58.695173    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bf5b8"
Jul 24 17:22:00.039688 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:00.039649    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/237e290b-e1f0-4685-b85f-a0cf55d124e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kp5nk\" (UID: \"237e290b-e1f0-4685-b85f-a0cf55d124e5\") " pod="openshift-insights/insights-runtime-extractor-kp5nk"
Jul 24 17:22:00.039688 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:00.039690    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/237e290b-e1f0-4685-b85f-a0cf55d124e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kp5nk\" (UID: \"237e290b-e1f0-4685-b85f-a0cf55d124e5\") " pod="openshift-insights/insights-runtime-extractor-kp5nk"
Jul 24 17:22:00.040346 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:00.040323    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/237e290b-e1f0-4685-b85f-a0cf55d124e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kp5nk\" (UID: \"237e290b-e1f0-4685-b85f-a0cf55d124e5\") " pod="openshift-insights/insights-runtime-extractor-kp5nk"
Jul 24 17:22:00.041904 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:00.041887    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/237e290b-e1f0-4685-b85f-a0cf55d124e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kp5nk\" (UID: \"237e290b-e1f0-4685-b85f-a0cf55d124e5\") " pod="openshift-insights/insights-runtime-extractor-kp5nk"
Jul 24 17:22:00.099852 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:00.099818    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kp5nk"
Jul 24 17:22:00.223291 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:00.223256    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kp5nk"]
Jul 24 17:22:00.226369 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:22:00.226335    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod237e290b_e1f0_4685_b85f_a0cf55d124e5.slice/crio-eca96b70bfce54e3da19f014e68d01d6a3f03dd7074ef61e3434a9db9610dc57 WatchSource:0}: Error finding container eca96b70bfce54e3da19f014e68d01d6a3f03dd7074ef61e3434a9db9610dc57: Status 404 returned error can't find the container with id eca96b70bfce54e3da19f014e68d01d6a3f03dd7074ef61e3434a9db9610dc57
Jul 24 17:22:00.702181 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:00.702127    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kp5nk" event={"ID":"237e290b-e1f0-4685-b85f-a0cf55d124e5","Type":"ContainerStarted","Data":"eca96b70bfce54e3da19f014e68d01d6a3f03dd7074ef61e3434a9db9610dc57"}
Jul 24 17:22:01.753585 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:01.753547    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls\") pod \"dns-default-bf5b8\" (UID: \"4438603e-bddf-4e8f-979b-9f7a58c7e698\") " pod="openshift-dns/dns-default-bf5b8"
Jul 24 17:22:01.754028 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:01.753600    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:22:01.754028 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:01.753642    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4cmjc\" (UID: \"f3d1e52c-502d-496f-bafd-2fcee624d002\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4cmjc"
Jul 24 17:22:01.756416 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:01.756391    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f3d1e52c-502d-496f-bafd-2fcee624d002-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4cmjc\" (UID: \"f3d1e52c-502d-496f-bafd-2fcee624d002\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4cmjc"
Jul 24 17:22:01.756590 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:01.756566    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4438603e-bddf-4e8f-979b-9f7a58c7e698-metrics-tls\") pod \"dns-default-bf5b8\" (UID: \"4438603e-bddf-4e8f-979b-9f7a58c7e698\") " pod="openshift-dns/dns-default-bf5b8"
Jul 24 17:22:01.756710 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:01.756695    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls\") pod \"image-registry-75594549dc-q5krg\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") " pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:22:01.999358 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:01.999325    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-tkvbh\""
Jul 24 17:22:01.999538 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:01.999366    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-r2b86\""
Jul 24 17:22:01.999538 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:01.999337    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-p7fvc\""
Jul 24 17:22:02.006127 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:02.006054    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4cmjc"
Jul 24 17:22:02.006127 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:02.006105    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:22:02.006332 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:02.006153    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bf5b8"
Jul 24 17:22:02.167552 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:02.166436    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bf5b8"]
Jul 24 17:22:02.170414 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:22:02.170385    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4438603e_bddf_4e8f_979b_9f7a58c7e698.slice/crio-3339a55a2ba7f2e79e146e5b2a92db80b804958a9fad152e3cdd50823b11d0f6 WatchSource:0}: Error finding container 3339a55a2ba7f2e79e146e5b2a92db80b804958a9fad152e3cdd50823b11d0f6: Status 404 returned error can't find the container with id 3339a55a2ba7f2e79e146e5b2a92db80b804958a9fad152e3cdd50823b11d0f6
Jul 24 17:22:02.393414 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:02.393385    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-4cmjc"]
Jul 24 17:22:02.397004 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:02.396976    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75594549dc-q5krg"]
Jul 24 17:22:02.397922 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:22:02.397897    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3d1e52c_502d_496f_bafd_2fcee624d002.slice/crio-497a0a1cf6bb7a18763e23b40a799251580d03f851da0fe07b79e81d53003c9c WatchSource:0}: Error finding container 497a0a1cf6bb7a18763e23b40a799251580d03f851da0fe07b79e81d53003c9c: Status 404 returned error can't find the container with id 497a0a1cf6bb7a18763e23b40a799251580d03f851da0fe07b79e81d53003c9c
Jul 24 17:22:02.402348 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:22:02.402321    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod615228ee_d6a8_4566_b5e7_82d5a27bf812.slice/crio-883d52b090d2048341a12d449f53861317ad89c9fae1dca5995ba2dc920b18ac WatchSource:0}: Error finding container 883d52b090d2048341a12d449f53861317ad89c9fae1dca5995ba2dc920b18ac: Status 404 returned error can't find the container with id 883d52b090d2048341a12d449f53861317ad89c9fae1dca5995ba2dc920b18ac
Jul 24 17:22:02.710201 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:02.710109    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bf5b8" event={"ID":"4438603e-bddf-4e8f-979b-9f7a58c7e698","Type":"ContainerStarted","Data":"3339a55a2ba7f2e79e146e5b2a92db80b804958a9fad152e3cdd50823b11d0f6"}
Jul 24 17:22:02.712105 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:02.712062    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75594549dc-q5krg" event={"ID":"615228ee-d6a8-4566-b5e7-82d5a27bf812","Type":"ContainerStarted","Data":"544ea5da72e9d95f353ea3e95093878da59987256e42460515e52351ace44361"}
Jul 24 17:22:02.712255 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:02.712110    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75594549dc-q5krg" event={"ID":"615228ee-d6a8-4566-b5e7-82d5a27bf812","Type":"ContainerStarted","Data":"883d52b090d2048341a12d449f53861317ad89c9fae1dca5995ba2dc920b18ac"}
Jul 24 17:22:02.712537 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:02.712477    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:22:02.713829 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:02.713804    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4cmjc" event={"ID":"f3d1e52c-502d-496f-bafd-2fcee624d002","Type":"ContainerStarted","Data":"497a0a1cf6bb7a18763e23b40a799251580d03f851da0fe07b79e81d53003c9c"}
Jul 24 17:22:02.750399 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:02.750355    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-75594549dc-q5krg" podStartSLOduration=284.750340365 podStartE2EDuration="4m44.750340365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:17:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:22:02.74937332 +0000 UTC m=+285.476529560" watchObservedRunningTime="2026-07-24 17:22:02.750340365 +0000 UTC m=+285.477496578"
Jul 24 17:22:06.730838 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:06.730761    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kp5nk" event={"ID":"237e290b-e1f0-4685-b85f-a0cf55d124e5","Type":"ContainerStarted","Data":"375ef1af87757e19becb092c63486611be8fd0e6af1160412f491abb5502315e"}
Jul 24 17:22:06.733684 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:06.733529    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bf5b8" event={"ID":"4438603e-bddf-4e8f-979b-9f7a58c7e698","Type":"ContainerStarted","Data":"879babc0f783fc4bd5c4260e947af21b41e39396f517f979b43784be85b5a66a"}
Jul 24 17:22:06.733684 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:06.733584    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bf5b8" event={"ID":"4438603e-bddf-4e8f-979b-9f7a58c7e698","Type":"ContainerStarted","Data":"fe50f49e7d8a5761ec6f07f1cd5da34a47c08e2adb8fc382dd20ca08a4aa3ee1"}
Jul 24 17:22:06.733684 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:06.733644    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bf5b8"
Jul 24 17:22:06.736410 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:06.735942    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4cmjc" event={"ID":"f3d1e52c-502d-496f-bafd-2fcee624d002","Type":"ContainerStarted","Data":"5ea1d5240358539445d9dff7f9b0817d3bfe17215e338a862c007a89608a0018"}
Jul 24 17:22:06.752338 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:06.752259    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-bf5b8" podStartSLOduration=251.905165943 podStartE2EDuration="4m15.752240703s" totalImagesPullingTime="3.84707476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:22:06.750874468 +0000 UTC m=+289.478030684" watchObservedRunningTime="2026-07-24 17:22:06.752240703 +0000 UTC m=+289.479396920"
Jul 24 17:22:09.747071 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:09.747041    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kp5nk" event={"ID":"237e290b-e1f0-4685-b85f-a0cf55d124e5","Type":"ContainerStarted","Data":"9fe8c222164a8d29da56938808867363c2e636c5dfcbecf59446e03d8450cb38"}
Jul 24 17:22:09.819051 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:22:09.818987    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-pjjdg" podUID="b87310bf-1f99-43b7-9731-29966b07be99"
Jul 24 17:22:13.765578 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:13.765547    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kp5nk" event={"ID":"237e290b-e1f0-4685-b85f-a0cf55d124e5","Type":"ContainerStarted","Data":"4974461e1325220fe82186b2801be3e8c0a4b8c9fed18e6f06f702d7d36fa796"}
Jul 24 17:22:13.782635 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:13.782556    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4cmjc" podStartSLOduration=263.116486116 podStartE2EDuration="4m26.782541863s" totalImagesPullingTime="3.666055747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:17:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:22:06.768463605 +0000 UTC m=+289.495619819" watchObservedRunningTime="2026-07-24 17:22:13.782541863 +0000 UTC m=+296.509698089"
Jul 24 17:22:13.783314 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:13.783284    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kp5nk" podStartSLOduration=17.151309539 podStartE2EDuration="29.783276414s" totalImagesPullingTime="12.631966875s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:21:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:22:13.782606633 +0000 UTC m=+296.509762874" watchObservedRunningTime="2026-07-24 17:22:13.783276414 +0000 UTC m=+296.510432628"
Jul 24 17:22:14.983932 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:14.983899    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-92hzw"]
Jul 24 17:22:14.986897 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:14.986881    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-92hzw"
Jul 24 17:22:14.989381 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:14.989349    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 24 17:22:14.989503 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:14.989360    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-hjhf7\""
Jul 24 17:22:14.994084 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:14.994061    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-92hzw"]
Jul 24 17:22:15.045387 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:15.045360    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b158ac16-fd79-4e1d-8cc3-7bdb970f49c1-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-92hzw\" (UID: \"b158ac16-fd79-4e1d-8cc3-7bdb970f49c1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-92hzw"
Jul 24 17:22:15.145841 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:15.145806    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b158ac16-fd79-4e1d-8cc3-7bdb970f49c1-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-92hzw\" (UID: \"b158ac16-fd79-4e1d-8cc3-7bdb970f49c1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-92hzw"
Jul 24 17:22:15.145996 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:22:15.145950    2562 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 24 17:22:15.146045 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:22:15.146010    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b158ac16-fd79-4e1d-8cc3-7bdb970f49c1-tls-certificates podName:b158ac16-fd79-4e1d-8cc3-7bdb970f49c1 nodeName:}" failed. No retries permitted until 2026-07-24 17:22:15.645992361 +0000 UTC m=+298.373148557 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/b158ac16-fd79-4e1d-8cc3-7bdb970f49c1-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-92hzw" (UID: "b158ac16-fd79-4e1d-8cc3-7bdb970f49c1") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 24 17:22:15.648332 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:15.648298    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b158ac16-fd79-4e1d-8cc3-7bdb970f49c1-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-92hzw\" (UID: \"b158ac16-fd79-4e1d-8cc3-7bdb970f49c1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-92hzw"
Jul 24 17:22:15.650631 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:15.650608    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b158ac16-fd79-4e1d-8cc3-7bdb970f49c1-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-92hzw\" (UID: \"b158ac16-fd79-4e1d-8cc3-7bdb970f49c1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-92hzw"
Jul 24 17:22:15.895731 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:15.895698    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-92hzw"
Jul 24 17:22:16.012588 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:16.012556    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-92hzw"]
Jul 24 17:22:16.015449 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:22:16.015419    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb158ac16_fd79_4e1d_8cc3_7bdb970f49c1.slice/crio-8f297eb8fe77c552b2234dca0cc442b407af37627f5ae6fc071a7b982c204f75 WatchSource:0}: Error finding container 8f297eb8fe77c552b2234dca0cc442b407af37627f5ae6fc071a7b982c204f75: Status 404 returned error can't find the container with id 8f297eb8fe77c552b2234dca0cc442b407af37627f5ae6fc071a7b982c204f75
Jul 24 17:22:16.741105 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:16.741074    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bf5b8"
Jul 24 17:22:16.776606 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:16.776569    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-92hzw" event={"ID":"b158ac16-fd79-4e1d-8cc3-7bdb970f49c1","Type":"ContainerStarted","Data":"8f297eb8fe77c552b2234dca0cc442b407af37627f5ae6fc071a7b982c204f75"}
Jul 24 17:22:17.659000 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:17.658966    2562 kubelet.go:1648] "Image garbage collection succeeded"
Jul 24 17:22:17.781282 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:17.781092    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-92hzw" event={"ID":"b158ac16-fd79-4e1d-8cc3-7bdb970f49c1","Type":"ContainerStarted","Data":"5a2d26e1a257d1994617fe0a3922e1094b49f522c9e87e8667f647cd37672803"}
Jul 24 17:22:17.806384 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:17.781300    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-92hzw"
Jul 24 17:22:17.806384 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:17.785771    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-92hzw"
Jul 24 17:22:17.806384 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:17.795954    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-92hzw" podStartSLOduration=2.65754851 podStartE2EDuration="3.795938321s" totalImagesPullingTime="1.138389811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:22:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:22:17.795468656 +0000 UTC m=+300.522624863" watchObservedRunningTime="2026-07-24 17:22:17.795938321 +0000 UTC m=+300.523094537"
Jul 24 17:22:22.010229 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.010191    2562 patch_prober.go:28] interesting pod/image-registry-75594549dc-q5krg container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 24 17:22:22.010599 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.010244    2562 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-75594549dc-q5krg" podUID="615228ee-d6a8-4566-b5e7-82d5a27bf812" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 17:22:22.408761 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.408731    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7"]
Jul 24 17:22:22.412583 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.412564    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7"
Jul 24 17:22:22.416156 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.415992    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 24 17:22:22.416156 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.415998    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 24 17:22:22.416156 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.416030    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 24 17:22:22.416156 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.416047    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 24 17:22:22.416156 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.416058    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 24 17:22:22.416156 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.416055    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 24 17:22:22.416465 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.416169    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-lvsjr\""
Jul 24 17:22:22.426629 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.426608    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7"]
Jul 24 17:22:22.437159 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.437119    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fxdjb"]
Jul 24 17:22:22.440612 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.440594    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.442929 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.442911    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 24 17:22:22.443034 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.442963    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 24 17:22:22.443266 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.443248    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2zddw\""
Jul 24 17:22:22.443266 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.443256    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 24 17:22:22.492941 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.492916    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/721fc7de-0768-4944-8f6b-8f76b09511f9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fxdjb\" (UID: \"721fc7de-0768-4944-8f6b-8f76b09511f9\") " pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.493024 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.492952    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8d189003-36d8-41e7-b7be-96a055c05f86-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-54mj7\" (UID: \"8d189003-36d8-41e7-b7be-96a055c05f86\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7"
Jul 24 17:22:22.493024 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.492970    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8d189003-36d8-41e7-b7be-96a055c05f86-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-54mj7\" (UID: \"8d189003-36d8-41e7-b7be-96a055c05f86\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7"
Jul 24 17:22:22.493024 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.492990    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/721fc7de-0768-4944-8f6b-8f76b09511f9-sys\") pod \"node-exporter-fxdjb\" (UID: \"721fc7de-0768-4944-8f6b-8f76b09511f9\") " pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.493024 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.493007    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d189003-36d8-41e7-b7be-96a055c05f86-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-54mj7\" (UID: \"8d189003-36d8-41e7-b7be-96a055c05f86\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7"
Jul 24 17:22:22.493200 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.493026    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/721fc7de-0768-4944-8f6b-8f76b09511f9-metrics-client-ca\") pod \"node-exporter-fxdjb\" (UID: \"721fc7de-0768-4944-8f6b-8f76b09511f9\") " pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.493200 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.493113    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6pt4\" (UniqueName: \"kubernetes.io/projected/8d189003-36d8-41e7-b7be-96a055c05f86-kube-api-access-k6pt4\") pod \"kube-state-metrics-56d9f944b4-54mj7\" (UID: \"8d189003-36d8-41e7-b7be-96a055c05f86\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7"
Jul 24 17:22:22.493200 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.493134    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/721fc7de-0768-4944-8f6b-8f76b09511f9-node-exporter-wtmp\") pod \"node-exporter-fxdjb\" (UID: \"721fc7de-0768-4944-8f6b-8f76b09511f9\") " pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.493200 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.493179    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/721fc7de-0768-4944-8f6b-8f76b09511f9-node-exporter-textfile\") pod \"node-exporter-fxdjb\" (UID: \"721fc7de-0768-4944-8f6b-8f76b09511f9\") " pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.493334 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.493203    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/721fc7de-0768-4944-8f6b-8f76b09511f9-node-exporter-accelerators-collector-config\") pod \"node-exporter-fxdjb\" (UID: \"721fc7de-0768-4944-8f6b-8f76b09511f9\") " pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.493334 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.493225    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9cst\" (UniqueName: \"kubernetes.io/projected/721fc7de-0768-4944-8f6b-8f76b09511f9-kube-api-access-v9cst\") pod \"node-exporter-fxdjb\" (UID: \"721fc7de-0768-4944-8f6b-8f76b09511f9\") " pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.493334 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.493245    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8d189003-36d8-41e7-b7be-96a055c05f86-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-54mj7\" (UID: \"8d189003-36d8-41e7-b7be-96a055c05f86\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7"
Jul 24 17:22:22.493334 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.493274    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8d189003-36d8-41e7-b7be-96a055c05f86-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-54mj7\" (UID: \"8d189003-36d8-41e7-b7be-96a055c05f86\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7"
Jul 24 17:22:22.493334 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.493290    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/721fc7de-0768-4944-8f6b-8f76b09511f9-node-exporter-tls\") pod \"node-exporter-fxdjb\" (UID: \"721fc7de-0768-4944-8f6b-8f76b09511f9\") " pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.493334 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.493304    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/721fc7de-0768-4944-8f6b-8f76b09511f9-root\") pod \"node-exporter-fxdjb\" (UID: \"721fc7de-0768-4944-8f6b-8f76b09511f9\") " pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.594087 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.594065    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8d189003-36d8-41e7-b7be-96a055c05f86-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-54mj7\" (UID: \"8d189003-36d8-41e7-b7be-96a055c05f86\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7"
Jul 24 17:22:22.594227 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.594093    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8d189003-36d8-41e7-b7be-96a055c05f86-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-54mj7\" (UID: \"8d189003-36d8-41e7-b7be-96a055c05f86\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7"
Jul 24 17:22:22.594227 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.594112    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/721fc7de-0768-4944-8f6b-8f76b09511f9-sys\") pod \"node-exporter-fxdjb\" (UID: \"721fc7de-0768-4944-8f6b-8f76b09511f9\") " pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.594227 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.594131    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d189003-36d8-41e7-b7be-96a055c05f86-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-54mj7\" (UID: \"8d189003-36d8-41e7-b7be-96a055c05f86\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7"
Jul 24 17:22:22.594392 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.594221    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/721fc7de-0768-4944-8f6b-8f76b09511f9-sys\") pod \"node-exporter-fxdjb\" (UID: \"721fc7de-0768-4944-8f6b-8f76b09511f9\") " pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.594392 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.594273    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/721fc7de-0768-4944-8f6b-8f76b09511f9-metrics-client-ca\") pod \"node-exporter-fxdjb\" (UID: \"721fc7de-0768-4944-8f6b-8f76b09511f9\") " pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.594392 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.594341    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6pt4\" (UniqueName: \"kubernetes.io/projected/8d189003-36d8-41e7-b7be-96a055c05f86-kube-api-access-k6pt4\") pod \"kube-state-metrics-56d9f944b4-54mj7\" (UID: \"8d189003-36d8-41e7-b7be-96a055c05f86\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7"
Jul 24 17:22:22.594392 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.594375    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/721fc7de-0768-4944-8f6b-8f76b09511f9-node-exporter-wtmp\") pod \"node-exporter-fxdjb\" (UID: \"721fc7de-0768-4944-8f6b-8f76b09511f9\") " pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.594591 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.594411    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/721fc7de-0768-4944-8f6b-8f76b09511f9-node-exporter-textfile\") pod \"node-exporter-fxdjb\" (UID: \"721fc7de-0768-4944-8f6b-8f76b09511f9\") " pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.594591 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.594441    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/721fc7de-0768-4944-8f6b-8f76b09511f9-node-exporter-accelerators-collector-config\") pod \"node-exporter-fxdjb\" (UID: \"721fc7de-0768-4944-8f6b-8f76b09511f9\") " pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.594591 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.594472    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9cst\" (UniqueName: \"kubernetes.io/projected/721fc7de-0768-4944-8f6b-8f76b09511f9-kube-api-access-v9cst\") pod \"node-exporter-fxdjb\" (UID: \"721fc7de-0768-4944-8f6b-8f76b09511f9\") " pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.594591 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.594509    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8d189003-36d8-41e7-b7be-96a055c05f86-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-54mj7\" (UID: \"8d189003-36d8-41e7-b7be-96a055c05f86\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7"
Jul 24 17:22:22.594591 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.594541    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8d189003-36d8-41e7-b7be-96a055c05f86-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-54mj7\" (UID: \"8d189003-36d8-41e7-b7be-96a055c05f86\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7"
Jul 24 17:22:22.594591 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.594546    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8d189003-36d8-41e7-b7be-96a055c05f86-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-54mj7\" (UID: \"8d189003-36d8-41e7-b7be-96a055c05f86\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7"
Jul 24 17:22:22.594977 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.594606    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/721fc7de-0768-4944-8f6b-8f76b09511f9-node-exporter-tls\") pod \"node-exporter-fxdjb\" (UID: \"721fc7de-0768-4944-8f6b-8f76b09511f9\") " pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.594977 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:22:22.594639    2562 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 24 17:22:22.594977 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.594648    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/721fc7de-0768-4944-8f6b-8f76b09511f9-root\") pod \"node-exporter-fxdjb\" (UID: \"721fc7de-0768-4944-8f6b-8f76b09511f9\") " pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.594977 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.594679    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/721fc7de-0768-4944-8f6b-8f76b09511f9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fxdjb\" (UID: \"721fc7de-0768-4944-8f6b-8f76b09511f9\") " pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.594977 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:22:22.594703    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d189003-36d8-41e7-b7be-96a055c05f86-kube-state-metrics-tls podName:8d189003-36d8-41e7-b7be-96a055c05f86 nodeName:}" failed. No retries permitted until 2026-07-24 17:22:23.094676247 +0000 UTC m=+305.821832455 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/8d189003-36d8-41e7-b7be-96a055c05f86-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-54mj7" (UID: "8d189003-36d8-41e7-b7be-96a055c05f86") : secret "kube-state-metrics-tls" not found
Jul 24 17:22:22.594977 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.594733    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/721fc7de-0768-4944-8f6b-8f76b09511f9-node-exporter-textfile\") pod \"node-exporter-fxdjb\" (UID: \"721fc7de-0768-4944-8f6b-8f76b09511f9\") " pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.594977 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.594854    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/721fc7de-0768-4944-8f6b-8f76b09511f9-root\") pod \"node-exporter-fxdjb\" (UID: \"721fc7de-0768-4944-8f6b-8f76b09511f9\") " pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.594977 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.594967    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d189003-36d8-41e7-b7be-96a055c05f86-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-54mj7\" (UID: \"8d189003-36d8-41e7-b7be-96a055c05f86\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7"
Jul 24 17:22:22.595358 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.595278    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8d189003-36d8-41e7-b7be-96a055c05f86-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-54mj7\" (UID: \"8d189003-36d8-41e7-b7be-96a055c05f86\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7"
Jul 24 17:22:22.595358 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.595299    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/721fc7de-0768-4944-8f6b-8f76b09511f9-node-exporter-wtmp\") pod \"node-exporter-fxdjb\" (UID: \"721fc7de-0768-4944-8f6b-8f76b09511f9\") " pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.595358 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.595350    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/721fc7de-0768-4944-8f6b-8f76b09511f9-metrics-client-ca\") pod \"node-exporter-fxdjb\" (UID: \"721fc7de-0768-4944-8f6b-8f76b09511f9\") " pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.595513 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.595361    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/721fc7de-0768-4944-8f6b-8f76b09511f9-node-exporter-accelerators-collector-config\") pod \"node-exporter-fxdjb\" (UID: \"721fc7de-0768-4944-8f6b-8f76b09511f9\") " pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.596860 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.596834    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8d189003-36d8-41e7-b7be-96a055c05f86-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-54mj7\" (UID: \"8d189003-36d8-41e7-b7be-96a055c05f86\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7"
Jul 24 17:22:22.596942 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.596923    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/721fc7de-0768-4944-8f6b-8f76b09511f9-node-exporter-tls\") pod \"node-exporter-fxdjb\" (UID: \"721fc7de-0768-4944-8f6b-8f76b09511f9\") " pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.597096 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.597079    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/721fc7de-0768-4944-8f6b-8f76b09511f9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fxdjb\" (UID: \"721fc7de-0768-4944-8f6b-8f76b09511f9\") " pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.604757 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.604731    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6pt4\" (UniqueName: \"kubernetes.io/projected/8d189003-36d8-41e7-b7be-96a055c05f86-kube-api-access-k6pt4\") pod \"kube-state-metrics-56d9f944b4-54mj7\" (UID: \"8d189003-36d8-41e7-b7be-96a055c05f86\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7"
Jul 24 17:22:22.604757 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.604742    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9cst\" (UniqueName: \"kubernetes.io/projected/721fc7de-0768-4944-8f6b-8f76b09511f9-kube-api-access-v9cst\") pod \"node-exporter-fxdjb\" (UID: \"721fc7de-0768-4944-8f6b-8f76b09511f9\") " pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.748539 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.748460    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fxdjb"
Jul 24 17:22:22.760580 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:22:22.760557    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod721fc7de_0768_4944_8f6b_8f76b09511f9.slice/crio-24d9d142269358eb0c163686b58d412776d6dc8438887933e25ee6284cf96862 WatchSource:0}: Error finding container 24d9d142269358eb0c163686b58d412776d6dc8438887933e25ee6284cf96862: Status 404 returned error can't find the container with id 24d9d142269358eb0c163686b58d412776d6dc8438887933e25ee6284cf96862
Jul 24 17:22:22.762499 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.762480    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 17:22:22.798189 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.798161    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fxdjb" event={"ID":"721fc7de-0768-4944-8f6b-8f76b09511f9","Type":"ContainerStarted","Data":"24d9d142269358eb0c163686b58d412776d6dc8438887933e25ee6284cf96862"}
Jul 24 17:22:22.818848 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:22.818827    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:22:23.097496 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.097467    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8d189003-36d8-41e7-b7be-96a055c05f86-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-54mj7\" (UID: \"8d189003-36d8-41e7-b7be-96a055c05f86\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7"
Jul 24 17:22:23.099789 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.099766    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8d189003-36d8-41e7-b7be-96a055c05f86-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-54mj7\" (UID: \"8d189003-36d8-41e7-b7be-96a055c05f86\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7"
Jul 24 17:22:23.321215 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.321184    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7"
Jul 24 17:22:23.464366 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.464342    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7"]
Jul 24 17:22:23.475358 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.475334    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 17:22:23.483932 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.483908    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.486327 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.486299    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 24 17:22:23.486446 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.486328    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 24 17:22:23.486519 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.486496    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 24 17:22:23.486565 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.486553    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 24 17:22:23.486786 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.486759    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 24 17:22:23.486922 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.486904    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 24 17:22:23.486986 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.486952    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 24 17:22:23.487199 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.487182    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 24 17:22:23.487280 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.487221    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-p6wcp\""
Jul 24 17:22:23.487598 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.487582    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 24 17:22:23.495422 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.495401    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 17:22:23.500341 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.500320    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b2aceee-d80a-40c0-a6f2-eca4f983d097-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.500459 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.500357    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.500459 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.500392    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq4bt\" (UniqueName: \"kubernetes.io/projected/5b2aceee-d80a-40c0-a6f2-eca4f983d097-kube-api-access-bq4bt\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.500459 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.500430    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.500613 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.500475    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-config-volume\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.500613 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.500515    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5b2aceee-d80a-40c0-a6f2-eca4f983d097-config-out\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.500613 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.500548    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5b2aceee-d80a-40c0-a6f2-eca4f983d097-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.500613 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.500584    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.500762 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.500614    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-web-config\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.500762 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.500634    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5b2aceee-d80a-40c0-a6f2-eca4f983d097-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.500762 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.500666    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5b2aceee-d80a-40c0-a6f2-eca4f983d097-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.500762 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.500698    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.500762 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.500732    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.601929 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.601892    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b2aceee-d80a-40c0-a6f2-eca4f983d097-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.601929 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.601939    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.602206 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.602076    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq4bt\" (UniqueName: \"kubernetes.io/projected/5b2aceee-d80a-40c0-a6f2-eca4f983d097-kube-api-access-bq4bt\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.602206 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:22:23.602099    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5b2aceee-d80a-40c0-a6f2-eca4f983d097-alertmanager-trusted-ca-bundle podName:5b2aceee-d80a-40c0-a6f2-eca4f983d097 nodeName:}" failed. No retries permitted until 2026-07-24 17:22:24.102072195 +0000 UTC m=+306.829228389 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/5b2aceee-d80a-40c0-a6f2-eca4f983d097-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "5b2aceee-d80a-40c0-a6f2-eca4f983d097") : configmap references non-existent config key: ca-bundle.crt
Jul 24 17:22:23.602206 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.602174    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.602376 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.602221    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-config-volume\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.602376 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.602246    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5b2aceee-d80a-40c0-a6f2-eca4f983d097-config-out\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.602376 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.602270    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5b2aceee-d80a-40c0-a6f2-eca4f983d097-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.602376 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.602304    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.602376 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.602348    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-web-config\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.602376 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.602375    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5b2aceee-d80a-40c0-a6f2-eca4f983d097-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.602744 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.602423    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5b2aceee-d80a-40c0-a6f2-eca4f983d097-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.602744 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.602451    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.602744 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.602481    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.604855 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.603022    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5b2aceee-d80a-40c0-a6f2-eca4f983d097-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.604855 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.604170    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5b2aceee-d80a-40c0-a6f2-eca4f983d097-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.606082 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.606038    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.606836 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.606415    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-web-config\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.606836 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.606424    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.606836 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.606429    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5b2aceee-d80a-40c0-a6f2-eca4f983d097-config-out\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.606836 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.606793    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.607107 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.606865    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.608038 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.607973    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5b2aceee-d80a-40c0-a6f2-eca4f983d097-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.608477 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.608449    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-config-volume\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.608870 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.608851    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.610106 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.610087    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq4bt\" (UniqueName: \"kubernetes.io/projected/5b2aceee-d80a-40c0-a6f2-eca4f983d097-kube-api-access-bq4bt\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:23.619097 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:22:23.619072    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d189003_36d8_41e7_b7be_96a055c05f86.slice/crio-82642e071c752c7f7f792ff4d3b4262052e66867adc21fe40f0b515113f0a913 WatchSource:0}: Error finding container 82642e071c752c7f7f792ff4d3b4262052e66867adc21fe40f0b515113f0a913: Status 404 returned error can't find the container with id 82642e071c752c7f7f792ff4d3b4262052e66867adc21fe40f0b515113f0a913
Jul 24 17:22:23.722412 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.722389    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:22:23.802119 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.802090    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7" event={"ID":"8d189003-36d8-41e7-b7be-96a055c05f86","Type":"ContainerStarted","Data":"82642e071c752c7f7f792ff4d3b4262052e66867adc21fe40f0b515113f0a913"}
Jul 24 17:22:23.803621 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.803596    2562 generic.go:356] "Generic (PLEG): container finished" podID="721fc7de-0768-4944-8f6b-8f76b09511f9" containerID="5a35a8762f987cf40bbdabc9a90e9682120828d456ee7e42766e2028f4117e0a" exitCode=0
Jul 24 17:22:23.803725 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:23.803633    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fxdjb" event={"ID":"721fc7de-0768-4944-8f6b-8f76b09511f9","Type":"ContainerDied","Data":"5a35a8762f987cf40bbdabc9a90e9682120828d456ee7e42766e2028f4117e0a"}
Jul 24 17:22:24.105941 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:24.105901    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b2aceee-d80a-40c0-a6f2-eca4f983d097-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:24.106744 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:24.106721    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b2aceee-d80a-40c0-a6f2-eca4f983d097-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:24.396117 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:24.396076    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:22:24.774762 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:24.774737    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 17:22:24.785271 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:22:24.785236    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b2aceee_d80a_40c0_a6f2_eca4f983d097.slice/crio-99808a515e8ce5db605aafca2a6cd3739c1909aaf6a6066314f8f3ad029b8a79 WatchSource:0}: Error finding container 99808a515e8ce5db605aafca2a6cd3739c1909aaf6a6066314f8f3ad029b8a79: Status 404 returned error can't find the container with id 99808a515e8ce5db605aafca2a6cd3739c1909aaf6a6066314f8f3ad029b8a79
Jul 24 17:22:24.811538 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:24.811494    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b2aceee-d80a-40c0-a6f2-eca4f983d097","Type":"ContainerStarted","Data":"99808a515e8ce5db605aafca2a6cd3739c1909aaf6a6066314f8f3ad029b8a79"}
Jul 24 17:22:24.815520 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:24.815239    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7" event={"ID":"8d189003-36d8-41e7-b7be-96a055c05f86","Type":"ContainerStarted","Data":"08d9b73b8b7b01fd0d6f35e8a519097c0af1404f4eabc59b78c7779af886ac26"}
Jul 24 17:22:24.818937 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:24.818852    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fxdjb" event={"ID":"721fc7de-0768-4944-8f6b-8f76b09511f9","Type":"ContainerStarted","Data":"07961a28ac20aea5a9c53878e298d58e777621e72c17cfa38c92002d1b5e15e9"}
Jul 24 17:22:24.818937 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:24.818904    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fxdjb" event={"ID":"721fc7de-0768-4944-8f6b-8f76b09511f9","Type":"ContainerStarted","Data":"12cc6ab75712be96bac2d5204c2829c18fd2c7b8603c3fbed27fabafc4cd28a5"}
Jul 24 17:22:24.845552 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:24.845470    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fxdjb" podStartSLOduration=1.879567549 podStartE2EDuration="2.845447356s" totalImagesPullingTime="903.056062ms" totalInitContainerRuntime="62.823745ms" isStatefulPod=true podCreationTimestamp="2026-07-24 17:22:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:22:24.843863927 +0000 UTC m=+307.571020142" watchObservedRunningTime="2026-07-24 17:22:24.845447356 +0000 UTC m=+307.572603571"
Jul 24 17:22:24.883717 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:24.883262    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-9qb88"]
Jul 24 17:22:24.887785 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:24.887766    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-9qb88"
Jul 24 17:22:24.890562 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:24.890539    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-kbrz4\""
Jul 24 17:22:24.890825 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:24.890683    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 24 17:22:24.891126 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:24.891110    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 24 17:22:24.896840 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:24.896817    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-9qb88"]
Jul 24 17:22:24.912010 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:24.911886    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzq8b\" (UniqueName: \"kubernetes.io/projected/85259239-c35e-4c64-8cf7-e47f80c22ec7-kube-api-access-kzq8b\") pod \"downloads-7c97dc4555-9qb88\" (UID: \"85259239-c35e-4c64-8cf7-e47f80c22ec7\") " pod="openshift-console/downloads-7c97dc4555-9qb88"
Jul 24 17:22:25.014111 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.014066    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzq8b\" (UniqueName: \"kubernetes.io/projected/85259239-c35e-4c64-8cf7-e47f80c22ec7-kube-api-access-kzq8b\") pod \"downloads-7c97dc4555-9qb88\" (UID: \"85259239-c35e-4c64-8cf7-e47f80c22ec7\") " pod="openshift-console/downloads-7c97dc4555-9qb88"
Jul 24 17:22:25.025252 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.025221    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzq8b\" (UniqueName: \"kubernetes.io/projected/85259239-c35e-4c64-8cf7-e47f80c22ec7-kube-api-access-kzq8b\") pod \"downloads-7c97dc4555-9qb88\" (UID: \"85259239-c35e-4c64-8cf7-e47f80c22ec7\") " pod="openshift-console/downloads-7c97dc4555-9qb88"
Jul 24 17:22:25.200324 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.200228    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-9qb88"
Jul 24 17:22:25.326240 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.326121    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-9qb88"]
Jul 24 17:22:25.329305 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:22:25.329275    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85259239_c35e_4c64_8cf7_e47f80c22ec7.slice/crio-fe1c781e7f2894f6c454381c36dafa222041d928ee1cc7646633870d4dd2540e WatchSource:0}: Error finding container fe1c781e7f2894f6c454381c36dafa222041d928ee1cc7646633870d4dd2540e: Status 404 returned error can't find the container with id fe1c781e7f2894f6c454381c36dafa222041d928ee1cc7646633870d4dd2540e
Jul 24 17:22:25.390426 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.390394    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"]
Jul 24 17:22:25.394744 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.394721    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"
Jul 24 17:22:25.397322 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.397301    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 24 17:22:25.397498 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.397330    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-h66h5\""
Jul 24 17:22:25.397578 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.397509    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 24 17:22:25.397638 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.397405    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 24 17:22:25.398102 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.398079    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 24 17:22:25.398212 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.398103    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-7jgkvgk20icm1\""
Jul 24 17:22:25.398315 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.398291    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 24 17:22:25.406468 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.406435    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"]
Jul 24 17:22:25.418512 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.418480    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/667d4e75-26be-4bfd-b20a-213dca25e280-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-84bb58cf8-l5sbx\" (UID: \"667d4e75-26be-4bfd-b20a-213dca25e280\") " pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"
Jul 24 17:22:25.418651 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.418536    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgmw2\" (UniqueName: \"kubernetes.io/projected/667d4e75-26be-4bfd-b20a-213dca25e280-kube-api-access-xgmw2\") pod \"thanos-querier-84bb58cf8-l5sbx\" (UID: \"667d4e75-26be-4bfd-b20a-213dca25e280\") " pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"
Jul 24 17:22:25.418651 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.418560    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/667d4e75-26be-4bfd-b20a-213dca25e280-secret-grpc-tls\") pod \"thanos-querier-84bb58cf8-l5sbx\" (UID: \"667d4e75-26be-4bfd-b20a-213dca25e280\") " pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"
Jul 24 17:22:25.418848 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.418671    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/667d4e75-26be-4bfd-b20a-213dca25e280-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-84bb58cf8-l5sbx\" (UID: \"667d4e75-26be-4bfd-b20a-213dca25e280\") " pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"
Jul 24 17:22:25.418848 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.418761    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/667d4e75-26be-4bfd-b20a-213dca25e280-secret-thanos-querier-tls\") pod \"thanos-querier-84bb58cf8-l5sbx\" (UID: \"667d4e75-26be-4bfd-b20a-213dca25e280\") " pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"
Jul 24 17:22:25.418848 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.418794    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/667d4e75-26be-4bfd-b20a-213dca25e280-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-84bb58cf8-l5sbx\" (UID: \"667d4e75-26be-4bfd-b20a-213dca25e280\") " pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"
Jul 24 17:22:25.418848 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.418824    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/667d4e75-26be-4bfd-b20a-213dca25e280-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-84bb58cf8-l5sbx\" (UID: \"667d4e75-26be-4bfd-b20a-213dca25e280\") " pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"
Jul 24 17:22:25.419053 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.418852    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/667d4e75-26be-4bfd-b20a-213dca25e280-metrics-client-ca\") pod \"thanos-querier-84bb58cf8-l5sbx\" (UID: \"667d4e75-26be-4bfd-b20a-213dca25e280\") " pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"
Jul 24 17:22:25.519360 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.519269    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/667d4e75-26be-4bfd-b20a-213dca25e280-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-84bb58cf8-l5sbx\" (UID: \"667d4e75-26be-4bfd-b20a-213dca25e280\") " pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"
Jul 24 17:22:25.519360 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.519327    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/667d4e75-26be-4bfd-b20a-213dca25e280-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-84bb58cf8-l5sbx\" (UID: \"667d4e75-26be-4bfd-b20a-213dca25e280\") " pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"
Jul 24 17:22:25.519587 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.519360    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/667d4e75-26be-4bfd-b20a-213dca25e280-metrics-client-ca\") pod \"thanos-querier-84bb58cf8-l5sbx\" (UID: \"667d4e75-26be-4bfd-b20a-213dca25e280\") " pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"
Jul 24 17:22:25.519587 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.519402    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/667d4e75-26be-4bfd-b20a-213dca25e280-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-84bb58cf8-l5sbx\" (UID: \"667d4e75-26be-4bfd-b20a-213dca25e280\") " pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"
Jul 24 17:22:25.519587 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.519454    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgmw2\" (UniqueName: \"kubernetes.io/projected/667d4e75-26be-4bfd-b20a-213dca25e280-kube-api-access-xgmw2\") pod \"thanos-querier-84bb58cf8-l5sbx\" (UID: \"667d4e75-26be-4bfd-b20a-213dca25e280\") " pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"
Jul 24 17:22:25.519587 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.519482    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/667d4e75-26be-4bfd-b20a-213dca25e280-secret-grpc-tls\") pod \"thanos-querier-84bb58cf8-l5sbx\" (UID: \"667d4e75-26be-4bfd-b20a-213dca25e280\") " pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"
Jul 24 17:22:25.519587 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.519532    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/667d4e75-26be-4bfd-b20a-213dca25e280-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-84bb58cf8-l5sbx\" (UID: \"667d4e75-26be-4bfd-b20a-213dca25e280\") " pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"
Jul 24 17:22:25.519823 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.519605    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/667d4e75-26be-4bfd-b20a-213dca25e280-secret-thanos-querier-tls\") pod \"thanos-querier-84bb58cf8-l5sbx\" (UID: \"667d4e75-26be-4bfd-b20a-213dca25e280\") " pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"
Jul 24 17:22:25.520800 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.520724    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/667d4e75-26be-4bfd-b20a-213dca25e280-metrics-client-ca\") pod \"thanos-querier-84bb58cf8-l5sbx\" (UID: \"667d4e75-26be-4bfd-b20a-213dca25e280\") " pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"
Jul 24 17:22:25.522364 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.522307    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/667d4e75-26be-4bfd-b20a-213dca25e280-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-84bb58cf8-l5sbx\" (UID: \"667d4e75-26be-4bfd-b20a-213dca25e280\") " pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"
Jul 24 17:22:25.522364 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.522348    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/667d4e75-26be-4bfd-b20a-213dca25e280-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-84bb58cf8-l5sbx\" (UID: \"667d4e75-26be-4bfd-b20a-213dca25e280\") " pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"
Jul 24 17:22:25.522817 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.522791    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/667d4e75-26be-4bfd-b20a-213dca25e280-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-84bb58cf8-l5sbx\" (UID: \"667d4e75-26be-4bfd-b20a-213dca25e280\") " pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"
Jul 24 17:22:25.522912 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.522834    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/667d4e75-26be-4bfd-b20a-213dca25e280-secret-thanos-querier-tls\") pod \"thanos-querier-84bb58cf8-l5sbx\" (UID: \"667d4e75-26be-4bfd-b20a-213dca25e280\") " pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"
Jul 24 17:22:25.522963 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.522909    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/667d4e75-26be-4bfd-b20a-213dca25e280-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-84bb58cf8-l5sbx\" (UID: \"667d4e75-26be-4bfd-b20a-213dca25e280\") " pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"
Jul 24 17:22:25.523118 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.523097    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/667d4e75-26be-4bfd-b20a-213dca25e280-secret-grpc-tls\") pod \"thanos-querier-84bb58cf8-l5sbx\" (UID: \"667d4e75-26be-4bfd-b20a-213dca25e280\") " pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"
Jul 24 17:22:25.530376 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.530356    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgmw2\" (UniqueName: \"kubernetes.io/projected/667d4e75-26be-4bfd-b20a-213dca25e280-kube-api-access-xgmw2\") pod \"thanos-querier-84bb58cf8-l5sbx\" (UID: \"667d4e75-26be-4bfd-b20a-213dca25e280\") " pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"
Jul 24 17:22:25.705683 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.705650    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"
Jul 24 17:22:25.827952 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.827921    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-9qb88" event={"ID":"85259239-c35e-4c64-8cf7-e47f80c22ec7","Type":"ContainerStarted","Data":"fe1c781e7f2894f6c454381c36dafa222041d928ee1cc7646633870d4dd2540e"}
Jul 24 17:22:25.830419 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.830393    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7" event={"ID":"8d189003-36d8-41e7-b7be-96a055c05f86","Type":"ContainerStarted","Data":"9b5a46dd1c61ccfdf013849d3e025e63a115dc45e5d239124675a37deacd2560"}
Jul 24 17:22:25.830566 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.830461    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7" event={"ID":"8d189003-36d8-41e7-b7be-96a055c05f86","Type":"ContainerStarted","Data":"3eef6dbe61095cb9800920ec42cc879c5982a4f5eccd86f2749576055fa7a3f5"}
Jul 24 17:22:25.851645 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.851189    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-54mj7" podStartSLOduration=2.782101014 podStartE2EDuration="3.851166756s" totalImagesPullingTime="1.069065742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:22:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:22:25.849932879 +0000 UTC m=+308.577089094" watchObservedRunningTime="2026-07-24 17:22:25.851166756 +0000 UTC m=+308.578322983"
Jul 24 17:22:25.907606 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:25.907581    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"]
Jul 24 17:22:25.948924 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:22:25.948891    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod667d4e75_26be_4bfd_b20a_213dca25e280.slice/crio-b3df0452001565598c45734ee58caa918dd2767fedaabcf0af6a47c66369d5c1 WatchSource:0}: Error finding container b3df0452001565598c45734ee58caa918dd2767fedaabcf0af6a47c66369d5c1: Status 404 returned error can't find the container with id b3df0452001565598c45734ee58caa918dd2767fedaabcf0af6a47c66369d5c1
Jul 24 17:22:26.837537 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:26.837286    2562 generic.go:356] "Generic (PLEG): container finished" podID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerID="cf490983fc97489347ffae6d3a16187f7e18ef62b24eaa8c66df0e5373183860" exitCode=0
Jul 24 17:22:26.837537 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:26.837376    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b2aceee-d80a-40c0-a6f2-eca4f983d097","Type":"ContainerDied","Data":"cf490983fc97489347ffae6d3a16187f7e18ef62b24eaa8c66df0e5373183860"}
Jul 24 17:22:26.839224 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:26.839104    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx" event={"ID":"667d4e75-26be-4bfd-b20a-213dca25e280","Type":"ContainerStarted","Data":"b3df0452001565598c45734ee58caa918dd2767fedaabcf0af6a47c66369d5c1"}
Jul 24 17:22:28.862623 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:28.862586    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b2aceee-d80a-40c0-a6f2-eca4f983d097","Type":"ContainerStarted","Data":"8a1ea63327a39153656e152a1415884fb53cfec40bb5d1bc22cfba64e17e0d80"}
Jul 24 17:22:28.862975 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:28.862681    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b2aceee-d80a-40c0-a6f2-eca4f983d097","Type":"ContainerStarted","Data":"f79fdd270f66afc5e410539bcf37e1aad9c4c52e43db7db2a19f0840a4add85e"}
Jul 24 17:22:28.862975 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:28.862729    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b2aceee-d80a-40c0-a6f2-eca4f983d097","Type":"ContainerStarted","Data":"2c36088233349ebe040f969a3d791c6128db60e49f3c50c49f4ec986aeb3453a"}
Jul 24 17:22:28.865302 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:28.865273    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx" event={"ID":"667d4e75-26be-4bfd-b20a-213dca25e280","Type":"ContainerStarted","Data":"d2525dce1d9b8f888c2abcf1f0c2c3af92622ce1d48bbbe0d01f42091a47f26d"}
Jul 24 17:22:28.865396 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:28.865356    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx" event={"ID":"667d4e75-26be-4bfd-b20a-213dca25e280","Type":"ContainerStarted","Data":"749602c1ed366aee4c40ea353447c754e0cef1596fb70e084a5f451f97158559"}
Jul 24 17:22:28.865451 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:28.865398    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx" event={"ID":"667d4e75-26be-4bfd-b20a-213dca25e280","Type":"ContainerStarted","Data":"3b575ff7a038d5ea4099e76db7d16a159b3191d759c66ad2240e35074444567f"}
Jul 24 17:22:29.874820 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:29.874781    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b2aceee-d80a-40c0-a6f2-eca4f983d097","Type":"ContainerStarted","Data":"5a6adabb001087b599cee723788f87aef7d914093cbb06704257fa8c2747348d"}
Jul 24 17:22:29.875277 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:29.874875    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b2aceee-d80a-40c0-a6f2-eca4f983d097","Type":"ContainerStarted","Data":"b078126283ad8c45d8eb7c648a465c1cb09801514f3a4f4a5f369f1bc2b4215b"}
Jul 24 17:22:29.875277 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:29.874918    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b2aceee-d80a-40c0-a6f2-eca4f983d097","Type":"ContainerStarted","Data":"64086b07b00cb77bbb10425109d5e76596d158a4fb918098452537643a101f0e"}
Jul 24 17:22:29.878780 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:29.878745    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx" event={"ID":"667d4e75-26be-4bfd-b20a-213dca25e280","Type":"ContainerStarted","Data":"3600afcb5bdca52400915c3864872c475161615f0ba387bd8752fe63cd58569f"}
Jul 24 17:22:29.878909 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:29.878830    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx" event={"ID":"667d4e75-26be-4bfd-b20a-213dca25e280","Type":"ContainerStarted","Data":"2ac3f73271d3ccc81a48281d6f21bc3556584101b893961011896fcc21ddf768"}
Jul 24 17:22:29.906782 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:29.903076    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.404654585 podStartE2EDuration="6.903055068s" totalImagesPullingTime="3.482233966s" totalInitContainerRuntime="16.166517ms" isStatefulPod=false podCreationTimestamp="2026-07-24 17:22:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:22:29.900196977 +0000 UTC m=+312.627353193" watchObservedRunningTime="2026-07-24 17:22:29.903055068 +0000 UTC m=+312.630211287"
Jul 24 17:22:30.887171 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:30.887090    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx" event={"ID":"667d4e75-26be-4bfd-b20a-213dca25e280","Type":"ContainerStarted","Data":"35b4aee268e8df09f87f2344eea0edcb96cb865b731c924a4ab8070482b1e0ee"}
Jul 24 17:22:30.912517 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:30.912452    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx" podStartSLOduration=2.435129307 podStartE2EDuration="5.91242729s" totalImagesPullingTime="3.477297983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:22:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:22:30.910336507 +0000 UTC m=+313.637492720" watchObservedRunningTime="2026-07-24 17:22:30.91242729 +0000 UTC m=+313.639583505"
Jul 24 17:22:31.892772 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:31.892736    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"
Jul 24 17:22:33.248217 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:33.248175    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-75594549dc-q5krg"]
Jul 24 17:22:37.905602 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:37.905569    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-84bb58cf8-l5sbx"
Jul 24 17:22:42.942657 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:42.942560    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-9qb88" event={"ID":"85259239-c35e-4c64-8cf7-e47f80c22ec7","Type":"ContainerStarted","Data":"d0cd82176156daf9e2e067c6c9762d7e1e640cd43c32d56786bb422575ad5f8c"}
Jul 24 17:22:42.943086 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:42.942802    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-9qb88"
Jul 24 17:22:42.944102 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:42.944076    2562 patch_prober.go:28] interesting pod/downloads-7c97dc4555-9qb88 container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.18:8080/\": dial tcp 10.132.0.18:8080: connect: connection refused" start-of-body=
Jul 24 17:22:42.944231 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:42.944120    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-7c97dc4555-9qb88" podUID="85259239-c35e-4c64-8cf7-e47f80c22ec7" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.18:8080/\": dial tcp 10.132.0.18:8080: connect: connection refused"
Jul 24 17:22:42.961885 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:42.961818    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-9qb88" podStartSLOduration=1.695602263 podStartE2EDuration="18.961802656s" totalImagesPullingTime="17.266200393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:22:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:22:42.960134263 +0000 UTC m=+325.687290478" watchObservedRunningTime="2026-07-24 17:22:42.961802656 +0000 UTC m=+325.688958870"
Jul 24 17:22:43.957577 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:43.957541    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-9qb88"
Jul 24 17:22:50.041320 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:50.041290    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5b2aceee-d80a-40c0-a6f2-eca4f983d097/init-config-reloader/0.log"
Jul 24 17:22:50.047612 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:50.047582    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5b2aceee-d80a-40c0-a6f2-eca4f983d097/alertmanager/0.log"
Jul 24 17:22:50.181602 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:50.181561    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5b2aceee-d80a-40c0-a6f2-eca4f983d097/config-reloader/0.log"
Jul 24 17:22:50.382162 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:50.382098    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5b2aceee-d80a-40c0-a6f2-eca4f983d097/kube-rbac-proxy-web/0.log"
Jul 24 17:22:50.582599 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:50.582569    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5b2aceee-d80a-40c0-a6f2-eca4f983d097/kube-rbac-proxy/0.log"
Jul 24 17:22:50.782677 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:50.782584    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5b2aceee-d80a-40c0-a6f2-eca4f983d097/kube-rbac-proxy-metric/0.log"
Jul 24 17:22:50.981420 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:50.981389    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5b2aceee-d80a-40c0-a6f2-eca4f983d097/prom-label-proxy/0.log"
Jul 24 17:22:51.382998 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:51.382962    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-54mj7_8d189003-36d8-41e7-b7be-96a055c05f86/kube-state-metrics/0.log"
Jul 24 17:22:51.585299 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:51.585272    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-54mj7_8d189003-36d8-41e7-b7be-96a055c05f86/kube-rbac-proxy-main/0.log"
Jul 24 17:22:51.782275 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:51.782184    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-54mj7_8d189003-36d8-41e7-b7be-96a055c05f86/kube-rbac-proxy-self/0.log"
Jul 24 17:22:53.585771 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:53.585742    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fxdjb_721fc7de-0768-4944-8f6b-8f76b09511f9/init-textfile/0.log"
Jul 24 17:22:53.782414 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:53.782384    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fxdjb_721fc7de-0768-4944-8f6b-8f76b09511f9/node-exporter/0.log"
Jul 24 17:22:53.981692 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:53.981616    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fxdjb_721fc7de-0768-4944-8f6b-8f76b09511f9/kube-rbac-proxy/0.log"
Jul 24 17:22:56.581454 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:56.581416    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-92hzw_b158ac16-fd79-4e1d-8cc3-7bdb970f49c1/prometheus-operator-admission-webhook/0.log"
Jul 24 17:22:57.383055 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:57.383027    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84bb58cf8-l5sbx_667d4e75-26be-4bfd-b20a-213dca25e280/thanos-query/0.log"
Jul 24 17:22:57.582472 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:57.582445    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84bb58cf8-l5sbx_667d4e75-26be-4bfd-b20a-213dca25e280/kube-rbac-proxy-web/0.log"
Jul 24 17:22:57.782074 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:57.781992    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84bb58cf8-l5sbx_667d4e75-26be-4bfd-b20a-213dca25e280/kube-rbac-proxy/0.log"
Jul 24 17:22:57.983390 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:57.983357    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84bb58cf8-l5sbx_667d4e75-26be-4bfd-b20a-213dca25e280/prom-label-proxy/0.log"
Jul 24 17:22:58.182398 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:58.182349    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84bb58cf8-l5sbx_667d4e75-26be-4bfd-b20a-213dca25e280/kube-rbac-proxy-rules/0.log"
Jul 24 17:22:58.274510 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:58.274465    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-75594549dc-q5krg" podUID="615228ee-d6a8-4566-b5e7-82d5a27bf812" containerName="registry" containerID="cri-o://544ea5da72e9d95f353ea3e95093878da59987256e42460515e52351ace44361" gracePeriod=30
Jul 24 17:22:58.382094 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:58.382059    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84bb58cf8-l5sbx_667d4e75-26be-4bfd-b20a-213dca25e280/kube-rbac-proxy-metrics/0.log"
Jul 24 17:22:58.582730 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:58.582634    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-4cmjc_f3d1e52c-502d-496f-bafd-2fcee624d002/networking-console-plugin/0.log"
Jul 24 17:22:59.011958 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:59.011917    2562 generic.go:356] "Generic (PLEG): container finished" podID="615228ee-d6a8-4566-b5e7-82d5a27bf812" containerID="544ea5da72e9d95f353ea3e95093878da59987256e42460515e52351ace44361" exitCode=0
Jul 24 17:22:59.012071 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:59.011991    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75594549dc-q5krg" event={"ID":"615228ee-d6a8-4566-b5e7-82d5a27bf812","Type":"ContainerDied","Data":"544ea5da72e9d95f353ea3e95093878da59987256e42460515e52351ace44361"}
Jul 24 17:22:59.039130 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:59.039106    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:22:59.123767 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:59.123738    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-certificates\") pod \"615228ee-d6a8-4566-b5e7-82d5a27bf812\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") "
Jul 24 17:22:59.123954 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:59.123781    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/615228ee-d6a8-4566-b5e7-82d5a27bf812-installation-pull-secrets\") pod \"615228ee-d6a8-4566-b5e7-82d5a27bf812\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") "
Jul 24 17:22:59.123954 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:59.123825    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/615228ee-d6a8-4566-b5e7-82d5a27bf812-ca-trust-extracted\") pod \"615228ee-d6a8-4566-b5e7-82d5a27bf812\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") "
Jul 24 17:22:59.123954 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:59.123873    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/615228ee-d6a8-4566-b5e7-82d5a27bf812-image-registry-private-configuration\") pod \"615228ee-d6a8-4566-b5e7-82d5a27bf812\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") "
Jul 24 17:22:59.123954 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:59.123901    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/615228ee-d6a8-4566-b5e7-82d5a27bf812-trusted-ca\") pod \"615228ee-d6a8-4566-b5e7-82d5a27bf812\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") "
Jul 24 17:22:59.123954 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:59.123932    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-bound-sa-token\") pod \"615228ee-d6a8-4566-b5e7-82d5a27bf812\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") "
Jul 24 17:22:59.124247 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:59.123965    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6nk4\" (UniqueName: \"kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-kube-api-access-f6nk4\") pod \"615228ee-d6a8-4566-b5e7-82d5a27bf812\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") "
Jul 24 17:22:59.124247 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:59.124000    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls\") pod \"615228ee-d6a8-4566-b5e7-82d5a27bf812\" (UID: \"615228ee-d6a8-4566-b5e7-82d5a27bf812\") "
Jul 24 17:22:59.124348 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:59.124257    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "615228ee-d6a8-4566-b5e7-82d5a27bf812" (UID: "615228ee-d6a8-4566-b5e7-82d5a27bf812"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:22:59.124451 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:59.124417    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/615228ee-d6a8-4566-b5e7-82d5a27bf812-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "615228ee-d6a8-4566-b5e7-82d5a27bf812" (UID: "615228ee-d6a8-4566-b5e7-82d5a27bf812"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:22:59.126543 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:59.126516    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/615228ee-d6a8-4566-b5e7-82d5a27bf812-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "615228ee-d6a8-4566-b5e7-82d5a27bf812" (UID: "615228ee-d6a8-4566-b5e7-82d5a27bf812"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:22:59.126645 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:59.126567    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "615228ee-d6a8-4566-b5e7-82d5a27bf812" (UID: "615228ee-d6a8-4566-b5e7-82d5a27bf812"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:22:59.126645 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:59.126562    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "615228ee-d6a8-4566-b5e7-82d5a27bf812" (UID: "615228ee-d6a8-4566-b5e7-82d5a27bf812"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:22:59.126847 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:59.126814    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/615228ee-d6a8-4566-b5e7-82d5a27bf812-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "615228ee-d6a8-4566-b5e7-82d5a27bf812" (UID: "615228ee-d6a8-4566-b5e7-82d5a27bf812"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:22:59.126929 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:59.126855    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-kube-api-access-f6nk4" (OuterVolumeSpecName: "kube-api-access-f6nk4") pod "615228ee-d6a8-4566-b5e7-82d5a27bf812" (UID: "615228ee-d6a8-4566-b5e7-82d5a27bf812"). InnerVolumeSpecName "kube-api-access-f6nk4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:22:59.132736 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:59.132714    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/615228ee-d6a8-4566-b5e7-82d5a27bf812-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "615228ee-d6a8-4566-b5e7-82d5a27bf812" (UID: "615228ee-d6a8-4566-b5e7-82d5a27bf812"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:22:59.224844 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:59.224782    2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/615228ee-d6a8-4566-b5e7-82d5a27bf812-installation-pull-secrets\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:22:59.224844 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:59.224807    2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/615228ee-d6a8-4566-b5e7-82d5a27bf812-ca-trust-extracted\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:22:59.224844 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:59.224817    2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/615228ee-d6a8-4566-b5e7-82d5a27bf812-image-registry-private-configuration\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:22:59.224844 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:59.224826    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/615228ee-d6a8-4566-b5e7-82d5a27bf812-trusted-ca\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:22:59.224844 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:59.224834    2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-bound-sa-token\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:22:59.224844 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:59.224843    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6nk4\" (UniqueName: \"kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-kube-api-access-f6nk4\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:22:59.225093 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:59.224851    2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-tls\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:22:59.225093 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:59.224860    2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/615228ee-d6a8-4566-b5e7-82d5a27bf812-registry-certificates\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:22:59.584894 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:22:59.584862    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-9qb88_85259239-c35e-4c64-8cf7-e47f80c22ec7/download-server/0.log"
Jul 24 17:23:00.017010 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:00.016930    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75594549dc-q5krg"
Jul 24 17:23:00.017010 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:00.016972    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75594549dc-q5krg" event={"ID":"615228ee-d6a8-4566-b5e7-82d5a27bf812","Type":"ContainerDied","Data":"883d52b090d2048341a12d449f53861317ad89c9fae1dca5995ba2dc920b18ac"}
Jul 24 17:23:00.017279 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:00.017031    2562 scope.go:117] "RemoveContainer" containerID="544ea5da72e9d95f353ea3e95093878da59987256e42460515e52351ace44361"
Jul 24 17:23:00.034118 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:00.034092    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-75594549dc-q5krg"]
Jul 24 17:23:00.038056 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:00.038030    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-75594549dc-q5krg"]
Jul 24 17:23:01.823843 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:01.823811    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="615228ee-d6a8-4566-b5e7-82d5a27bf812" path="/var/lib/kubelet/pods/615228ee-d6a8-4566-b5e7-82d5a27bf812/volumes"
Jul 24 17:23:31.667352 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:31.667311    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs\") pod \"network-metrics-daemon-pjjdg\" (UID: \"b87310bf-1f99-43b7-9731-29966b07be99\") " pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:23:31.669584 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:31.669564    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b87310bf-1f99-43b7-9731-29966b07be99-metrics-certs\") pod \"network-metrics-daemon-pjjdg\" (UID: \"b87310bf-1f99-43b7-9731-29966b07be99\") " pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:23:31.822072 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:31.822040    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ntfrz\""
Jul 24 17:23:31.830082 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:31.830063    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjjdg"
Jul 24 17:23:31.950654 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:31.950628    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pjjdg"]
Jul 24 17:23:31.952918 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:23:31.952883    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb87310bf_1f99_43b7_9731_29966b07be99.slice/crio-87c815d5d51688d173c070c8f56573d42aa28bfa9daa2caea9765112d9cfa7b5 WatchSource:0}: Error finding container 87c815d5d51688d173c070c8f56573d42aa28bfa9daa2caea9765112d9cfa7b5: Status 404 returned error can't find the container with id 87c815d5d51688d173c070c8f56573d42aa28bfa9daa2caea9765112d9cfa7b5
Jul 24 17:23:32.141210 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:32.141180    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pjjdg" event={"ID":"b87310bf-1f99-43b7-9731-29966b07be99","Type":"ContainerStarted","Data":"87c815d5d51688d173c070c8f56573d42aa28bfa9daa2caea9765112d9cfa7b5"}
Jul 24 17:23:33.146372 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:33.146333    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pjjdg" event={"ID":"b87310bf-1f99-43b7-9731-29966b07be99","Type":"ContainerStarted","Data":"7e10c05017afb49c6363db26420308cf1639dfd10ee4cc4e543a9bdb47e14ff0"}
Jul 24 17:23:34.151909 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:34.151871    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pjjdg" event={"ID":"b87310bf-1f99-43b7-9731-29966b07be99","Type":"ContainerStarted","Data":"05c22d48cdaf442e49a2d333fc7f7c79f32dd8e58f7a82f3b92badaa531ff68b"}
Jul 24 17:23:34.167862 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:34.167805    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pjjdg" podStartSLOduration=376.119405211 podStartE2EDuration="6m17.167782077s" totalImagesPullingTime="1.048376866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:17:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:23:34.167479856 +0000 UTC m=+376.894636073" watchObservedRunningTime="2026-07-24 17:23:34.167782077 +0000 UTC m=+376.894938292"
Jul 24 17:23:42.984747 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:42.984712    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 17:23:43.183513 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:43.183453    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerName="alertmanager" containerID="cri-o://2c36088233349ebe040f969a3d791c6128db60e49f3c50c49f4ec986aeb3453a" gracePeriod=120
Jul 24 17:23:43.183730 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:43.183517    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerName="kube-rbac-proxy-metric" containerID="cri-o://b078126283ad8c45d8eb7c648a465c1cb09801514f3a4f4a5f369f1bc2b4215b" gracePeriod=120
Jul 24 17:23:43.183730 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:43.183545    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerName="kube-rbac-proxy-web" containerID="cri-o://8a1ea63327a39153656e152a1415884fb53cfec40bb5d1bc22cfba64e17e0d80" gracePeriod=120
Jul 24 17:23:43.183730 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:43.183566    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerName="prom-label-proxy" containerID="cri-o://5a6adabb001087b599cee723788f87aef7d914093cbb06704257fa8c2747348d" gracePeriod=120
Jul 24 17:23:43.183730 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:43.183597    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerName="config-reloader" containerID="cri-o://f79fdd270f66afc5e410539bcf37e1aad9c4c52e43db7db2a19f0840a4add85e" gracePeriod=120
Jul 24 17:23:43.183730 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:43.183582    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerName="kube-rbac-proxy" containerID="cri-o://64086b07b00cb77bbb10425109d5e76596d158a4fb918098452537643a101f0e" gracePeriod=120
Jul 24 17:23:44.190304 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.190273    2562 generic.go:356] "Generic (PLEG): container finished" podID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerID="5a6adabb001087b599cee723788f87aef7d914093cbb06704257fa8c2747348d" exitCode=0
Jul 24 17:23:44.190304 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.190299    2562 generic.go:356] "Generic (PLEG): container finished" podID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerID="64086b07b00cb77bbb10425109d5e76596d158a4fb918098452537643a101f0e" exitCode=0
Jul 24 17:23:44.190304 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.190306    2562 generic.go:356] "Generic (PLEG): container finished" podID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerID="f79fdd270f66afc5e410539bcf37e1aad9c4c52e43db7db2a19f0840a4add85e" exitCode=0
Jul 24 17:23:44.190304 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.190312    2562 generic.go:356] "Generic (PLEG): container finished" podID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerID="2c36088233349ebe040f969a3d791c6128db60e49f3c50c49f4ec986aeb3453a" exitCode=0
Jul 24 17:23:44.190747 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.190343    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b2aceee-d80a-40c0-a6f2-eca4f983d097","Type":"ContainerDied","Data":"5a6adabb001087b599cee723788f87aef7d914093cbb06704257fa8c2747348d"}
Jul 24 17:23:44.190747 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.190408    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b2aceee-d80a-40c0-a6f2-eca4f983d097","Type":"ContainerDied","Data":"64086b07b00cb77bbb10425109d5e76596d158a4fb918098452537643a101f0e"}
Jul 24 17:23:44.190747 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.190434    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b2aceee-d80a-40c0-a6f2-eca4f983d097","Type":"ContainerDied","Data":"f79fdd270f66afc5e410539bcf37e1aad9c4c52e43db7db2a19f0840a4add85e"}
Jul 24 17:23:44.190747 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.190461    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b2aceee-d80a-40c0-a6f2-eca4f983d097","Type":"ContainerDied","Data":"2c36088233349ebe040f969a3d791c6128db60e49f3c50c49f4ec986aeb3453a"}
Jul 24 17:23:44.426320 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.426299    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:44.458745 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.458161    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-secret-alertmanager-kube-rbac-proxy-metric\") pod \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") "
Jul 24 17:23:44.458745 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.458212    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-secret-alertmanager-main-tls\") pod \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") "
Jul 24 17:23:44.458745 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.458250    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b2aceee-d80a-40c0-a6f2-eca4f983d097-alertmanager-trusted-ca-bundle\") pod \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") "
Jul 24 17:23:44.458745 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.458276    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-secret-alertmanager-kube-rbac-proxy\") pod \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") "
Jul 24 17:23:44.458745 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.458305    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-cluster-tls-config\") pod \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") "
Jul 24 17:23:44.458745 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.458332    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-secret-alertmanager-kube-rbac-proxy-web\") pod \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") "
Jul 24 17:23:44.458745 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.458366    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5b2aceee-d80a-40c0-a6f2-eca4f983d097-config-out\") pod \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") "
Jul 24 17:23:44.458745 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.458398    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5b2aceee-d80a-40c0-a6f2-eca4f983d097-alertmanager-main-db\") pod \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") "
Jul 24 17:23:44.458745 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.458423    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-web-config\") pod \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") "
Jul 24 17:23:44.458745 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.458466    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5b2aceee-d80a-40c0-a6f2-eca4f983d097-metrics-client-ca\") pod \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") "
Jul 24 17:23:44.458745 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.458498    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-config-volume\") pod \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") "
Jul 24 17:23:44.458745 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.458530    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bq4bt\" (UniqueName: \"kubernetes.io/projected/5b2aceee-d80a-40c0-a6f2-eca4f983d097-kube-api-access-bq4bt\") pod \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") "
Jul 24 17:23:44.458745 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.458560    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5b2aceee-d80a-40c0-a6f2-eca4f983d097-tls-assets\") pod \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\" (UID: \"5b2aceee-d80a-40c0-a6f2-eca4f983d097\") "
Jul 24 17:23:44.459540 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.459350    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b2aceee-d80a-40c0-a6f2-eca4f983d097-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "5b2aceee-d80a-40c0-a6f2-eca4f983d097" (UID: "5b2aceee-d80a-40c0-a6f2-eca4f983d097"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:23:44.460222 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.459682    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b2aceee-d80a-40c0-a6f2-eca4f983d097-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "5b2aceee-d80a-40c0-a6f2-eca4f983d097" (UID: "5b2aceee-d80a-40c0-a6f2-eca4f983d097"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:23:44.461538 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.461508    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b2aceee-d80a-40c0-a6f2-eca4f983d097-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "5b2aceee-d80a-40c0-a6f2-eca4f983d097" (UID: "5b2aceee-d80a-40c0-a6f2-eca4f983d097"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:23:44.463046 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.463016    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "5b2aceee-d80a-40c0-a6f2-eca4f983d097" (UID: "5b2aceee-d80a-40c0-a6f2-eca4f983d097"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:23:44.465007 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.464938    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "5b2aceee-d80a-40c0-a6f2-eca4f983d097" (UID: "5b2aceee-d80a-40c0-a6f2-eca4f983d097"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:23:44.465007 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.464960    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-config-volume" (OuterVolumeSpecName: "config-volume") pod "5b2aceee-d80a-40c0-a6f2-eca4f983d097" (UID: "5b2aceee-d80a-40c0-a6f2-eca4f983d097"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:23:44.465007 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.464982    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "5b2aceee-d80a-40c0-a6f2-eca4f983d097" (UID: "5b2aceee-d80a-40c0-a6f2-eca4f983d097"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:23:44.465240 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.465100    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b2aceee-d80a-40c0-a6f2-eca4f983d097-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5b2aceee-d80a-40c0-a6f2-eca4f983d097" (UID: "5b2aceee-d80a-40c0-a6f2-eca4f983d097"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:23:44.465390 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.465363    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b2aceee-d80a-40c0-a6f2-eca4f983d097-config-out" (OuterVolumeSpecName: "config-out") pod "5b2aceee-d80a-40c0-a6f2-eca4f983d097" (UID: "5b2aceee-d80a-40c0-a6f2-eca4f983d097"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:23:44.465679 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.465647    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "5b2aceee-d80a-40c0-a6f2-eca4f983d097" (UID: "5b2aceee-d80a-40c0-a6f2-eca4f983d097"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:23:44.467323 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.467291    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b2aceee-d80a-40c0-a6f2-eca4f983d097-kube-api-access-bq4bt" (OuterVolumeSpecName: "kube-api-access-bq4bt") pod "5b2aceee-d80a-40c0-a6f2-eca4f983d097" (UID: "5b2aceee-d80a-40c0-a6f2-eca4f983d097"). InnerVolumeSpecName "kube-api-access-bq4bt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:23:44.470336 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.470131    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "5b2aceee-d80a-40c0-a6f2-eca4f983d097" (UID: "5b2aceee-d80a-40c0-a6f2-eca4f983d097"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:23:44.476278 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.476257    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-web-config" (OuterVolumeSpecName: "web-config") pod "5b2aceee-d80a-40c0-a6f2-eca4f983d097" (UID: "5b2aceee-d80a-40c0-a6f2-eca4f983d097"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:23:44.559565 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.559524    2562 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-config-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:23:44.559565 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.559557    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bq4bt\" (UniqueName: \"kubernetes.io/projected/5b2aceee-d80a-40c0-a6f2-eca4f983d097-kube-api-access-bq4bt\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:23:44.559565 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.559568    2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5b2aceee-d80a-40c0-a6f2-eca4f983d097-tls-assets\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:23:44.559801 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.559578    2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:23:44.559801 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.559587    2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-secret-alertmanager-main-tls\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:23:44.559801 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.559596    2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b2aceee-d80a-40c0-a6f2-eca4f983d097-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:23:44.559801 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.559605    2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:23:44.559801 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.559615    2562 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-cluster-tls-config\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:23:44.559801 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.559623    2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:23:44.559801 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.559631    2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5b2aceee-d80a-40c0-a6f2-eca4f983d097-config-out\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:23:44.559801 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.559640    2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5b2aceee-d80a-40c0-a6f2-eca4f983d097-alertmanager-main-db\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:23:44.559801 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.559649    2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5b2aceee-d80a-40c0-a6f2-eca4f983d097-web-config\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:23:44.559801 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:44.559658    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5b2aceee-d80a-40c0-a6f2-eca4f983d097-metrics-client-ca\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:23:45.197202 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.197165    2562 generic.go:356] "Generic (PLEG): container finished" podID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerID="b078126283ad8c45d8eb7c648a465c1cb09801514f3a4f4a5f369f1bc2b4215b" exitCode=0
Jul 24 17:23:45.197202 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.197193    2562 generic.go:356] "Generic (PLEG): container finished" podID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerID="8a1ea63327a39153656e152a1415884fb53cfec40bb5d1bc22cfba64e17e0d80" exitCode=0
Jul 24 17:23:45.197624 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.197230    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b2aceee-d80a-40c0-a6f2-eca4f983d097","Type":"ContainerDied","Data":"b078126283ad8c45d8eb7c648a465c1cb09801514f3a4f4a5f369f1bc2b4215b"}
Jul 24 17:23:45.197624 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.197253    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.197624 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.197292    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b2aceee-d80a-40c0-a6f2-eca4f983d097","Type":"ContainerDied","Data":"8a1ea63327a39153656e152a1415884fb53cfec40bb5d1bc22cfba64e17e0d80"}
Jul 24 17:23:45.197624 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.197322    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b2aceee-d80a-40c0-a6f2-eca4f983d097","Type":"ContainerDied","Data":"99808a515e8ce5db605aafca2a6cd3739c1909aaf6a6066314f8f3ad029b8a79"}
Jul 24 17:23:45.197624 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.197354    2562 scope.go:117] "RemoveContainer" containerID="5a6adabb001087b599cee723788f87aef7d914093cbb06704257fa8c2747348d"
Jul 24 17:23:45.205289 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.205183    2562 scope.go:117] "RemoveContainer" containerID="b078126283ad8c45d8eb7c648a465c1cb09801514f3a4f4a5f369f1bc2b4215b"
Jul 24 17:23:45.213346 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.213325    2562 scope.go:117] "RemoveContainer" containerID="64086b07b00cb77bbb10425109d5e76596d158a4fb918098452537643a101f0e"
Jul 24 17:23:45.220505 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.220486    2562 scope.go:117] "RemoveContainer" containerID="8a1ea63327a39153656e152a1415884fb53cfec40bb5d1bc22cfba64e17e0d80"
Jul 24 17:23:45.225780 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.225758    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 17:23:45.227872 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.227851    2562 scope.go:117] "RemoveContainer" containerID="f79fdd270f66afc5e410539bcf37e1aad9c4c52e43db7db2a19f0840a4add85e"
Jul 24 17:23:45.233224 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.233201    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 17:23:45.235296 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.235280    2562 scope.go:117] "RemoveContainer" containerID="2c36088233349ebe040f969a3d791c6128db60e49f3c50c49f4ec986aeb3453a"
Jul 24 17:23:45.241977 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.241962    2562 scope.go:117] "RemoveContainer" containerID="cf490983fc97489347ffae6d3a16187f7e18ef62b24eaa8c66df0e5373183860"
Jul 24 17:23:45.248608 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.248590    2562 scope.go:117] "RemoveContainer" containerID="5a6adabb001087b599cee723788f87aef7d914093cbb06704257fa8c2747348d"
Jul 24 17:23:45.248841 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:23:45.248821    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a6adabb001087b599cee723788f87aef7d914093cbb06704257fa8c2747348d\": container with ID starting with 5a6adabb001087b599cee723788f87aef7d914093cbb06704257fa8c2747348d not found: ID does not exist" containerID="5a6adabb001087b599cee723788f87aef7d914093cbb06704257fa8c2747348d"
Jul 24 17:23:45.248890 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.248850    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a6adabb001087b599cee723788f87aef7d914093cbb06704257fa8c2747348d"} err="failed to get container status \"5a6adabb001087b599cee723788f87aef7d914093cbb06704257fa8c2747348d\": rpc error: code = NotFound desc = could not find container \"5a6adabb001087b599cee723788f87aef7d914093cbb06704257fa8c2747348d\": container with ID starting with 5a6adabb001087b599cee723788f87aef7d914093cbb06704257fa8c2747348d not found: ID does not exist"
Jul 24 17:23:45.248890 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.248881    2562 scope.go:117] "RemoveContainer" containerID="b078126283ad8c45d8eb7c648a465c1cb09801514f3a4f4a5f369f1bc2b4215b"
Jul 24 17:23:45.249086 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:23:45.249066    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b078126283ad8c45d8eb7c648a465c1cb09801514f3a4f4a5f369f1bc2b4215b\": container with ID starting with b078126283ad8c45d8eb7c648a465c1cb09801514f3a4f4a5f369f1bc2b4215b not found: ID does not exist" containerID="b078126283ad8c45d8eb7c648a465c1cb09801514f3a4f4a5f369f1bc2b4215b"
Jul 24 17:23:45.249209 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.249089    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b078126283ad8c45d8eb7c648a465c1cb09801514f3a4f4a5f369f1bc2b4215b"} err="failed to get container status \"b078126283ad8c45d8eb7c648a465c1cb09801514f3a4f4a5f369f1bc2b4215b\": rpc error: code = NotFound desc = could not find container \"b078126283ad8c45d8eb7c648a465c1cb09801514f3a4f4a5f369f1bc2b4215b\": container with ID starting with b078126283ad8c45d8eb7c648a465c1cb09801514f3a4f4a5f369f1bc2b4215b not found: ID does not exist"
Jul 24 17:23:45.249209 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.249110    2562 scope.go:117] "RemoveContainer" containerID="64086b07b00cb77bbb10425109d5e76596d158a4fb918098452537643a101f0e"
Jul 24 17:23:45.249355 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:23:45.249336    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64086b07b00cb77bbb10425109d5e76596d158a4fb918098452537643a101f0e\": container with ID starting with 64086b07b00cb77bbb10425109d5e76596d158a4fb918098452537643a101f0e not found: ID does not exist" containerID="64086b07b00cb77bbb10425109d5e76596d158a4fb918098452537643a101f0e"
Jul 24 17:23:45.249403 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.249362    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64086b07b00cb77bbb10425109d5e76596d158a4fb918098452537643a101f0e"} err="failed to get container status \"64086b07b00cb77bbb10425109d5e76596d158a4fb918098452537643a101f0e\": rpc error: code = NotFound desc = could not find container \"64086b07b00cb77bbb10425109d5e76596d158a4fb918098452537643a101f0e\": container with ID starting with 64086b07b00cb77bbb10425109d5e76596d158a4fb918098452537643a101f0e not found: ID does not exist"
Jul 24 17:23:45.249403 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.249378    2562 scope.go:117] "RemoveContainer" containerID="8a1ea63327a39153656e152a1415884fb53cfec40bb5d1bc22cfba64e17e0d80"
Jul 24 17:23:45.249617 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:23:45.249603    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a1ea63327a39153656e152a1415884fb53cfec40bb5d1bc22cfba64e17e0d80\": container with ID starting with 8a1ea63327a39153656e152a1415884fb53cfec40bb5d1bc22cfba64e17e0d80 not found: ID does not exist" containerID="8a1ea63327a39153656e152a1415884fb53cfec40bb5d1bc22cfba64e17e0d80"
Jul 24 17:23:45.249655 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.249622    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a1ea63327a39153656e152a1415884fb53cfec40bb5d1bc22cfba64e17e0d80"} err="failed to get container status \"8a1ea63327a39153656e152a1415884fb53cfec40bb5d1bc22cfba64e17e0d80\": rpc error: code = NotFound desc = could not find container \"8a1ea63327a39153656e152a1415884fb53cfec40bb5d1bc22cfba64e17e0d80\": container with ID starting with 8a1ea63327a39153656e152a1415884fb53cfec40bb5d1bc22cfba64e17e0d80 not found: ID does not exist"
Jul 24 17:23:45.249655 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.249634    2562 scope.go:117] "RemoveContainer" containerID="f79fdd270f66afc5e410539bcf37e1aad9c4c52e43db7db2a19f0840a4add85e"
Jul 24 17:23:45.249898 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:23:45.249882    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f79fdd270f66afc5e410539bcf37e1aad9c4c52e43db7db2a19f0840a4add85e\": container with ID starting with f79fdd270f66afc5e410539bcf37e1aad9c4c52e43db7db2a19f0840a4add85e not found: ID does not exist" containerID="f79fdd270f66afc5e410539bcf37e1aad9c4c52e43db7db2a19f0840a4add85e"
Jul 24 17:23:45.249939 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.249901    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f79fdd270f66afc5e410539bcf37e1aad9c4c52e43db7db2a19f0840a4add85e"} err="failed to get container status \"f79fdd270f66afc5e410539bcf37e1aad9c4c52e43db7db2a19f0840a4add85e\": rpc error: code = NotFound desc = could not find container \"f79fdd270f66afc5e410539bcf37e1aad9c4c52e43db7db2a19f0840a4add85e\": container with ID starting with f79fdd270f66afc5e410539bcf37e1aad9c4c52e43db7db2a19f0840a4add85e not found: ID does not exist"
Jul 24 17:23:45.249939 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.249915    2562 scope.go:117] "RemoveContainer" containerID="2c36088233349ebe040f969a3d791c6128db60e49f3c50c49f4ec986aeb3453a"
Jul 24 17:23:45.250108 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:23:45.250093    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c36088233349ebe040f969a3d791c6128db60e49f3c50c49f4ec986aeb3453a\": container with ID starting with 2c36088233349ebe040f969a3d791c6128db60e49f3c50c49f4ec986aeb3453a not found: ID does not exist" containerID="2c36088233349ebe040f969a3d791c6128db60e49f3c50c49f4ec986aeb3453a"
Jul 24 17:23:45.250170 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.250112    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c36088233349ebe040f969a3d791c6128db60e49f3c50c49f4ec986aeb3453a"} err="failed to get container status \"2c36088233349ebe040f969a3d791c6128db60e49f3c50c49f4ec986aeb3453a\": rpc error: code = NotFound desc = could not find container \"2c36088233349ebe040f969a3d791c6128db60e49f3c50c49f4ec986aeb3453a\": container with ID starting with 2c36088233349ebe040f969a3d791c6128db60e49f3c50c49f4ec986aeb3453a not found: ID does not exist"
Jul 24 17:23:45.250170 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.250122    2562 scope.go:117] "RemoveContainer" containerID="cf490983fc97489347ffae6d3a16187f7e18ef62b24eaa8c66df0e5373183860"
Jul 24 17:23:45.250359 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:23:45.250341    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf490983fc97489347ffae6d3a16187f7e18ef62b24eaa8c66df0e5373183860\": container with ID starting with cf490983fc97489347ffae6d3a16187f7e18ef62b24eaa8c66df0e5373183860 not found: ID does not exist" containerID="cf490983fc97489347ffae6d3a16187f7e18ef62b24eaa8c66df0e5373183860"
Jul 24 17:23:45.250422 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.250366    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf490983fc97489347ffae6d3a16187f7e18ef62b24eaa8c66df0e5373183860"} err="failed to get container status \"cf490983fc97489347ffae6d3a16187f7e18ef62b24eaa8c66df0e5373183860\": rpc error: code = NotFound desc = could not find container \"cf490983fc97489347ffae6d3a16187f7e18ef62b24eaa8c66df0e5373183860\": container with ID starting with cf490983fc97489347ffae6d3a16187f7e18ef62b24eaa8c66df0e5373183860 not found: ID does not exist"
Jul 24 17:23:45.250422 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.250385    2562 scope.go:117] "RemoveContainer" containerID="5a6adabb001087b599cee723788f87aef7d914093cbb06704257fa8c2747348d"
Jul 24 17:23:45.250592 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.250575    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a6adabb001087b599cee723788f87aef7d914093cbb06704257fa8c2747348d"} err="failed to get container status \"5a6adabb001087b599cee723788f87aef7d914093cbb06704257fa8c2747348d\": rpc error: code = NotFound desc = could not find container \"5a6adabb001087b599cee723788f87aef7d914093cbb06704257fa8c2747348d\": container with ID starting with 5a6adabb001087b599cee723788f87aef7d914093cbb06704257fa8c2747348d not found: ID does not exist"
Jul 24 17:23:45.250639 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.250593    2562 scope.go:117] "RemoveContainer" containerID="b078126283ad8c45d8eb7c648a465c1cb09801514f3a4f4a5f369f1bc2b4215b"
Jul 24 17:23:45.250839 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.250816    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b078126283ad8c45d8eb7c648a465c1cb09801514f3a4f4a5f369f1bc2b4215b"} err="failed to get container status \"b078126283ad8c45d8eb7c648a465c1cb09801514f3a4f4a5f369f1bc2b4215b\": rpc error: code = NotFound desc = could not find container \"b078126283ad8c45d8eb7c648a465c1cb09801514f3a4f4a5f369f1bc2b4215b\": container with ID starting with b078126283ad8c45d8eb7c648a465c1cb09801514f3a4f4a5f369f1bc2b4215b not found: ID does not exist"
Jul 24 17:23:45.250934 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.250840    2562 scope.go:117] "RemoveContainer" containerID="64086b07b00cb77bbb10425109d5e76596d158a4fb918098452537643a101f0e"
Jul 24 17:23:45.251053 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.251032    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64086b07b00cb77bbb10425109d5e76596d158a4fb918098452537643a101f0e"} err="failed to get container status \"64086b07b00cb77bbb10425109d5e76596d158a4fb918098452537643a101f0e\": rpc error: code = NotFound desc = could not find container \"64086b07b00cb77bbb10425109d5e76596d158a4fb918098452537643a101f0e\": container with ID starting with 64086b07b00cb77bbb10425109d5e76596d158a4fb918098452537643a101f0e not found: ID does not exist"
Jul 24 17:23:45.251106 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.251055    2562 scope.go:117] "RemoveContainer" containerID="8a1ea63327a39153656e152a1415884fb53cfec40bb5d1bc22cfba64e17e0d80"
Jul 24 17:23:45.251416 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.251381    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a1ea63327a39153656e152a1415884fb53cfec40bb5d1bc22cfba64e17e0d80"} err="failed to get container status \"8a1ea63327a39153656e152a1415884fb53cfec40bb5d1bc22cfba64e17e0d80\": rpc error: code = NotFound desc = could not find container \"8a1ea63327a39153656e152a1415884fb53cfec40bb5d1bc22cfba64e17e0d80\": container with ID starting with 8a1ea63327a39153656e152a1415884fb53cfec40bb5d1bc22cfba64e17e0d80 not found: ID does not exist"
Jul 24 17:23:45.251416 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.251400    2562 scope.go:117] "RemoveContainer" containerID="f79fdd270f66afc5e410539bcf37e1aad9c4c52e43db7db2a19f0840a4add85e"
Jul 24 17:23:45.251668 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.251642    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f79fdd270f66afc5e410539bcf37e1aad9c4c52e43db7db2a19f0840a4add85e"} err="failed to get container status \"f79fdd270f66afc5e410539bcf37e1aad9c4c52e43db7db2a19f0840a4add85e\": rpc error: code = NotFound desc = could not find container \"f79fdd270f66afc5e410539bcf37e1aad9c4c52e43db7db2a19f0840a4add85e\": container with ID starting with f79fdd270f66afc5e410539bcf37e1aad9c4c52e43db7db2a19f0840a4add85e not found: ID does not exist"
Jul 24 17:23:45.251668 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.251665    2562 scope.go:117] "RemoveContainer" containerID="2c36088233349ebe040f969a3d791c6128db60e49f3c50c49f4ec986aeb3453a"
Jul 24 17:23:45.251934 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.251917    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c36088233349ebe040f969a3d791c6128db60e49f3c50c49f4ec986aeb3453a"} err="failed to get container status \"2c36088233349ebe040f969a3d791c6128db60e49f3c50c49f4ec986aeb3453a\": rpc error: code = NotFound desc = could not find container \"2c36088233349ebe040f969a3d791c6128db60e49f3c50c49f4ec986aeb3453a\": container with ID starting with 2c36088233349ebe040f969a3d791c6128db60e49f3c50c49f4ec986aeb3453a not found: ID does not exist"
Jul 24 17:23:45.251986 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.251934    2562 scope.go:117] "RemoveContainer" containerID="cf490983fc97489347ffae6d3a16187f7e18ef62b24eaa8c66df0e5373183860"
Jul 24 17:23:45.252235 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.252137    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf490983fc97489347ffae6d3a16187f7e18ef62b24eaa8c66df0e5373183860"} err="failed to get container status \"cf490983fc97489347ffae6d3a16187f7e18ef62b24eaa8c66df0e5373183860\": rpc error: code = NotFound desc = could not find container \"cf490983fc97489347ffae6d3a16187f7e18ef62b24eaa8c66df0e5373183860\": container with ID starting with cf490983fc97489347ffae6d3a16187f7e18ef62b24eaa8c66df0e5373183860 not found: ID does not exist"
Jul 24 17:23:45.262032 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.262011    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 17:23:45.262309 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.262296    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerName="alertmanager"
Jul 24 17:23:45.262356 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.262311    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerName="alertmanager"
Jul 24 17:23:45.262356 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.262320    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="615228ee-d6a8-4566-b5e7-82d5a27bf812" containerName="registry"
Jul 24 17:23:45.262356 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.262325    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="615228ee-d6a8-4566-b5e7-82d5a27bf812" containerName="registry"
Jul 24 17:23:45.262356 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.262333    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerName="prom-label-proxy"
Jul 24 17:23:45.262356 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.262338    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerName="prom-label-proxy"
Jul 24 17:23:45.262356 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.262348    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerName="kube-rbac-proxy-web"
Jul 24 17:23:45.262356 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.262354    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerName="kube-rbac-proxy-web"
Jul 24 17:23:45.262615 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.262362    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerName="kube-rbac-proxy"
Jul 24 17:23:45.262615 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.262367    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerName="kube-rbac-proxy"
Jul 24 17:23:45.262615 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.262379    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerName="init-config-reloader"
Jul 24 17:23:45.262615 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.262384    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerName="init-config-reloader"
Jul 24 17:23:45.262615 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.262392    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerName="kube-rbac-proxy-metric"
Jul 24 17:23:45.262615 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.262397    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerName="kube-rbac-proxy-metric"
Jul 24 17:23:45.262615 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.262405    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerName="config-reloader"
Jul 24 17:23:45.262615 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.262409    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerName="config-reloader"
Jul 24 17:23:45.262615 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.262464    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerName="kube-rbac-proxy"
Jul 24 17:23:45.262615 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.262471    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerName="alertmanager"
Jul 24 17:23:45.262615 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.262478    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerName="config-reloader"
Jul 24 17:23:45.262615 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.262484    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerName="prom-label-proxy"
Jul 24 17:23:45.262615 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.262491    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerName="kube-rbac-proxy-metric"
Jul 24 17:23:45.262615 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.262496    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="615228ee-d6a8-4566-b5e7-82d5a27bf812" containerName="registry"
Jul 24 17:23:45.262615 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.262501    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" containerName="kube-rbac-proxy-web"
Jul 24 17:23:45.266043 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.266025    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.268866 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.268848    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 24 17:23:45.268866 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.268858    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-p6wcp\""
Jul 24 17:23:45.268999 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.268849    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 24 17:23:45.269051 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.269033    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 24 17:23:45.269252 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.269236    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 24 17:23:45.269327 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.269312    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 24 17:23:45.269423 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.269406    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 24 17:23:45.269520 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.269446    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 24 17:23:45.269577 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.269527    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 24 17:23:45.277735 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.277716    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 24 17:23:45.282408 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.282387    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 17:23:45.366714 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.366678    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-web-config\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.366714 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.366718    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.366926 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.366741    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.366926 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.366781    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.366926 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.366812    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-config-volume\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.366926 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.366865    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.366926 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.366895    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vcnn\" (UniqueName: \"kubernetes.io/projected/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-kube-api-access-4vcnn\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.366926 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.366920    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-config-out\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.367108 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.366959    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.367108 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.366973    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.367108 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.366995    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.367108 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.367010    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.367108 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.367033    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.467812 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.467705    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.467812 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.467751    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.468044 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.467810    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.468044 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.467874    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-web-config\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.468044 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.467894    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.468246 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.468054    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.468246 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.468100    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.468246 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.468136    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-config-volume\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.468246 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.468203    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.468246 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.468228    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vcnn\" (UniqueName: \"kubernetes.io/projected/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-kube-api-access-4vcnn\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.468502 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.468256    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-config-out\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.468502 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.468295    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.468502 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.468319    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.468502 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.468345    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.469015 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.468977    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.471277 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.470933    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-web-config\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.471277 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.470939    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.471277 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.471061    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.471277 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.471080    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.471277 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.471092    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.471277 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.471239    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-config-out\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.471643 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.471440    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-config-volume\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.471737 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.471700    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.471737 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.471727    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.473102 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.473075    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.476762 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.476743    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vcnn\" (UniqueName: \"kubernetes.io/projected/b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e-kube-api-access-4vcnn\") pod \"alertmanager-main-0\" (UID: \"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.578836 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.578782    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:23:45.714932 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.714901    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 17:23:45.717497 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:23:45.717470    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb81cc4cf_8821_4ca7_8d3a_dbf14e46aa9e.slice/crio-85aa829bafcd435be8e99a62a172c081fc78455cf193bd44b2ccf4a340a83c85 WatchSource:0}: Error finding container 85aa829bafcd435be8e99a62a172c081fc78455cf193bd44b2ccf4a340a83c85: Status 404 returned error can't find the container with id 85aa829bafcd435be8e99a62a172c081fc78455cf193bd44b2ccf4a340a83c85
Jul 24 17:23:45.823776 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:45.823741    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b2aceee-d80a-40c0-a6f2-eca4f983d097" path="/var/lib/kubelet/pods/5b2aceee-d80a-40c0-a6f2-eca4f983d097/volumes"
Jul 24 17:23:46.203107 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:46.203073    2562 generic.go:356] "Generic (PLEG): container finished" podID="b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e" containerID="b1824c339ae05e9437574504039067a53902a7a5f934956419578684de94f497" exitCode=0
Jul 24 17:23:46.203530 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:46.203121    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e","Type":"ContainerDied","Data":"b1824c339ae05e9437574504039067a53902a7a5f934956419578684de94f497"}
Jul 24 17:23:46.203530 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:46.203201    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e","Type":"ContainerStarted","Data":"85aa829bafcd435be8e99a62a172c081fc78455cf193bd44b2ccf4a340a83c85"}
Jul 24 17:23:47.210413 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.210376    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e","Type":"ContainerStarted","Data":"ce853ca20a6c533f338d02c6d9cd2e339a08d97520b66687b5c025aed48bc859"}
Jul 24 17:23:47.210791 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.210460    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e","Type":"ContainerStarted","Data":"2b7f196fa7ea62ec446eff450724c1a5c9ac8855d2ccf0a35ed94a46ad35b31b"}
Jul 24 17:23:47.210791 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.210499    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e","Type":"ContainerStarted","Data":"814c452c1875f329dc5bf61f8b05d1a7f81e8bd8fcdb6efb4dbbac479a9424f6"}
Jul 24 17:23:47.210791 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.210570    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e","Type":"ContainerStarted","Data":"b9557551f24f55e77150dce3fed646bb7748d4ee6fe1af35ef65bfe5971c3c48"}
Jul 24 17:23:47.210791 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.210609    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e","Type":"ContainerStarted","Data":"d988f2e59e257cc14ad238915a9e4b8b7e6dae0c35543b27a5782a38700eb7c8"}
Jul 24 17:23:47.210791 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.210644    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e","Type":"ContainerStarted","Data":"25a0c847c1ab97d99dc9c7475740b7bbb15741e325162a276f4ea9a619409c18"}
Jul 24 17:23:47.241308 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.241255    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.22481364 podStartE2EDuration="2.241238827s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.425187ms" isStatefulPod=false podCreationTimestamp="2026-07-24 17:23:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:23:47.24021708 +0000 UTC m=+389.967373338" watchObservedRunningTime="2026-07-24 17:23:47.241238827 +0000 UTC m=+389.968395074"
Jul 24 17:23:47.935709 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.935676    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 17:23:47.938519 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.938500    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:47.941509 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.941483    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 24 17:23:47.942074 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.942051    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 24 17:23:47.942196 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.942116    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-eq40iv97hq7q2\""
Jul 24 17:23:47.942196 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.942182    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 24 17:23:47.942405 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.942392    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 24 17:23:47.942526 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.942509    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 24 17:23:47.942617 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.942599    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 24 17:23:47.942678 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.942601    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 24 17:23:47.942966 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.942950    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 24 17:23:47.943031 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.942950    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xbvp7\""
Jul 24 17:23:47.943221 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.943200    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 24 17:23:47.943318 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.943297    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 24 17:23:47.943392 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.943352    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 24 17:23:47.943588 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.943558    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 24 17:23:47.943665 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.943631    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 24 17:23:47.945315 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.945297    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 24 17:23:47.949411 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.949394    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 24 17:23:47.958400 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.958376    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 17:23:47.990319 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.990295    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79phz\" (UniqueName: \"kubernetes.io/projected/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-kube-api-access-79phz\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:47.990399 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.990322    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:47.990399 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.990344    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-config\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:47.990399 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.990368    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:47.990399 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.990391    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:47.990537 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.990407    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:47.990537 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.990468    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:47.990537 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.990496    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:47.990537 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.990531    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:47.990652 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.990550    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:47.990652 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.990572    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-config-out\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:47.990652 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.990595    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:47.990652 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.990637    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:47.990767 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.990663    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:47.990767 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.990680    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-web-config\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:47.990767 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.990701    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:47.990767 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.990718    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:47.990767 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.990735    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:47.990767 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.990757    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:47.990936 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:47.990790    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.091339 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.091307    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.091468 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.091349    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.091468 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.091376    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-web-config\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.091468 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.091421    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.091468 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.091447    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.091658 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.091474    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.091658 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.091500    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.091884 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.091858    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.091960 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.091906    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79phz\" (UniqueName: \"kubernetes.io/projected/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-kube-api-access-79phz\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.091960 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.091941    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.092054 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.091986    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-config\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.092054 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.092019    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.092054 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.092043    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.092232 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.092070    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.092232 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.092126    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.092232 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.092177    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.092232 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.092212    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.092420 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.092234    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.092420 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.092240    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.092420 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.092275    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.092420 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.092287    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-config-out\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.092420 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.092338    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.092420 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.092346    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.092927 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.092903    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.093006 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.092972    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.093242 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.093218    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.094570 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.094543    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.094665 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.094545    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.094665 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.094597    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-web-config\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.094805 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.094783    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-config-out\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.094940 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.094790    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.095686 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.095660    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.096203 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.096166    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.097089 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.097061    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.097196 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.097174    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-config\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.097409 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.097382    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.097484 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.097462    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.097546 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.097532    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.097693 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.097672    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.103536 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.103518    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79phz\" (UniqueName: \"kubernetes.io/projected/2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07-kube-api-access-79phz\") pod \"prometheus-k8s-0\" (UID: \"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.249433 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.249341    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:48.385907 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:48.385882    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 17:23:48.387628 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:23:48.387596    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bd02c4c_3a3d_460e_8d8f_2fc00ad2fe07.slice/crio-027a6f38afbfe9756771e47e041f509fa51837d8a42b271779719c25a567560b WatchSource:0}: Error finding container 027a6f38afbfe9756771e47e041f509fa51837d8a42b271779719c25a567560b: Status 404 returned error can't find the container with id 027a6f38afbfe9756771e47e041f509fa51837d8a42b271779719c25a567560b
Jul 24 17:23:49.220441 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:49.220405    2562 generic.go:356] "Generic (PLEG): container finished" podID="2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07" containerID="5e08d45c1335c443c523aaa823be2514cc08a4654f999eb9739266d977fa78a2" exitCode=0
Jul 24 17:23:49.220624 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:49.220486    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07","Type":"ContainerDied","Data":"5e08d45c1335c443c523aaa823be2514cc08a4654f999eb9739266d977fa78a2"}
Jul 24 17:23:49.220624 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:49.220553    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07","Type":"ContainerStarted","Data":"027a6f38afbfe9756771e47e041f509fa51837d8a42b271779719c25a567560b"}
Jul 24 17:23:52.236943 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:52.236892    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07","Type":"ContainerStarted","Data":"8c1c4eccebbefb57fff09938806a9bc6689c37fc43208fb570f57381867f22cd"}
Jul 24 17:23:53.243674 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:53.243633    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07","Type":"ContainerStarted","Data":"568f4011d6397c35006a08d223b2a88dbe0715b19bc64eb6046749f5a01d2574"}
Jul 24 17:23:53.244030 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:53.243716    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07","Type":"ContainerStarted","Data":"c21ae43755bacfcda77f2892a3e755a239c0a6470dbb203911d90b0453b87978"}
Jul 24 17:23:53.244030 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:53.243759    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07","Type":"ContainerStarted","Data":"3970cb7ae1abdec049f7f7d01debd3516fbc705b47a7abf4c269030949715c96"}
Jul 24 17:23:53.244030 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:53.243800    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07","Type":"ContainerStarted","Data":"79177054c9f5bd3834ce97bf086c86a0d31b2594cf12230a45250b456d14927e"}
Jul 24 17:23:53.244030 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:53.243841    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07","Type":"ContainerStarted","Data":"521baa7d39cc77094d24aa419cb5f2822da3977d0f8046bca2419e0293dcdffa"}
Jul 24 17:23:53.250038 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:53.250011    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:23:53.273914 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:23:53.273857    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.317090958 podStartE2EDuration="6.273840574s" totalImagesPullingTime="2.935169623s" totalInitContainerRuntime="21.579993ms" isStatefulPod=false podCreationTimestamp="2026-07-24 17:23:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:23:53.272186287 +0000 UTC m=+395.999342502" watchObservedRunningTime="2026-07-24 17:23:53.273840574 +0000 UTC m=+396.000996788"
Jul 24 17:24:48.249633 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:24:48.249538    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:24:48.268957 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:24:48.268930    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:24:48.467097 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:24:48.467070    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:25:36.687603 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:36.687569    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g4zd7"]
Jul 24 17:25:36.690260 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:36.690242    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g4zd7"
Jul 24 17:25:36.692916 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:36.692896    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 24 17:25:36.693031 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:36.692917    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 24 17:25:36.693944 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:36.693928    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-qvtw5\""
Jul 24 17:25:36.698714 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:36.698695    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g4zd7"]
Jul 24 17:25:36.864052 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:36.864016    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05e12860-30d7-4839-b51e-4dff2dc438fc-tmp\") pod \"openshift-lws-operator-fd8ccff4c-g4zd7\" (UID: \"05e12860-30d7-4839-b51e-4dff2dc438fc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g4zd7"
Jul 24 17:25:36.864247 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:36.864057    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjcqt\" (UniqueName: \"kubernetes.io/projected/05e12860-30d7-4839-b51e-4dff2dc438fc-kube-api-access-gjcqt\") pod \"openshift-lws-operator-fd8ccff4c-g4zd7\" (UID: \"05e12860-30d7-4839-b51e-4dff2dc438fc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g4zd7"
Jul 24 17:25:36.965611 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:36.965519    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05e12860-30d7-4839-b51e-4dff2dc438fc-tmp\") pod \"openshift-lws-operator-fd8ccff4c-g4zd7\" (UID: \"05e12860-30d7-4839-b51e-4dff2dc438fc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g4zd7"
Jul 24 17:25:36.965611 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:36.965587    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjcqt\" (UniqueName: \"kubernetes.io/projected/05e12860-30d7-4839-b51e-4dff2dc438fc-kube-api-access-gjcqt\") pod \"openshift-lws-operator-fd8ccff4c-g4zd7\" (UID: \"05e12860-30d7-4839-b51e-4dff2dc438fc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g4zd7"
Jul 24 17:25:36.966054 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:36.966034    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05e12860-30d7-4839-b51e-4dff2dc438fc-tmp\") pod \"openshift-lws-operator-fd8ccff4c-g4zd7\" (UID: \"05e12860-30d7-4839-b51e-4dff2dc438fc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g4zd7"
Jul 24 17:25:36.973865 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:36.973837    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjcqt\" (UniqueName: \"kubernetes.io/projected/05e12860-30d7-4839-b51e-4dff2dc438fc-kube-api-access-gjcqt\") pod \"openshift-lws-operator-fd8ccff4c-g4zd7\" (UID: \"05e12860-30d7-4839-b51e-4dff2dc438fc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g4zd7"
Jul 24 17:25:37.010712 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:37.010686    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g4zd7"
Jul 24 17:25:37.138957 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:37.138916    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g4zd7"]
Jul 24 17:25:37.141684 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:25:37.141656    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05e12860_30d7_4839_b51e_4dff2dc438fc.slice/crio-a4cba0d8271e0e980369cb0ebcd60e3b178e4e110e291d74e77e6ab19dc8e024 WatchSource:0}: Error finding container a4cba0d8271e0e980369cb0ebcd60e3b178e4e110e291d74e77e6ab19dc8e024: Status 404 returned error can't find the container with id a4cba0d8271e0e980369cb0ebcd60e3b178e4e110e291d74e77e6ab19dc8e024
Jul 24 17:25:37.644645 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:37.644615    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g4zd7" event={"ID":"05e12860-30d7-4839-b51e-4dff2dc438fc","Type":"ContainerStarted","Data":"a4cba0d8271e0e980369cb0ebcd60e3b178e4e110e291d74e77e6ab19dc8e024"}
Jul 24 17:25:40.662082 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:40.662046    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g4zd7" event={"ID":"05e12860-30d7-4839-b51e-4dff2dc438fc","Type":"ContainerStarted","Data":"19165aeb72190e56b4ff00ef20395b97c8e9e0e4b2817951bbd1b32b5aba7854"}
Jul 24 17:25:40.680901 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:40.680854    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g4zd7" podStartSLOduration=2.229726481 podStartE2EDuration="4.680839906s" totalImagesPullingTime="2.451113425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:25:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:25:40.679567688 +0000 UTC m=+503.406723914" watchObservedRunningTime="2026-07-24 17:25:40.680839906 +0000 UTC m=+503.407996146"
Jul 24 17:25:56.111873 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:56.111838    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8574785b6-rzf89"]
Jul 24 17:25:56.118128 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:56.118107    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8574785b6-rzf89"
Jul 24 17:25:56.121688 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:56.121670    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 24 17:25:56.123914 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:56.123895    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 24 17:25:56.124242 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:56.124210    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 24 17:25:56.124625 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:56.124458    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-xxn55\""
Jul 24 17:25:56.127982 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:56.127965    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 24 17:25:56.168153 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:56.168124    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8574785b6-rzf89"]
Jul 24 17:25:56.227252 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:56.227221    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89dvt\" (UniqueName: \"kubernetes.io/projected/43188074-d619-4727-959f-b834ccf6393e-kube-api-access-89dvt\") pod \"opendatahub-operator-controller-manager-8574785b6-rzf89\" (UID: \"43188074-d619-4727-959f-b834ccf6393e\") " pod="opendatahub/opendatahub-operator-controller-manager-8574785b6-rzf89"
Jul 24 17:25:56.227252 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:56.227253    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/43188074-d619-4727-959f-b834ccf6393e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8574785b6-rzf89\" (UID: \"43188074-d619-4727-959f-b834ccf6393e\") " pod="opendatahub/opendatahub-operator-controller-manager-8574785b6-rzf89"
Jul 24 17:25:56.227405 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:56.227270    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/43188074-d619-4727-959f-b834ccf6393e-manifests\") pod \"opendatahub-operator-controller-manager-8574785b6-rzf89\" (UID: \"43188074-d619-4727-959f-b834ccf6393e\") " pod="opendatahub/opendatahub-operator-controller-manager-8574785b6-rzf89"
Jul 24 17:25:56.227405 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:56.227290    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/43188074-d619-4727-959f-b834ccf6393e-webhook-cert\") pod \"opendatahub-operator-controller-manager-8574785b6-rzf89\" (UID: \"43188074-d619-4727-959f-b834ccf6393e\") " pod="opendatahub/opendatahub-operator-controller-manager-8574785b6-rzf89"
Jul 24 17:25:56.227405 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:56.227353    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43188074-d619-4727-959f-b834ccf6393e-tmp\") pod \"opendatahub-operator-controller-manager-8574785b6-rzf89\" (UID: \"43188074-d619-4727-959f-b834ccf6393e\") " pod="opendatahub/opendatahub-operator-controller-manager-8574785b6-rzf89"
Jul 24 17:25:56.328528 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:56.328501    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89dvt\" (UniqueName: \"kubernetes.io/projected/43188074-d619-4727-959f-b834ccf6393e-kube-api-access-89dvt\") pod \"opendatahub-operator-controller-manager-8574785b6-rzf89\" (UID: \"43188074-d619-4727-959f-b834ccf6393e\") " pod="opendatahub/opendatahub-operator-controller-manager-8574785b6-rzf89"
Jul 24 17:25:56.328685 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:56.328533    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/43188074-d619-4727-959f-b834ccf6393e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8574785b6-rzf89\" (UID: \"43188074-d619-4727-959f-b834ccf6393e\") " pod="opendatahub/opendatahub-operator-controller-manager-8574785b6-rzf89"
Jul 24 17:25:56.328685 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:56.328552    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/43188074-d619-4727-959f-b834ccf6393e-manifests\") pod \"opendatahub-operator-controller-manager-8574785b6-rzf89\" (UID: \"43188074-d619-4727-959f-b834ccf6393e\") " pod="opendatahub/opendatahub-operator-controller-manager-8574785b6-rzf89"
Jul 24 17:25:56.328685 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:56.328571    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/43188074-d619-4727-959f-b834ccf6393e-webhook-cert\") pod \"opendatahub-operator-controller-manager-8574785b6-rzf89\" (UID: \"43188074-d619-4727-959f-b834ccf6393e\") " pod="opendatahub/opendatahub-operator-controller-manager-8574785b6-rzf89"
Jul 24 17:25:56.328826 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:56.328696    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43188074-d619-4727-959f-b834ccf6393e-tmp\") pod \"opendatahub-operator-controller-manager-8574785b6-rzf89\" (UID: \"43188074-d619-4727-959f-b834ccf6393e\") " pod="opendatahub/opendatahub-operator-controller-manager-8574785b6-rzf89"
Jul 24 17:25:56.328965 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:56.328942    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/43188074-d619-4727-959f-b834ccf6393e-manifests\") pod \"opendatahub-operator-controller-manager-8574785b6-rzf89\" (UID: \"43188074-d619-4727-959f-b834ccf6393e\") " pod="opendatahub/opendatahub-operator-controller-manager-8574785b6-rzf89"
Jul 24 17:25:56.330957 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:56.330935    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43188074-d619-4727-959f-b834ccf6393e-tmp\") pod \"opendatahub-operator-controller-manager-8574785b6-rzf89\" (UID: \"43188074-d619-4727-959f-b834ccf6393e\") " pod="opendatahub/opendatahub-operator-controller-manager-8574785b6-rzf89"
Jul 24 17:25:56.331180 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:56.331160    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/43188074-d619-4727-959f-b834ccf6393e-webhook-cert\") pod \"opendatahub-operator-controller-manager-8574785b6-rzf89\" (UID: \"43188074-d619-4727-959f-b834ccf6393e\") " pod="opendatahub/opendatahub-operator-controller-manager-8574785b6-rzf89"
Jul 24 17:25:56.331236 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:56.331214    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/43188074-d619-4727-959f-b834ccf6393e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8574785b6-rzf89\" (UID: \"43188074-d619-4727-959f-b834ccf6393e\") " pod="opendatahub/opendatahub-operator-controller-manager-8574785b6-rzf89"
Jul 24 17:25:56.337331 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:56.337313    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89dvt\" (UniqueName: \"kubernetes.io/projected/43188074-d619-4727-959f-b834ccf6393e-kube-api-access-89dvt\") pod \"opendatahub-operator-controller-manager-8574785b6-rzf89\" (UID: \"43188074-d619-4727-959f-b834ccf6393e\") " pod="opendatahub/opendatahub-operator-controller-manager-8574785b6-rzf89"
Jul 24 17:25:56.428537 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:56.428454    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8574785b6-rzf89"
Jul 24 17:25:56.559603 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:56.559568    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8574785b6-rzf89"]
Jul 24 17:25:56.563401 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:25:56.563371    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43188074_d619_4727_959f_b834ccf6393e.slice/crio-0f04b4beeb41ea943b2bc16d11dd0ba05095502e82065ae299d29c1b600989f9 WatchSource:0}: Error finding container 0f04b4beeb41ea943b2bc16d11dd0ba05095502e82065ae299d29c1b600989f9: Status 404 returned error can't find the container with id 0f04b4beeb41ea943b2bc16d11dd0ba05095502e82065ae299d29c1b600989f9
Jul 24 17:25:56.729555 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:25:56.729473    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8574785b6-rzf89" event={"ID":"43188074-d619-4727-959f-b834ccf6393e","Type":"ContainerStarted","Data":"0f04b4beeb41ea943b2bc16d11dd0ba05095502e82065ae299d29c1b600989f9"}
Jul 24 17:26:00.750976 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:00.750932    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8574785b6-rzf89" event={"ID":"43188074-d619-4727-959f-b834ccf6393e","Type":"ContainerStarted","Data":"83120a0cd667297e8617aa34394b59ad9591b47d7f02532dbb401aa18f3b1d50"}
Jul 24 17:26:01.756916 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:01.756879    2562 generic.go:356] "Generic (PLEG): container finished" podID="43188074-d619-4727-959f-b834ccf6393e" containerID="83120a0cd667297e8617aa34394b59ad9591b47d7f02532dbb401aa18f3b1d50" exitCode=0
Jul 24 17:26:01.757306 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:01.756964    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8574785b6-rzf89" event={"ID":"43188074-d619-4727-959f-b834ccf6393e","Type":"ContainerDied","Data":"83120a0cd667297e8617aa34394b59ad9591b47d7f02532dbb401aa18f3b1d50"}
Jul 24 17:26:02.763932 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:02.763892    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8574785b6-rzf89" event={"ID":"43188074-d619-4727-959f-b834ccf6393e","Type":"ContainerStarted","Data":"9de7de04b7f653e47eb2c37cdc56da9bfc6421235e04b27ed37254c70aa67bff"}
Jul 24 17:26:02.764355 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:02.764035    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-8574785b6-rzf89"
Jul 24 17:26:02.786808 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:02.786752    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-8574785b6-rzf89" podStartSLOduration=2.047931822 podStartE2EDuration="6.786738365s" totalImagesPullingTime="3.677912366s" totalInitContainerRuntime="1.060894177s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:25:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:26:02.785801157 +0000 UTC m=+525.512957383" watchObservedRunningTime="2026-07-24 17:26:02.786738365 +0000 UTC m=+525.513894578"
Jul 24 17:26:13.771450 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:13.771417    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-8574785b6-rzf89"
Jul 24 17:26:23.947025 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:23.946986    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-cc7897f66-6ffhv"]
Jul 24 17:26:23.952370 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:23.952338    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-cc7897f66-6ffhv"
Jul 24 17:26:23.955677 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:23.955649    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-mmrsm\""
Jul 24 17:26:23.955809 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:23.955655    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 24 17:26:23.955809 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:23.955654    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 24 17:26:23.955809 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:23.955656    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 24 17:26:23.958830 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:23.958806    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-cc7897f66-6ffhv"]
Jul 24 17:26:24.071076 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:24.071045    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d68e6bb3-0b59-4ae7-ad1a-c8178e6fb582-manager-config\") pod \"lws-controller-manager-cc7897f66-6ffhv\" (UID: \"d68e6bb3-0b59-4ae7-ad1a-c8178e6fb582\") " pod="openshift-lws-operator/lws-controller-manager-cc7897f66-6ffhv"
Jul 24 17:26:24.071301 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:24.071085    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d68e6bb3-0b59-4ae7-ad1a-c8178e6fb582-cert\") pod \"lws-controller-manager-cc7897f66-6ffhv\" (UID: \"d68e6bb3-0b59-4ae7-ad1a-c8178e6fb582\") " pod="openshift-lws-operator/lws-controller-manager-cc7897f66-6ffhv"
Jul 24 17:26:24.071301 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:24.071105    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d68e6bb3-0b59-4ae7-ad1a-c8178e6fb582-metrics-cert\") pod \"lws-controller-manager-cc7897f66-6ffhv\" (UID: \"d68e6bb3-0b59-4ae7-ad1a-c8178e6fb582\") " pod="openshift-lws-operator/lws-controller-manager-cc7897f66-6ffhv"
Jul 24 17:26:24.071301 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:24.071207    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj9gz\" (UniqueName: \"kubernetes.io/projected/d68e6bb3-0b59-4ae7-ad1a-c8178e6fb582-kube-api-access-fj9gz\") pod \"lws-controller-manager-cc7897f66-6ffhv\" (UID: \"d68e6bb3-0b59-4ae7-ad1a-c8178e6fb582\") " pod="openshift-lws-operator/lws-controller-manager-cc7897f66-6ffhv"
Jul 24 17:26:24.172329 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:24.172297    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d68e6bb3-0b59-4ae7-ad1a-c8178e6fb582-manager-config\") pod \"lws-controller-manager-cc7897f66-6ffhv\" (UID: \"d68e6bb3-0b59-4ae7-ad1a-c8178e6fb582\") " pod="openshift-lws-operator/lws-controller-manager-cc7897f66-6ffhv"
Jul 24 17:26:24.172524 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:24.172337    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d68e6bb3-0b59-4ae7-ad1a-c8178e6fb582-cert\") pod \"lws-controller-manager-cc7897f66-6ffhv\" (UID: \"d68e6bb3-0b59-4ae7-ad1a-c8178e6fb582\") " pod="openshift-lws-operator/lws-controller-manager-cc7897f66-6ffhv"
Jul 24 17:26:24.172524 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:24.172352    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d68e6bb3-0b59-4ae7-ad1a-c8178e6fb582-metrics-cert\") pod \"lws-controller-manager-cc7897f66-6ffhv\" (UID: \"d68e6bb3-0b59-4ae7-ad1a-c8178e6fb582\") " pod="openshift-lws-operator/lws-controller-manager-cc7897f66-6ffhv"
Jul 24 17:26:24.172524 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:24.172386    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj9gz\" (UniqueName: \"kubernetes.io/projected/d68e6bb3-0b59-4ae7-ad1a-c8178e6fb582-kube-api-access-fj9gz\") pod \"lws-controller-manager-cc7897f66-6ffhv\" (UID: \"d68e6bb3-0b59-4ae7-ad1a-c8178e6fb582\") " pod="openshift-lws-operator/lws-controller-manager-cc7897f66-6ffhv"
Jul 24 17:26:24.173054 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:24.173027    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d68e6bb3-0b59-4ae7-ad1a-c8178e6fb582-manager-config\") pod \"lws-controller-manager-cc7897f66-6ffhv\" (UID: \"d68e6bb3-0b59-4ae7-ad1a-c8178e6fb582\") " pod="openshift-lws-operator/lws-controller-manager-cc7897f66-6ffhv"
Jul 24 17:26:24.174826 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:24.174807    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d68e6bb3-0b59-4ae7-ad1a-c8178e6fb582-metrics-cert\") pod \"lws-controller-manager-cc7897f66-6ffhv\" (UID: \"d68e6bb3-0b59-4ae7-ad1a-c8178e6fb582\") " pod="openshift-lws-operator/lws-controller-manager-cc7897f66-6ffhv"
Jul 24 17:26:24.174917 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:24.174881    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d68e6bb3-0b59-4ae7-ad1a-c8178e6fb582-cert\") pod \"lws-controller-manager-cc7897f66-6ffhv\" (UID: \"d68e6bb3-0b59-4ae7-ad1a-c8178e6fb582\") " pod="openshift-lws-operator/lws-controller-manager-cc7897f66-6ffhv"
Jul 24 17:26:24.180995 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:24.180970    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj9gz\" (UniqueName: \"kubernetes.io/projected/d68e6bb3-0b59-4ae7-ad1a-c8178e6fb582-kube-api-access-fj9gz\") pod \"lws-controller-manager-cc7897f66-6ffhv\" (UID: \"d68e6bb3-0b59-4ae7-ad1a-c8178e6fb582\") " pod="openshift-lws-operator/lws-controller-manager-cc7897f66-6ffhv"
Jul 24 17:26:24.263015 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:24.262924    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-cc7897f66-6ffhv"
Jul 24 17:26:24.406514 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:24.406438    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-cc7897f66-6ffhv"]
Jul 24 17:26:24.408861 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:26:24.408830    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd68e6bb3_0b59_4ae7_ad1a_c8178e6fb582.slice/crio-3906d07f4f2ec1eea6508be4100684942cccacae2ae35ea75c1af1dfed49d699 WatchSource:0}: Error finding container 3906d07f4f2ec1eea6508be4100684942cccacae2ae35ea75c1af1dfed49d699: Status 404 returned error can't find the container with id 3906d07f4f2ec1eea6508be4100684942cccacae2ae35ea75c1af1dfed49d699
Jul 24 17:26:24.858581 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:24.858548    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-cc7897f66-6ffhv" event={"ID":"d68e6bb3-0b59-4ae7-ad1a-c8178e6fb582","Type":"ContainerStarted","Data":"3906d07f4f2ec1eea6508be4100684942cccacae2ae35ea75c1af1dfed49d699"}
Jul 24 17:26:26.045585 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.045548    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv"]
Jul 24 17:26:26.049672 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.049646    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv"
Jul 24 17:26:26.052525 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.052500    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 24 17:26:26.052669 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.052572    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 24 17:26:26.052669 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.052611    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 24 17:26:26.052786 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.052670    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 24 17:26:26.052786 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.052756    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-ncljf\""
Jul 24 17:26:26.052980 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.052962    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 24 17:26:26.053079 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.053059    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 24 17:26:26.064099 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.064071    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv"]
Jul 24 17:26:26.190333 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.190297    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3f147b19-7e7d-48c3-81a1-b12bcefee1e1-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-4ngvv\" (UID: \"3f147b19-7e7d-48c3-81a1-b12bcefee1e1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv"
Jul 24 17:26:26.190484 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.190344    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3f147b19-7e7d-48c3-81a1-b12bcefee1e1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-4ngvv\" (UID: \"3f147b19-7e7d-48c3-81a1-b12bcefee1e1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv"
Jul 24 17:26:26.190484 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.190371    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f147b19-7e7d-48c3-81a1-b12bcefee1e1-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-4ngvv\" (UID: \"3f147b19-7e7d-48c3-81a1-b12bcefee1e1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv"
Jul 24 17:26:26.190484 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.190434    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-288g8\" (UniqueName: \"kubernetes.io/projected/3f147b19-7e7d-48c3-81a1-b12bcefee1e1-kube-api-access-288g8\") pod \"istiod-openshift-gateway-5677ccb49b-4ngvv\" (UID: \"3f147b19-7e7d-48c3-81a1-b12bcefee1e1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv"
Jul 24 17:26:26.190484 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.190478    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3f147b19-7e7d-48c3-81a1-b12bcefee1e1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-4ngvv\" (UID: \"3f147b19-7e7d-48c3-81a1-b12bcefee1e1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv"
Jul 24 17:26:26.190615 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.190498    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f147b19-7e7d-48c3-81a1-b12bcefee1e1-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-4ngvv\" (UID: \"3f147b19-7e7d-48c3-81a1-b12bcefee1e1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv"
Jul 24 17:26:26.190615 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.190558    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3f147b19-7e7d-48c3-81a1-b12bcefee1e1-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-4ngvv\" (UID: \"3f147b19-7e7d-48c3-81a1-b12bcefee1e1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv"
Jul 24 17:26:26.291605 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.291574    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3f147b19-7e7d-48c3-81a1-b12bcefee1e1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-4ngvv\" (UID: \"3f147b19-7e7d-48c3-81a1-b12bcefee1e1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv"
Jul 24 17:26:26.291605 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.291605    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f147b19-7e7d-48c3-81a1-b12bcefee1e1-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-4ngvv\" (UID: \"3f147b19-7e7d-48c3-81a1-b12bcefee1e1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv"
Jul 24 17:26:26.291818 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.291634    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-288g8\" (UniqueName: \"kubernetes.io/projected/3f147b19-7e7d-48c3-81a1-b12bcefee1e1-kube-api-access-288g8\") pod \"istiod-openshift-gateway-5677ccb49b-4ngvv\" (UID: \"3f147b19-7e7d-48c3-81a1-b12bcefee1e1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv"
Jul 24 17:26:26.291818 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.291675    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3f147b19-7e7d-48c3-81a1-b12bcefee1e1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-4ngvv\" (UID: \"3f147b19-7e7d-48c3-81a1-b12bcefee1e1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv"
Jul 24 17:26:26.291818 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.291699    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f147b19-7e7d-48c3-81a1-b12bcefee1e1-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-4ngvv\" (UID: \"3f147b19-7e7d-48c3-81a1-b12bcefee1e1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv"
Jul 24 17:26:26.291967 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.291902    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3f147b19-7e7d-48c3-81a1-b12bcefee1e1-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-4ngvv\" (UID: \"3f147b19-7e7d-48c3-81a1-b12bcefee1e1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv"
Jul 24 17:26:26.292018 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.291961    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3f147b19-7e7d-48c3-81a1-b12bcefee1e1-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-4ngvv\" (UID: \"3f147b19-7e7d-48c3-81a1-b12bcefee1e1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv"
Jul 24 17:26:26.292384 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.292362    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3f147b19-7e7d-48c3-81a1-b12bcefee1e1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-4ngvv\" (UID: \"3f147b19-7e7d-48c3-81a1-b12bcefee1e1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv"
Jul 24 17:26:26.293977 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.293958    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3f147b19-7e7d-48c3-81a1-b12bcefee1e1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-4ngvv\" (UID: \"3f147b19-7e7d-48c3-81a1-b12bcefee1e1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv"
Jul 24 17:26:26.294386 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.294369    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3f147b19-7e7d-48c3-81a1-b12bcefee1e1-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-4ngvv\" (UID: \"3f147b19-7e7d-48c3-81a1-b12bcefee1e1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv"
Jul 24 17:26:26.294446 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.294421    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f147b19-7e7d-48c3-81a1-b12bcefee1e1-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-4ngvv\" (UID: \"3f147b19-7e7d-48c3-81a1-b12bcefee1e1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv"
Jul 24 17:26:26.294483 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.294452    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3f147b19-7e7d-48c3-81a1-b12bcefee1e1-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-4ngvv\" (UID: \"3f147b19-7e7d-48c3-81a1-b12bcefee1e1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv"
Jul 24 17:26:26.306764 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.306735    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-288g8\" (UniqueName: \"kubernetes.io/projected/3f147b19-7e7d-48c3-81a1-b12bcefee1e1-kube-api-access-288g8\") pod \"istiod-openshift-gateway-5677ccb49b-4ngvv\" (UID: \"3f147b19-7e7d-48c3-81a1-b12bcefee1e1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv"
Jul 24 17:26:26.307693 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.307672    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f147b19-7e7d-48c3-81a1-b12bcefee1e1-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-4ngvv\" (UID: \"3f147b19-7e7d-48c3-81a1-b12bcefee1e1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv"
Jul 24 17:26:26.360802 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.360763    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv"
Jul 24 17:26:26.500479 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.500448    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv"]
Jul 24 17:26:26.503767 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:26:26.503737    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f147b19_7e7d_48c3_81a1_b12bcefee1e1.slice/crio-b532245a8ed8ca973a246631145575786faff62fe7e7f70145cddbca20fd38be WatchSource:0}: Error finding container b532245a8ed8ca973a246631145575786faff62fe7e7f70145cddbca20fd38be: Status 404 returned error can't find the container with id b532245a8ed8ca973a246631145575786faff62fe7e7f70145cddbca20fd38be
Jul 24 17:26:26.869482 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.869440    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-cc7897f66-6ffhv" event={"ID":"d68e6bb3-0b59-4ae7-ad1a-c8178e6fb582","Type":"ContainerStarted","Data":"afdfbc470cf405990805725ff0c63a56c990cd441271867e68e30c814db7db18"}
Jul 24 17:26:26.869675 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.869498    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-cc7897f66-6ffhv"
Jul 24 17:26:26.870679 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.870659    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv" event={"ID":"3f147b19-7e7d-48c3-81a1-b12bcefee1e1","Type":"ContainerStarted","Data":"b532245a8ed8ca973a246631145575786faff62fe7e7f70145cddbca20fd38be"}
Jul 24 17:26:26.902833 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:26.902760    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-cc7897f66-6ffhv" podStartSLOduration=2.090601457 podStartE2EDuration="3.902739894s" totalImagesPullingTime="1.812138437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:26:26.900538091 +0000 UTC m=+549.627694306" watchObservedRunningTime="2026-07-24 17:26:26.902739894 +0000 UTC m=+549.629896109"
Jul 24 17:26:29.182774 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:29.182735    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 24 17:26:29.183048 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:29.182813    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 24 17:26:29.890280 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:29.890244    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv" event={"ID":"3f147b19-7e7d-48c3-81a1-b12bcefee1e1","Type":"ContainerStarted","Data":"6203dc2f39352050df83dad7a3cf987d6b9572aafad2f9f9dfb2795aa37251e5"}
Jul 24 17:26:29.890445 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:29.890377    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv"
Jul 24 17:26:29.911669 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:29.911619    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv" podStartSLOduration=1.235117633 podStartE2EDuration="3.911604821s" totalImagesPullingTime="2.676487188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:26:29.910903849 +0000 UTC m=+552.638060062" watchObservedRunningTime="2026-07-24 17:26:29.911604821 +0000 UTC m=+552.638761035"
Jul 24 17:26:30.897226 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:30.897197    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4ngvv"
Jul 24 17:26:37.881373 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:26:37.881344    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-cc7897f66-6ffhv"
Jul 24 17:28:11.558250 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:28:11.558213    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fppqv"]
Jul 24 17:28:11.560745 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:28:11.560724    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fppqv"
Jul 24 17:28:11.563374 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:28:11.563349    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 24 17:28:11.563482 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:28:11.563399    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 24 17:28:11.563482 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:28:11.563415    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-csc7m\""
Jul 24 17:28:11.578611 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:28:11.578589    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fppqv"]
Jul 24 17:28:11.604177 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:28:11.604129    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa3a665c-675d-419d-8577-94075f56e37e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fppqv\" (UID: \"aa3a665c-675d-419d-8577-94075f56e37e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fppqv"
Jul 24 17:28:11.604274 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:28:11.604207    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wk9b2\" (UniqueName: \"kubernetes.io/projected/aa3a665c-675d-419d-8577-94075f56e37e-kube-api-access-wk9b2\") pod \"kuadrant-operator-controller-manager-86878c769f-fppqv\" (UID: \"aa3a665c-675d-419d-8577-94075f56e37e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fppqv"
Jul 24 17:28:11.705420 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:28:11.705392    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk9b2\" (UniqueName: \"kubernetes.io/projected/aa3a665c-675d-419d-8577-94075f56e37e-kube-api-access-wk9b2\") pod \"kuadrant-operator-controller-manager-86878c769f-fppqv\" (UID: \"aa3a665c-675d-419d-8577-94075f56e37e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fppqv"
Jul 24 17:28:11.705560 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:28:11.705452    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa3a665c-675d-419d-8577-94075f56e37e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fppqv\" (UID: \"aa3a665c-675d-419d-8577-94075f56e37e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fppqv"
Jul 24 17:28:11.705790 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:28:11.705774    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa3a665c-675d-419d-8577-94075f56e37e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fppqv\" (UID: \"aa3a665c-675d-419d-8577-94075f56e37e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fppqv"
Jul 24 17:28:11.715404 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:28:11.715377    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk9b2\" (UniqueName: \"kubernetes.io/projected/aa3a665c-675d-419d-8577-94075f56e37e-kube-api-access-wk9b2\") pod \"kuadrant-operator-controller-manager-86878c769f-fppqv\" (UID: \"aa3a665c-675d-419d-8577-94075f56e37e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fppqv"
Jul 24 17:28:11.870619 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:28:11.870593    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fppqv"
Jul 24 17:28:12.003354 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:28:12.003319    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fppqv"]
Jul 24 17:28:12.009596 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:28:12.009561    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa3a665c_675d_419d_8577_94075f56e37e.slice/crio-064f92839805d94b05b3208c0d8aa3021accba4aed1833228ef72a9e8d9ab58f WatchSource:0}: Error finding container 064f92839805d94b05b3208c0d8aa3021accba4aed1833228ef72a9e8d9ab58f: Status 404 returned error can't find the container with id 064f92839805d94b05b3208c0d8aa3021accba4aed1833228ef72a9e8d9ab58f
Jul 24 17:28:12.011809 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:28:12.011789    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 17:28:12.345225 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:28:12.345195    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fppqv" event={"ID":"aa3a665c-675d-419d-8577-94075f56e37e","Type":"ContainerStarted","Data":"064f92839805d94b05b3208c0d8aa3021accba4aed1833228ef72a9e8d9ab58f"}
Jul 24 17:28:16.366588 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:28:16.366549    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fppqv" event={"ID":"aa3a665c-675d-419d-8577-94075f56e37e","Type":"ContainerStarted","Data":"c2a53ace6d06d6abeaa2211c2f396fd03ab70c492bf1d2fcad20955caf6cbe8a"}
Jul 24 17:28:16.367043 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:28:16.366777    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fppqv"
Jul 24 17:28:16.388537 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:28:16.388485    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fppqv" podStartSLOduration=1.251846289 podStartE2EDuration="5.388472391s" totalImagesPullingTime="4.136626102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:28:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:28:16.385761459 +0000 UTC m=+659.112917687" watchObservedRunningTime="2026-07-24 17:28:16.388472391 +0000 UTC m=+659.115628606"
Jul 24 17:28:27.374068 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:28:27.374038    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fppqv"
Jul 24 17:29:28.858231 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:28.858197    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f5f66dd5d-qsxsk"]
Jul 24 17:29:28.861122 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:28.861099    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f5f66dd5d-qsxsk"
Jul 24 17:29:28.863981 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:28.863959    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 24 17:29:28.864089 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:28.863958    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-46q8n\""
Jul 24 17:29:28.864089 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:28.864008    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 24 17:29:28.874547 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:28.874513    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f5f66dd5d-qsxsk"]
Jul 24 17:29:28.971005 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:28.970972    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/110030f6-93be-4d2d-b010-89f009440f73-webhook-cert\") pod \"maas-controller-5f5f66dd5d-qsxsk\" (UID: \"110030f6-93be-4d2d-b010-89f009440f73\") " pod="opendatahub/maas-controller-5f5f66dd5d-qsxsk"
Jul 24 17:29:28.971203 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:28.971011    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnn2h\" (UniqueName: \"kubernetes.io/projected/110030f6-93be-4d2d-b010-89f009440f73-kube-api-access-cnn2h\") pod \"maas-controller-5f5f66dd5d-qsxsk\" (UID: \"110030f6-93be-4d2d-b010-89f009440f73\") " pod="opendatahub/maas-controller-5f5f66dd5d-qsxsk"
Jul 24 17:29:29.072265 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:29.072214    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/110030f6-93be-4d2d-b010-89f009440f73-webhook-cert\") pod \"maas-controller-5f5f66dd5d-qsxsk\" (UID: \"110030f6-93be-4d2d-b010-89f009440f73\") " pod="opendatahub/maas-controller-5f5f66dd5d-qsxsk"
Jul 24 17:29:29.072453 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:29.072281    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnn2h\" (UniqueName: \"kubernetes.io/projected/110030f6-93be-4d2d-b010-89f009440f73-kube-api-access-cnn2h\") pod \"maas-controller-5f5f66dd5d-qsxsk\" (UID: \"110030f6-93be-4d2d-b010-89f009440f73\") " pod="opendatahub/maas-controller-5f5f66dd5d-qsxsk"
Jul 24 17:29:29.074665 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:29.074639    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/110030f6-93be-4d2d-b010-89f009440f73-webhook-cert\") pod \"maas-controller-5f5f66dd5d-qsxsk\" (UID: \"110030f6-93be-4d2d-b010-89f009440f73\") " pod="opendatahub/maas-controller-5f5f66dd5d-qsxsk"
Jul 24 17:29:29.086043 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:29.086012    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnn2h\" (UniqueName: \"kubernetes.io/projected/110030f6-93be-4d2d-b010-89f009440f73-kube-api-access-cnn2h\") pod \"maas-controller-5f5f66dd5d-qsxsk\" (UID: \"110030f6-93be-4d2d-b010-89f009440f73\") " pod="opendatahub/maas-controller-5f5f66dd5d-qsxsk"
Jul 24 17:29:29.171480 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:29.171407    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f5f66dd5d-qsxsk"
Jul 24 17:29:29.298169 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:29.298127    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f5f66dd5d-qsxsk"]
Jul 24 17:29:29.299886 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:29:29.299857    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod110030f6_93be_4d2d_b010_89f009440f73.slice/crio-dcef7e36858e2d77a1629af4703be3aea0ca40ffb939e74cacbbca302a170051 WatchSource:0}: Error finding container dcef7e36858e2d77a1629af4703be3aea0ca40ffb939e74cacbbca302a170051: Status 404 returned error can't find the container with id dcef7e36858e2d77a1629af4703be3aea0ca40ffb939e74cacbbca302a170051
Jul 24 17:29:29.705856 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:29.705816    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f5f66dd5d-qsxsk" event={"ID":"110030f6-93be-4d2d-b010-89f009440f73","Type":"ContainerStarted","Data":"dcef7e36858e2d77a1629af4703be3aea0ca40ffb939e74cacbbca302a170051"}
Jul 24 17:29:31.717359 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:31.717313    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f5f66dd5d-qsxsk" event={"ID":"110030f6-93be-4d2d-b010-89f009440f73","Type":"ContainerStarted","Data":"e931e4fbf9e0c7ab2e68c82ea236fbc5925eaf10588a96c23cbb145154c4d9d9"}
Jul 24 17:29:31.717758 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:31.717525    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f5f66dd5d-qsxsk"
Jul 24 17:29:31.735875 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:31.735822    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5f5f66dd5d-qsxsk" podStartSLOduration=1.544729765 podStartE2EDuration="3.735803259s" totalImagesPullingTime="2.191073494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:29:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:29:31.733704227 +0000 UTC m=+734.460860441" watchObservedRunningTime="2026-07-24 17:29:31.735803259 +0000 UTC m=+734.462959472"
Jul 24 17:29:42.724801 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:42.724770    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f5f66dd5d-qsxsk"
Jul 24 17:29:42.993334 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:42.993251    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f5f66dd5d-qsxsk"]
Jul 24 17:29:42.993476 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:42.993450    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5f5f66dd5d-qsxsk" podUID="110030f6-93be-4d2d-b010-89f009440f73" containerName="manager" containerID="cri-o://e931e4fbf9e0c7ab2e68c82ea236fbc5925eaf10588a96c23cbb145154c4d9d9" gracePeriod=30
Jul 24 17:29:43.241777 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:43.241754    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f5f66dd5d-qsxsk"
Jul 24 17:29:43.292616 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:43.292545    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/110030f6-93be-4d2d-b010-89f009440f73-webhook-cert\") pod \"110030f6-93be-4d2d-b010-89f009440f73\" (UID: \"110030f6-93be-4d2d-b010-89f009440f73\") "
Jul 24 17:29:43.292616 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:43.292612    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnn2h\" (UniqueName: \"kubernetes.io/projected/110030f6-93be-4d2d-b010-89f009440f73-kube-api-access-cnn2h\") pod \"110030f6-93be-4d2d-b010-89f009440f73\" (UID: \"110030f6-93be-4d2d-b010-89f009440f73\") "
Jul 24 17:29:43.294628 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:43.294601    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/110030f6-93be-4d2d-b010-89f009440f73-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "110030f6-93be-4d2d-b010-89f009440f73" (UID: "110030f6-93be-4d2d-b010-89f009440f73"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:29:43.294731 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:43.294637    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/110030f6-93be-4d2d-b010-89f009440f73-kube-api-access-cnn2h" (OuterVolumeSpecName: "kube-api-access-cnn2h") pod "110030f6-93be-4d2d-b010-89f009440f73" (UID: "110030f6-93be-4d2d-b010-89f009440f73"). InnerVolumeSpecName "kube-api-access-cnn2h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:29:43.393304 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:43.393250    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnn2h\" (UniqueName: \"kubernetes.io/projected/110030f6-93be-4d2d-b010-89f009440f73-kube-api-access-cnn2h\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:29:43.393304 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:43.393300    2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/110030f6-93be-4d2d-b010-89f009440f73-webhook-cert\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:29:43.773459 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:43.773423    2562 generic.go:356] "Generic (PLEG): container finished" podID="110030f6-93be-4d2d-b010-89f009440f73" containerID="e931e4fbf9e0c7ab2e68c82ea236fbc5925eaf10588a96c23cbb145154c4d9d9" exitCode=0
Jul 24 17:29:43.773852 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:43.773468    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f5f66dd5d-qsxsk"
Jul 24 17:29:43.773852 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:43.773505    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f5f66dd5d-qsxsk" event={"ID":"110030f6-93be-4d2d-b010-89f009440f73","Type":"ContainerDied","Data":"e931e4fbf9e0c7ab2e68c82ea236fbc5925eaf10588a96c23cbb145154c4d9d9"}
Jul 24 17:29:43.773852 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:43.773556    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f5f66dd5d-qsxsk" event={"ID":"110030f6-93be-4d2d-b010-89f009440f73","Type":"ContainerDied","Data":"dcef7e36858e2d77a1629af4703be3aea0ca40ffb939e74cacbbca302a170051"}
Jul 24 17:29:43.773852 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:43.773582    2562 scope.go:117] "RemoveContainer" containerID="e931e4fbf9e0c7ab2e68c82ea236fbc5925eaf10588a96c23cbb145154c4d9d9"
Jul 24 17:29:43.782648 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:43.782632    2562 scope.go:117] "RemoveContainer" containerID="e931e4fbf9e0c7ab2e68c82ea236fbc5925eaf10588a96c23cbb145154c4d9d9"
Jul 24 17:29:43.782891 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:29:43.782872    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e931e4fbf9e0c7ab2e68c82ea236fbc5925eaf10588a96c23cbb145154c4d9d9\": container with ID starting with e931e4fbf9e0c7ab2e68c82ea236fbc5925eaf10588a96c23cbb145154c4d9d9 not found: ID does not exist" containerID="e931e4fbf9e0c7ab2e68c82ea236fbc5925eaf10588a96c23cbb145154c4d9d9"
Jul 24 17:29:43.782942 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:43.782900    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e931e4fbf9e0c7ab2e68c82ea236fbc5925eaf10588a96c23cbb145154c4d9d9"} err="failed to get container status \"e931e4fbf9e0c7ab2e68c82ea236fbc5925eaf10588a96c23cbb145154c4d9d9\": rpc error: code = NotFound desc = could not find container \"e931e4fbf9e0c7ab2e68c82ea236fbc5925eaf10588a96c23cbb145154c4d9d9\": container with ID starting with e931e4fbf9e0c7ab2e68c82ea236fbc5925eaf10588a96c23cbb145154c4d9d9 not found: ID does not exist"
Jul 24 17:29:43.795393 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:43.795372    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f5f66dd5d-qsxsk"]
Jul 24 17:29:43.797070 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:43.797050    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5f5f66dd5d-qsxsk"]
Jul 24 17:29:43.823699 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:29:43.823671    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="110030f6-93be-4d2d-b010-89f009440f73" path="/var/lib/kubelet/pods/110030f6-93be-4d2d-b010-89f009440f73/volumes"
Jul 24 17:30:04.451870 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:04.451828    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-78bb94d85f-94pdg"]
Jul 24 17:30:04.452355 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:04.452248    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="110030f6-93be-4d2d-b010-89f009440f73" containerName="manager"
Jul 24 17:30:04.452355 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:04.452263    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="110030f6-93be-4d2d-b010-89f009440f73" containerName="manager"
Jul 24 17:30:04.452355 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:04.452325    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="110030f6-93be-4d2d-b010-89f009440f73" containerName="manager"
Jul 24 17:30:04.458901 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:04.458873    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-78bb94d85f-94pdg"
Jul 24 17:30:04.461562 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:04.461540    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:30:04.462549 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:04.462530    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 17:30:04.462653 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:04.462553    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-rt4c4\""
Jul 24 17:30:04.462653 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:04.462566    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 24 17:30:04.467429 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:04.467405    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-78bb94d85f-94pdg"]
Jul 24 17:30:04.584624 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:04.584586    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f26a81fb-73fe-47cf-9c7e-f1a1c84eb905-maas-api-tls\") pod \"maas-api-78bb94d85f-94pdg\" (UID: \"f26a81fb-73fe-47cf-9c7e-f1a1c84eb905\") " pod="odh-ai-gateway-infra/maas-api-78bb94d85f-94pdg"
Jul 24 17:30:04.584806 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:04.584635    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtwtj\" (UniqueName: \"kubernetes.io/projected/f26a81fb-73fe-47cf-9c7e-f1a1c84eb905-kube-api-access-qtwtj\") pod \"maas-api-78bb94d85f-94pdg\" (UID: \"f26a81fb-73fe-47cf-9c7e-f1a1c84eb905\") " pod="odh-ai-gateway-infra/maas-api-78bb94d85f-94pdg"
Jul 24 17:30:04.584806 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:04.584669    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f26a81fb-73fe-47cf-9c7e-f1a1c84eb905-openshift-service-ca\") pod \"maas-api-78bb94d85f-94pdg\" (UID: \"f26a81fb-73fe-47cf-9c7e-f1a1c84eb905\") " pod="odh-ai-gateway-infra/maas-api-78bb94d85f-94pdg"
Jul 24 17:30:04.685907 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:04.685870    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f26a81fb-73fe-47cf-9c7e-f1a1c84eb905-maas-api-tls\") pod \"maas-api-78bb94d85f-94pdg\" (UID: \"f26a81fb-73fe-47cf-9c7e-f1a1c84eb905\") " pod="odh-ai-gateway-infra/maas-api-78bb94d85f-94pdg"
Jul 24 17:30:04.685907 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:04.685910    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtwtj\" (UniqueName: \"kubernetes.io/projected/f26a81fb-73fe-47cf-9c7e-f1a1c84eb905-kube-api-access-qtwtj\") pod \"maas-api-78bb94d85f-94pdg\" (UID: \"f26a81fb-73fe-47cf-9c7e-f1a1c84eb905\") " pod="odh-ai-gateway-infra/maas-api-78bb94d85f-94pdg"
Jul 24 17:30:04.686133 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:04.686091    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f26a81fb-73fe-47cf-9c7e-f1a1c84eb905-openshift-service-ca\") pod \"maas-api-78bb94d85f-94pdg\" (UID: \"f26a81fb-73fe-47cf-9c7e-f1a1c84eb905\") " pod="odh-ai-gateway-infra/maas-api-78bb94d85f-94pdg"
Jul 24 17:30:04.686744 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:04.686719    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f26a81fb-73fe-47cf-9c7e-f1a1c84eb905-openshift-service-ca\") pod \"maas-api-78bb94d85f-94pdg\" (UID: \"f26a81fb-73fe-47cf-9c7e-f1a1c84eb905\") " pod="odh-ai-gateway-infra/maas-api-78bb94d85f-94pdg"
Jul 24 17:30:04.688548 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:04.688527    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f26a81fb-73fe-47cf-9c7e-f1a1c84eb905-maas-api-tls\") pod \"maas-api-78bb94d85f-94pdg\" (UID: \"f26a81fb-73fe-47cf-9c7e-f1a1c84eb905\") " pod="odh-ai-gateway-infra/maas-api-78bb94d85f-94pdg"
Jul 24 17:30:04.693533 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:04.693508    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtwtj\" (UniqueName: \"kubernetes.io/projected/f26a81fb-73fe-47cf-9c7e-f1a1c84eb905-kube-api-access-qtwtj\") pod \"maas-api-78bb94d85f-94pdg\" (UID: \"f26a81fb-73fe-47cf-9c7e-f1a1c84eb905\") " pod="odh-ai-gateway-infra/maas-api-78bb94d85f-94pdg"
Jul 24 17:30:04.770343 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:04.770259    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-78bb94d85f-94pdg"
Jul 24 17:30:04.912275 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:04.912247    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-78bb94d85f-94pdg"]
Jul 24 17:30:04.913826 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:30:04.913795    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf26a81fb_73fe_47cf_9c7e_f1a1c84eb905.slice/crio-803e87c49ca7c9ce66e011ef79c850c982c1779239d985bb548507d04dd6c8d7 WatchSource:0}: Error finding container 803e87c49ca7c9ce66e011ef79c850c982c1779239d985bb548507d04dd6c8d7: Status 404 returned error can't find the container with id 803e87c49ca7c9ce66e011ef79c850c982c1779239d985bb548507d04dd6c8d7
Jul 24 17:30:04.960866 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:04.960834    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-dqxnq"]
Jul 24 17:30:04.965435 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:04.965416    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-dqxnq"
Jul 24 17:30:04.967797 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:04.967780    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 24 17:30:04.975507 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:04.975485    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-dqxnq"]
Jul 24 17:30:05.000034 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:05.000000    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-d2cqc"]
Jul 24 17:30:05.003441 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:05.003422    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-d2cqc"
Jul 24 17:30:05.008301 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:05.008280    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-l9dqc\""
Jul 24 17:30:05.019305 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:05.019273    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-d2cqc"]
Jul 24 17:30:05.089829 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:05.089797    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rstz2\" (UniqueName: \"kubernetes.io/projected/ae3f4133-08e5-429c-8122-a6fb03ac5bba-kube-api-access-rstz2\") pod \"payload-pre-processing-687dbb6cf6-dqxnq\" (UID: \"ae3f4133-08e5-429c-8122-a6fb03ac5bba\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-dqxnq"
Jul 24 17:30:05.090000 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:05.089836    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae3f4133-08e5-429c-8122-a6fb03ac5bba-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-dqxnq\" (UID: \"ae3f4133-08e5-429c-8122-a6fb03ac5bba\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-dqxnq"
Jul 24 17:30:05.090000 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:05.089950    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phz9h\" (UniqueName: \"kubernetes.io/projected/bf7f97e0-b594-4228-a7c6-197df6a2b4d7-kube-api-access-phz9h\") pod \"payload-processing-5cc6cd444b-d2cqc\" (UID: \"bf7f97e0-b594-4228-a7c6-197df6a2b4d7\") " pod="openshift-ingress/payload-processing-5cc6cd444b-d2cqc"
Jul 24 17:30:05.090077 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:05.090011    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf7f97e0-b594-4228-a7c6-197df6a2b4d7-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-d2cqc\" (UID: \"bf7f97e0-b594-4228-a7c6-197df6a2b4d7\") " pod="openshift-ingress/payload-processing-5cc6cd444b-d2cqc"
Jul 24 17:30:05.191272 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:05.191230    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf7f97e0-b594-4228-a7c6-197df6a2b4d7-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-d2cqc\" (UID: \"bf7f97e0-b594-4228-a7c6-197df6a2b4d7\") " pod="openshift-ingress/payload-processing-5cc6cd444b-d2cqc"
Jul 24 17:30:05.191453 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:05.191288    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rstz2\" (UniqueName: \"kubernetes.io/projected/ae3f4133-08e5-429c-8122-a6fb03ac5bba-kube-api-access-rstz2\") pod \"payload-pre-processing-687dbb6cf6-dqxnq\" (UID: \"ae3f4133-08e5-429c-8122-a6fb03ac5bba\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-dqxnq"
Jul 24 17:30:05.191453 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:05.191310    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae3f4133-08e5-429c-8122-a6fb03ac5bba-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-dqxnq\" (UID: \"ae3f4133-08e5-429c-8122-a6fb03ac5bba\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-dqxnq"
Jul 24 17:30:05.191453 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:05.191363    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phz9h\" (UniqueName: \"kubernetes.io/projected/bf7f97e0-b594-4228-a7c6-197df6a2b4d7-kube-api-access-phz9h\") pod \"payload-processing-5cc6cd444b-d2cqc\" (UID: \"bf7f97e0-b594-4228-a7c6-197df6a2b4d7\") " pod="openshift-ingress/payload-processing-5cc6cd444b-d2cqc"
Jul 24 17:30:05.192213 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:05.192188    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf7f97e0-b594-4228-a7c6-197df6a2b4d7-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-d2cqc\" (UID: \"bf7f97e0-b594-4228-a7c6-197df6a2b4d7\") " pod="openshift-ingress/payload-processing-5cc6cd444b-d2cqc"
Jul 24 17:30:05.192262 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:05.192199    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae3f4133-08e5-429c-8122-a6fb03ac5bba-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-dqxnq\" (UID: \"ae3f4133-08e5-429c-8122-a6fb03ac5bba\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-dqxnq"
Jul 24 17:30:05.200929 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:05.200899    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phz9h\" (UniqueName: \"kubernetes.io/projected/bf7f97e0-b594-4228-a7c6-197df6a2b4d7-kube-api-access-phz9h\") pod \"payload-processing-5cc6cd444b-d2cqc\" (UID: \"bf7f97e0-b594-4228-a7c6-197df6a2b4d7\") " pod="openshift-ingress/payload-processing-5cc6cd444b-d2cqc"
Jul 24 17:30:05.201035 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:05.200929    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rstz2\" (UniqueName: \"kubernetes.io/projected/ae3f4133-08e5-429c-8122-a6fb03ac5bba-kube-api-access-rstz2\") pod \"payload-pre-processing-687dbb6cf6-dqxnq\" (UID: \"ae3f4133-08e5-429c-8122-a6fb03ac5bba\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-dqxnq"
Jul 24 17:30:05.277613 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:05.277574    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-dqxnq"
Jul 24 17:30:05.314191 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:05.314135    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-d2cqc"
Jul 24 17:30:05.423186 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:05.423134    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-dqxnq"]
Jul 24 17:30:05.425798 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:30:05.425774    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae3f4133_08e5_429c_8122_a6fb03ac5bba.slice/crio-33f4299feebdd94f3ae63812ab0d30cbc0c23fd980310efcdb6e2e2b112135a8 WatchSource:0}: Error finding container 33f4299feebdd94f3ae63812ab0d30cbc0c23fd980310efcdb6e2e2b112135a8: Status 404 returned error can't find the container with id 33f4299feebdd94f3ae63812ab0d30cbc0c23fd980310efcdb6e2e2b112135a8
Jul 24 17:30:05.468938 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:05.468906    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-d2cqc"]
Jul 24 17:30:05.472260 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:30:05.472224    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf7f97e0_b594_4228_a7c6_197df6a2b4d7.slice/crio-750e08db828d2c1407e94c1c73c1e6adb72818fb38f78c2c57ccdaab93e303e4 WatchSource:0}: Error finding container 750e08db828d2c1407e94c1c73c1e6adb72818fb38f78c2c57ccdaab93e303e4: Status 404 returned error can't find the container with id 750e08db828d2c1407e94c1c73c1e6adb72818fb38f78c2c57ccdaab93e303e4
Jul 24 17:30:05.880862 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:05.880818    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-d2cqc" event={"ID":"bf7f97e0-b594-4228-a7c6-197df6a2b4d7","Type":"ContainerStarted","Data":"750e08db828d2c1407e94c1c73c1e6adb72818fb38f78c2c57ccdaab93e303e4"}
Jul 24 17:30:05.881883 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:05.881857    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-78bb94d85f-94pdg" event={"ID":"f26a81fb-73fe-47cf-9c7e-f1a1c84eb905","Type":"ContainerStarted","Data":"803e87c49ca7c9ce66e011ef79c850c982c1779239d985bb548507d04dd6c8d7"}
Jul 24 17:30:05.882873 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:05.882855    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-dqxnq" event={"ID":"ae3f4133-08e5-429c-8122-a6fb03ac5bba","Type":"ContainerStarted","Data":"33f4299feebdd94f3ae63812ab0d30cbc0c23fd980310efcdb6e2e2b112135a8"}
Jul 24 17:30:08.900687 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:08.900647    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-d2cqc" event={"ID":"bf7f97e0-b594-4228-a7c6-197df6a2b4d7","Type":"ContainerStarted","Data":"d6b97959ce45d9be97ea2097165616d817ecfb3bb30fbc8411c506d7d94a18d4"}
Jul 24 17:30:08.901129 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:08.900742    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-d2cqc"
Jul 24 17:30:08.902348 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:08.902317    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-78bb94d85f-94pdg" event={"ID":"f26a81fb-73fe-47cf-9c7e-f1a1c84eb905","Type":"ContainerStarted","Data":"7bd9c285707acccb6aed53c0fda359b5623207f696a2e98481757c10b6703763"}
Jul 24 17:30:08.902463 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:08.902417    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-78bb94d85f-94pdg"
Jul 24 17:30:08.904186 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:08.904123    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-dqxnq" event={"ID":"ae3f4133-08e5-429c-8122-a6fb03ac5bba","Type":"ContainerStarted","Data":"ea07b348cf07ade3cba3d40d25d320164ed2e92462ad83a089a125518515e6e5"}
Jul 24 17:30:08.904306 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:08.904204    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-dqxnq"
Jul 24 17:30:08.920134 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:08.920075    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-d2cqc" podStartSLOduration=2.376244172 podStartE2EDuration="4.920056676s" totalImagesPullingTime="2.543812504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:30:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:30:08.917131393 +0000 UTC m=+771.644287608" watchObservedRunningTime="2026-07-24 17:30:08.920056676 +0000 UTC m=+771.647212887"
Jul 24 17:30:08.935037 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:08.934983    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-78bb94d85f-94pdg" podStartSLOduration=1.900626138 podStartE2EDuration="4.934967415s" totalImagesPullingTime="3.034341277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:30:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:30:08.933662086 +0000 UTC m=+771.660818303" watchObservedRunningTime="2026-07-24 17:30:08.934967415 +0000 UTC m=+771.662123648"
Jul 24 17:30:08.949435 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:08.949381    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-dqxnq" podStartSLOduration=2.3737831480000002 podStartE2EDuration="4.949359307s" totalImagesPullingTime="2.575576159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:30:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:30:08.948178091 +0000 UTC m=+771.675334301" watchObservedRunningTime="2026-07-24 17:30:08.949359307 +0000 UTC m=+771.676515521"
Jul 24 17:30:14.916410 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:14.916377    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-78bb94d85f-94pdg"
Jul 24 17:30:19.912174 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:19.912120    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-d2cqc"
Jul 24 17:30:19.912539 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:19.912228    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-dqxnq"
Jul 24 17:30:35.391430 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:35.391391    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd"]
Jul 24 17:30:35.395806 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:35.395788    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd"
Jul 24 17:30:35.399530 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:35.399501    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 24 17:30:35.399530 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:35.399518    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 24 17:30:35.399715 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:35.399570    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 24 17:30:35.399715 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:35.399512    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4mjtq\""
Jul 24 17:30:35.403402 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:35.402996    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd"]
Jul 24 17:30:35.462980 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:35.462944    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2feb1d79-5ba4-4bcf-8887-f1b664e7085a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lxlwd\" (UID: \"2feb1d79-5ba4-4bcf-8887-f1b664e7085a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd"
Jul 24 17:30:35.463173 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:35.462983    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2feb1d79-5ba4-4bcf-8887-f1b664e7085a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lxlwd\" (UID: \"2feb1d79-5ba4-4bcf-8887-f1b664e7085a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd"
Jul 24 17:30:35.463173 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:35.463084    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2feb1d79-5ba4-4bcf-8887-f1b664e7085a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lxlwd\" (UID: \"2feb1d79-5ba4-4bcf-8887-f1b664e7085a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd"
Jul 24 17:30:35.463173 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:35.463128    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dfmd\" (UniqueName: \"kubernetes.io/projected/2feb1d79-5ba4-4bcf-8887-f1b664e7085a-kube-api-access-4dfmd\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lxlwd\" (UID: \"2feb1d79-5ba4-4bcf-8887-f1b664e7085a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd"
Jul 24 17:30:35.463348 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:35.463240    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2feb1d79-5ba4-4bcf-8887-f1b664e7085a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lxlwd\" (UID: \"2feb1d79-5ba4-4bcf-8887-f1b664e7085a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd"
Jul 24 17:30:35.463348 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:35.463277    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2feb1d79-5ba4-4bcf-8887-f1b664e7085a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lxlwd\" (UID: \"2feb1d79-5ba4-4bcf-8887-f1b664e7085a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd"
Jul 24 17:30:35.564081 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:35.564043    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2feb1d79-5ba4-4bcf-8887-f1b664e7085a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lxlwd\" (UID: \"2feb1d79-5ba4-4bcf-8887-f1b664e7085a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd"
Jul 24 17:30:35.564081 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:35.564083    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dfmd\" (UniqueName: \"kubernetes.io/projected/2feb1d79-5ba4-4bcf-8887-f1b664e7085a-kube-api-access-4dfmd\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lxlwd\" (UID: \"2feb1d79-5ba4-4bcf-8887-f1b664e7085a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd"
Jul 24 17:30:35.564352 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:35.564122    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2feb1d79-5ba4-4bcf-8887-f1b664e7085a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lxlwd\" (UID: \"2feb1d79-5ba4-4bcf-8887-f1b664e7085a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd"
Jul 24 17:30:35.564352 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:35.564174    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2feb1d79-5ba4-4bcf-8887-f1b664e7085a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lxlwd\" (UID: \"2feb1d79-5ba4-4bcf-8887-f1b664e7085a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd"
Jul 24 17:30:35.564352 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:35.564219    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2feb1d79-5ba4-4bcf-8887-f1b664e7085a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lxlwd\" (UID: \"2feb1d79-5ba4-4bcf-8887-f1b664e7085a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd"
Jul 24 17:30:35.564352 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:35.564257    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2feb1d79-5ba4-4bcf-8887-f1b664e7085a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lxlwd\" (UID: \"2feb1d79-5ba4-4bcf-8887-f1b664e7085a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd"
Jul 24 17:30:35.564570 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:35.564548    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2feb1d79-5ba4-4bcf-8887-f1b664e7085a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lxlwd\" (UID: \"2feb1d79-5ba4-4bcf-8887-f1b664e7085a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd"
Jul 24 17:30:35.564631 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:35.564583    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2feb1d79-5ba4-4bcf-8887-f1b664e7085a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lxlwd\" (UID: \"2feb1d79-5ba4-4bcf-8887-f1b664e7085a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd"
Jul 24 17:30:35.564688 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:35.564649    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2feb1d79-5ba4-4bcf-8887-f1b664e7085a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lxlwd\" (UID: \"2feb1d79-5ba4-4bcf-8887-f1b664e7085a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd"
Jul 24 17:30:35.566475 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:35.566455    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2feb1d79-5ba4-4bcf-8887-f1b664e7085a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lxlwd\" (UID: \"2feb1d79-5ba4-4bcf-8887-f1b664e7085a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd"
Jul 24 17:30:35.566766 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:35.566746    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2feb1d79-5ba4-4bcf-8887-f1b664e7085a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lxlwd\" (UID: \"2feb1d79-5ba4-4bcf-8887-f1b664e7085a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd"
Jul 24 17:30:35.573321 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:35.573289    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dfmd\" (UniqueName: \"kubernetes.io/projected/2feb1d79-5ba4-4bcf-8887-f1b664e7085a-kube-api-access-4dfmd\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lxlwd\" (UID: \"2feb1d79-5ba4-4bcf-8887-f1b664e7085a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd"
Jul 24 17:30:35.707747 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:35.707657    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd"
Jul 24 17:30:35.852783 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:35.852651    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd"]
Jul 24 17:30:35.855163 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:30:35.855125    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2feb1d79_5ba4_4bcf_8887_f1b664e7085a.slice/crio-96f25d3847ee4c259a6be90f8708a14a59c190dce33ee6bbac305a74ae11ef6e WatchSource:0}: Error finding container 96f25d3847ee4c259a6be90f8708a14a59c190dce33ee6bbac305a74ae11ef6e: Status 404 returned error can't find the container with id 96f25d3847ee4c259a6be90f8708a14a59c190dce33ee6bbac305a74ae11ef6e
Jul 24 17:30:36.036618 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:36.036533    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd" event={"ID":"2feb1d79-5ba4-4bcf-8887-f1b664e7085a","Type":"ContainerStarted","Data":"96f25d3847ee4c259a6be90f8708a14a59c190dce33ee6bbac305a74ae11ef6e"}
Jul 24 17:30:37.043571 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:37.043539    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd" event={"ID":"2feb1d79-5ba4-4bcf-8887-f1b664e7085a","Type":"ContainerStarted","Data":"0ae1bcae1c8985bcdfb81256d99b1185d20c94c65e85783ab0a5b50ad1baa154"}
Jul 24 17:30:37.062071 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:37.061963    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd" podStartSLOduration=1.105423526 podStartE2EDuration="2.061948172s" totalImagesPullingTime="956.524646ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:30:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:30:37.059778891 +0000 UTC m=+799.786935105" watchObservedRunningTime="2026-07-24 17:30:37.061948172 +0000 UTC m=+799.789104387"
Jul 24 17:30:45.708073 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:45.707981    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd"
Jul 24 17:30:45.708533 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:45.708100    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd"
Jul 24 17:30:45.721117 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:45.721092    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd"
Jul 24 17:30:46.097051 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:46.097019    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lxlwd"
Jul 24 17:30:59.581525 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:59.581486    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg"]
Jul 24 17:30:59.585184 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:59.585161    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg"
Jul 24 17:30:59.588059 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:59.588041    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 24 17:30:59.595871 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:59.595846    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg"]
Jul 24 17:30:59.675614 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:59.675569    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6900cebf-8424-419b-94ad-1c631161e1bd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cxztg\" (UID: \"6900cebf-8424-419b-94ad-1c631161e1bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg"
Jul 24 17:30:59.675937 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:59.675906    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6900cebf-8424-419b-94ad-1c631161e1bd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cxztg\" (UID: \"6900cebf-8424-419b-94ad-1c631161e1bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg"
Jul 24 17:30:59.676052 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:59.675944    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6900cebf-8424-419b-94ad-1c631161e1bd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cxztg\" (UID: \"6900cebf-8424-419b-94ad-1c631161e1bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg"
Jul 24 17:30:59.676052 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:59.675979    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6900cebf-8424-419b-94ad-1c631161e1bd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cxztg\" (UID: \"6900cebf-8424-419b-94ad-1c631161e1bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg"
Jul 24 17:30:59.676052 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:59.676019    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6900cebf-8424-419b-94ad-1c631161e1bd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cxztg\" (UID: \"6900cebf-8424-419b-94ad-1c631161e1bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg"
Jul 24 17:30:59.676260 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:59.676108    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5l8p\" (UniqueName: \"kubernetes.io/projected/6900cebf-8424-419b-94ad-1c631161e1bd-kube-api-access-f5l8p\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cxztg\" (UID: \"6900cebf-8424-419b-94ad-1c631161e1bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg"
Jul 24 17:30:59.777374 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:59.777334    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6900cebf-8424-419b-94ad-1c631161e1bd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cxztg\" (UID: \"6900cebf-8424-419b-94ad-1c631161e1bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg"
Jul 24 17:30:59.777374 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:59.777376    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6900cebf-8424-419b-94ad-1c631161e1bd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cxztg\" (UID: \"6900cebf-8424-419b-94ad-1c631161e1bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg"
Jul 24 17:30:59.777629 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:59.777402    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6900cebf-8424-419b-94ad-1c631161e1bd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cxztg\" (UID: \"6900cebf-8424-419b-94ad-1c631161e1bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg"
Jul 24 17:30:59.777629 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:59.777448    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6900cebf-8424-419b-94ad-1c631161e1bd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cxztg\" (UID: \"6900cebf-8424-419b-94ad-1c631161e1bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg"
Jul 24 17:30:59.777629 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:59.777483    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5l8p\" (UniqueName: \"kubernetes.io/projected/6900cebf-8424-419b-94ad-1c631161e1bd-kube-api-access-f5l8p\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cxztg\" (UID: \"6900cebf-8424-419b-94ad-1c631161e1bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg"
Jul 24 17:30:59.777629 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:59.777547    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6900cebf-8424-419b-94ad-1c631161e1bd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cxztg\" (UID: \"6900cebf-8424-419b-94ad-1c631161e1bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg"
Jul 24 17:30:59.777861 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:59.777834    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6900cebf-8424-419b-94ad-1c631161e1bd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cxztg\" (UID: \"6900cebf-8424-419b-94ad-1c631161e1bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg"
Jul 24 17:30:59.777920 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:59.777868    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6900cebf-8424-419b-94ad-1c631161e1bd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cxztg\" (UID: \"6900cebf-8424-419b-94ad-1c631161e1bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg"
Jul 24 17:30:59.777958 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:59.777932    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6900cebf-8424-419b-94ad-1c631161e1bd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cxztg\" (UID: \"6900cebf-8424-419b-94ad-1c631161e1bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg"
Jul 24 17:30:59.780290 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:59.780249    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6900cebf-8424-419b-94ad-1c631161e1bd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cxztg\" (UID: \"6900cebf-8424-419b-94ad-1c631161e1bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg"
Jul 24 17:30:59.780290 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:59.780248    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6900cebf-8424-419b-94ad-1c631161e1bd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cxztg\" (UID: \"6900cebf-8424-419b-94ad-1c631161e1bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg"
Jul 24 17:30:59.786834 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:59.786807    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5l8p\" (UniqueName: \"kubernetes.io/projected/6900cebf-8424-419b-94ad-1c631161e1bd-kube-api-access-f5l8p\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cxztg\" (UID: \"6900cebf-8424-419b-94ad-1c631161e1bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg"
Jul 24 17:30:59.896967 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:30:59.896921    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg"
Jul 24 17:31:00.035875 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:31:00.035843    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg"]
Jul 24 17:31:00.038591 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:31:00.038554    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6900cebf_8424_419b_94ad_1c631161e1bd.slice/crio-7b5ead01c0c6565349652a85d933f61b1f9e5345498e8db65a10d8c73139048f WatchSource:0}: Error finding container 7b5ead01c0c6565349652a85d933f61b1f9e5345498e8db65a10d8c73139048f: Status 404 returned error can't find the container with id 7b5ead01c0c6565349652a85d933f61b1f9e5345498e8db65a10d8c73139048f
Jul 24 17:31:00.152220 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:31:00.152118    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg" event={"ID":"6900cebf-8424-419b-94ad-1c631161e1bd","Type":"ContainerStarted","Data":"7b5ead01c0c6565349652a85d933f61b1f9e5345498e8db65a10d8c73139048f"}
Jul 24 17:31:01.158543 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:31:01.158509    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg" event={"ID":"6900cebf-8424-419b-94ad-1c631161e1bd","Type":"ContainerStarted","Data":"3b1d87f38d9de66b6a73f0ffb5a21919a9b30824b9e46f3b949048e28fb3c286"}
Jul 24 17:31:01.178304 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:31:01.178251    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg" podStartSLOduration=1.980219883 podStartE2EDuration="2.178234644s" totalImagesPullingTime="198.014761ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:30:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:31:01.175836202 +0000 UTC m=+823.902992418" watchObservedRunningTime="2026-07-24 17:31:01.178234644 +0000 UTC m=+823.905390857"
Jul 24 17:31:09.897217 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:31:09.897181    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg"
Jul 24 17:31:09.897721 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:31:09.897243    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg"
Jul 24 17:31:09.909963 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:31:09.909939    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg"
Jul 24 17:31:10.214884 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:31:10.214794    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cxztg"
Jul 24 17:34:03.779374 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:03.779336    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5ff575bb75-h68jp"]
Jul 24 17:34:03.782514 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:03.782494    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5ff575bb75-h68jp"
Jul 24 17:34:03.786610 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:03.786583    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-46q8n\""
Jul 24 17:34:03.786610 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:03.786584    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 24 17:34:03.786750 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:03.786592    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 24 17:34:03.799966 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:03.799944    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5ff575bb75-h68jp"]
Jul 24 17:34:03.852920 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:03.852894    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6cd1c489-6360-4c70-8492-dba66b5bc327-webhook-cert\") pod \"maas-controller-5ff575bb75-h68jp\" (UID: \"6cd1c489-6360-4c70-8492-dba66b5bc327\") " pod="opendatahub/maas-controller-5ff575bb75-h68jp"
Jul 24 17:34:03.852920 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:03.852921    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94kz6\" (UniqueName: \"kubernetes.io/projected/6cd1c489-6360-4c70-8492-dba66b5bc327-kube-api-access-94kz6\") pod \"maas-controller-5ff575bb75-h68jp\" (UID: \"6cd1c489-6360-4c70-8492-dba66b5bc327\") " pod="opendatahub/maas-controller-5ff575bb75-h68jp"
Jul 24 17:34:03.953488 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:03.953461    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6cd1c489-6360-4c70-8492-dba66b5bc327-webhook-cert\") pod \"maas-controller-5ff575bb75-h68jp\" (UID: \"6cd1c489-6360-4c70-8492-dba66b5bc327\") " pod="opendatahub/maas-controller-5ff575bb75-h68jp"
Jul 24 17:34:03.953488 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:03.953494    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94kz6\" (UniqueName: \"kubernetes.io/projected/6cd1c489-6360-4c70-8492-dba66b5bc327-kube-api-access-94kz6\") pod \"maas-controller-5ff575bb75-h68jp\" (UID: \"6cd1c489-6360-4c70-8492-dba66b5bc327\") " pod="opendatahub/maas-controller-5ff575bb75-h68jp"
Jul 24 17:34:03.955893 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:03.955870    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6cd1c489-6360-4c70-8492-dba66b5bc327-webhook-cert\") pod \"maas-controller-5ff575bb75-h68jp\" (UID: \"6cd1c489-6360-4c70-8492-dba66b5bc327\") " pod="opendatahub/maas-controller-5ff575bb75-h68jp"
Jul 24 17:34:03.970608 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:03.970588    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94kz6\" (UniqueName: \"kubernetes.io/projected/6cd1c489-6360-4c70-8492-dba66b5bc327-kube-api-access-94kz6\") pod \"maas-controller-5ff575bb75-h68jp\" (UID: \"6cd1c489-6360-4c70-8492-dba66b5bc327\") " pod="opendatahub/maas-controller-5ff575bb75-h68jp"
Jul 24 17:34:04.092520 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:04.092487    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5ff575bb75-h68jp"
Jul 24 17:34:04.229383 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:04.229356    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5ff575bb75-h68jp"]
Jul 24 17:34:04.231499 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:34:04.231472    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cd1c489_6360_4c70_8492_dba66b5bc327.slice/crio-c09b18963ad2e842f83c5719dd3bcb65cc9ababc84b1e27ee75c17ccebd199d1 WatchSource:0}: Error finding container c09b18963ad2e842f83c5719dd3bcb65cc9ababc84b1e27ee75c17ccebd199d1: Status 404 returned error can't find the container with id c09b18963ad2e842f83c5719dd3bcb65cc9ababc84b1e27ee75c17ccebd199d1
Jul 24 17:34:04.232854 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:04.232839    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 17:34:05.059528 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:05.059494    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5ff575bb75-h68jp" event={"ID":"6cd1c489-6360-4c70-8492-dba66b5bc327","Type":"ContainerStarted","Data":"906d8733b016d121fafd5011fd8455eb56bfadbbcaac386fa92d89b4729bfc44"}
Jul 24 17:34:05.059976 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:05.059550    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5ff575bb75-h68jp" event={"ID":"6cd1c489-6360-4c70-8492-dba66b5bc327","Type":"ContainerStarted","Data":"c09b18963ad2e842f83c5719dd3bcb65cc9ababc84b1e27ee75c17ccebd199d1"}
Jul 24 17:34:05.059976 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:05.059577    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5ff575bb75-h68jp"
Jul 24 17:34:05.078030 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:05.077943    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5ff575bb75-h68jp" podStartSLOduration=1.494925216 podStartE2EDuration="2.077927426s" totalImagesPullingTime="583.00221ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:34:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:34:05.076130538 +0000 UTC m=+1007.803286753" watchObservedRunningTime="2026-07-24 17:34:05.077927426 +0000 UTC m=+1007.805083639"
Jul 24 17:34:16.067928 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:16.067897    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5ff575bb75-h68jp"
Jul 24 17:34:20.728838 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:20.728805    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5ff575bb75-h68jp"]
Jul 24 17:34:20.729305 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:20.728993    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5ff575bb75-h68jp" podUID="6cd1c489-6360-4c70-8492-dba66b5bc327" containerName="manager" containerID="cri-o://906d8733b016d121fafd5011fd8455eb56bfadbbcaac386fa92d89b4729bfc44" gracePeriod=30
Jul 24 17:34:20.981425 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:20.981366    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5ff575bb75-h68jp"
Jul 24 17:34:21.102617 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:21.102578    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94kz6\" (UniqueName: \"kubernetes.io/projected/6cd1c489-6360-4c70-8492-dba66b5bc327-kube-api-access-94kz6\") pod \"6cd1c489-6360-4c70-8492-dba66b5bc327\" (UID: \"6cd1c489-6360-4c70-8492-dba66b5bc327\") "
Jul 24 17:34:21.102792 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:21.102637    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6cd1c489-6360-4c70-8492-dba66b5bc327-webhook-cert\") pod \"6cd1c489-6360-4c70-8492-dba66b5bc327\" (UID: \"6cd1c489-6360-4c70-8492-dba66b5bc327\") "
Jul 24 17:34:21.104792 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:21.104764    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cd1c489-6360-4c70-8492-dba66b5bc327-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6cd1c489-6360-4c70-8492-dba66b5bc327" (UID: "6cd1c489-6360-4c70-8492-dba66b5bc327"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:34:21.104792 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:21.104771    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cd1c489-6360-4c70-8492-dba66b5bc327-kube-api-access-94kz6" (OuterVolumeSpecName: "kube-api-access-94kz6") pod "6cd1c489-6360-4c70-8492-dba66b5bc327" (UID: "6cd1c489-6360-4c70-8492-dba66b5bc327"). InnerVolumeSpecName "kube-api-access-94kz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:34:21.140515 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:21.140480    2562 generic.go:356] "Generic (PLEG): container finished" podID="6cd1c489-6360-4c70-8492-dba66b5bc327" containerID="906d8733b016d121fafd5011fd8455eb56bfadbbcaac386fa92d89b4729bfc44" exitCode=0
Jul 24 17:34:21.140645 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:21.140531    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5ff575bb75-h68jp"
Jul 24 17:34:21.140645 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:21.140562    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5ff575bb75-h68jp" event={"ID":"6cd1c489-6360-4c70-8492-dba66b5bc327","Type":"ContainerDied","Data":"906d8733b016d121fafd5011fd8455eb56bfadbbcaac386fa92d89b4729bfc44"}
Jul 24 17:34:21.140645 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:21.140621    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5ff575bb75-h68jp" event={"ID":"6cd1c489-6360-4c70-8492-dba66b5bc327","Type":"ContainerDied","Data":"c09b18963ad2e842f83c5719dd3bcb65cc9ababc84b1e27ee75c17ccebd199d1"}
Jul 24 17:34:21.140781 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:21.140645    2562 scope.go:117] "RemoveContainer" containerID="906d8733b016d121fafd5011fd8455eb56bfadbbcaac386fa92d89b4729bfc44"
Jul 24 17:34:21.149626 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:21.149605    2562 scope.go:117] "RemoveContainer" containerID="906d8733b016d121fafd5011fd8455eb56bfadbbcaac386fa92d89b4729bfc44"
Jul 24 17:34:21.149898 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:34:21.149878    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"906d8733b016d121fafd5011fd8455eb56bfadbbcaac386fa92d89b4729bfc44\": container with ID starting with 906d8733b016d121fafd5011fd8455eb56bfadbbcaac386fa92d89b4729bfc44 not found: ID does not exist" containerID="906d8733b016d121fafd5011fd8455eb56bfadbbcaac386fa92d89b4729bfc44"
Jul 24 17:34:21.149946 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:21.149907    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"906d8733b016d121fafd5011fd8455eb56bfadbbcaac386fa92d89b4729bfc44"} err="failed to get container status \"906d8733b016d121fafd5011fd8455eb56bfadbbcaac386fa92d89b4729bfc44\": rpc error: code = NotFound desc = could not find container \"906d8733b016d121fafd5011fd8455eb56bfadbbcaac386fa92d89b4729bfc44\": container with ID starting with 906d8733b016d121fafd5011fd8455eb56bfadbbcaac386fa92d89b4729bfc44 not found: ID does not exist"
Jul 24 17:34:21.162923 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:21.162890    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5ff575bb75-h68jp"]
Jul 24 17:34:21.167038 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:21.167017    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5ff575bb75-h68jp"]
Jul 24 17:34:21.204371 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:21.204324    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94kz6\" (UniqueName: \"kubernetes.io/projected/6cd1c489-6360-4c70-8492-dba66b5bc327-kube-api-access-94kz6\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:34:21.204371 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:21.204366    2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6cd1c489-6360-4c70-8492-dba66b5bc327-webhook-cert\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:34:21.824629 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:21.824593    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cd1c489-6360-4c70-8492-dba66b5bc327" path="/var/lib/kubelet/pods/6cd1c489-6360-4c70-8492-dba66b5bc327/volumes"
Jul 24 17:34:21.986740 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:21.986709    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5ff575bb75-g5kzh"]
Jul 24 17:34:21.987114 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:21.987101    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cd1c489-6360-4c70-8492-dba66b5bc327" containerName="manager"
Jul 24 17:34:21.987181 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:21.987116    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cd1c489-6360-4c70-8492-dba66b5bc327" containerName="manager"
Jul 24 17:34:21.987215 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:21.987204    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cd1c489-6360-4c70-8492-dba66b5bc327" containerName="manager"
Jul 24 17:34:21.991736 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:21.991714    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5ff575bb75-g5kzh"
Jul 24 17:34:21.994243 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:21.994224    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 24 17:34:21.995081 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:21.995054    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 24 17:34:21.995200 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:21.995059    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-46q8n\""
Jul 24 17:34:22.002410 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:22.002385    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5ff575bb75-g5kzh"]
Jul 24 17:34:22.113606 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:22.113566    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnt97\" (UniqueName: \"kubernetes.io/projected/f1cf923e-bc12-4e8e-aaf5-0b0e78243975-kube-api-access-pnt97\") pod \"maas-controller-5ff575bb75-g5kzh\" (UID: \"f1cf923e-bc12-4e8e-aaf5-0b0e78243975\") " pod="opendatahub/maas-controller-5ff575bb75-g5kzh"
Jul 24 17:34:22.113790 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:22.113634    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f1cf923e-bc12-4e8e-aaf5-0b0e78243975-webhook-cert\") pod \"maas-controller-5ff575bb75-g5kzh\" (UID: \"f1cf923e-bc12-4e8e-aaf5-0b0e78243975\") " pod="opendatahub/maas-controller-5ff575bb75-g5kzh"
Jul 24 17:34:22.215249 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:22.215212    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnt97\" (UniqueName: \"kubernetes.io/projected/f1cf923e-bc12-4e8e-aaf5-0b0e78243975-kube-api-access-pnt97\") pod \"maas-controller-5ff575bb75-g5kzh\" (UID: \"f1cf923e-bc12-4e8e-aaf5-0b0e78243975\") " pod="opendatahub/maas-controller-5ff575bb75-g5kzh"
Jul 24 17:34:22.215437 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:22.215285    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f1cf923e-bc12-4e8e-aaf5-0b0e78243975-webhook-cert\") pod \"maas-controller-5ff575bb75-g5kzh\" (UID: \"f1cf923e-bc12-4e8e-aaf5-0b0e78243975\") " pod="opendatahub/maas-controller-5ff575bb75-g5kzh"
Jul 24 17:34:22.217810 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:22.217786    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f1cf923e-bc12-4e8e-aaf5-0b0e78243975-webhook-cert\") pod \"maas-controller-5ff575bb75-g5kzh\" (UID: \"f1cf923e-bc12-4e8e-aaf5-0b0e78243975\") " pod="opendatahub/maas-controller-5ff575bb75-g5kzh"
Jul 24 17:34:22.226858 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:22.226830    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnt97\" (UniqueName: \"kubernetes.io/projected/f1cf923e-bc12-4e8e-aaf5-0b0e78243975-kube-api-access-pnt97\") pod \"maas-controller-5ff575bb75-g5kzh\" (UID: \"f1cf923e-bc12-4e8e-aaf5-0b0e78243975\") " pod="opendatahub/maas-controller-5ff575bb75-g5kzh"
Jul 24 17:34:22.303031 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:22.302988    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5ff575bb75-g5kzh"
Jul 24 17:34:22.442454 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:22.442425    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5ff575bb75-g5kzh"]
Jul 24 17:34:22.444579 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:34:22.444549    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1cf923e_bc12_4e8e_aaf5_0b0e78243975.slice/crio-a015369cad2a5c018db5cadbfc4840e2fb7f0f71a6c3e782e6b1a00fb4043c1e WatchSource:0}: Error finding container a015369cad2a5c018db5cadbfc4840e2fb7f0f71a6c3e782e6b1a00fb4043c1e: Status 404 returned error can't find the container with id a015369cad2a5c018db5cadbfc4840e2fb7f0f71a6c3e782e6b1a00fb4043c1e
Jul 24 17:34:23.154674 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:23.154639    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5ff575bb75-g5kzh" event={"ID":"f1cf923e-bc12-4e8e-aaf5-0b0e78243975","Type":"ContainerStarted","Data":"873c1abd61331d088e14bfe0bcbd134e5ec8beffb68a86e14d23fa37bc89ce0d"}
Jul 24 17:34:23.155171 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:23.154699    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5ff575bb75-g5kzh" event={"ID":"f1cf923e-bc12-4e8e-aaf5-0b0e78243975","Type":"ContainerStarted","Data":"a015369cad2a5c018db5cadbfc4840e2fb7f0f71a6c3e782e6b1a00fb4043c1e"}
Jul 24 17:34:23.155171 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:23.154739    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5ff575bb75-g5kzh"
Jul 24 17:34:23.172867 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:23.172814    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5ff575bb75-g5kzh" podStartSLOduration=1.857409596 podStartE2EDuration="2.172797099s" totalImagesPullingTime="315.387503ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:34:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:34:23.171899037 +0000 UTC m=+1025.899055251" watchObservedRunningTime="2026-07-24 17:34:23.172797099 +0000 UTC m=+1025.899953313"
Jul 24 17:34:34.161705 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:34:34.161669    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5ff575bb75-g5kzh"
Jul 24 17:39:15.148808 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:39:15.148774    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fppqv"]
Jul 24 17:39:15.151056 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:39:15.148966    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fppqv" podUID="aa3a665c-675d-419d-8577-94075f56e37e" containerName="manager" containerID="cri-o://c2a53ace6d06d6abeaa2211c2f396fd03ab70c492bf1d2fcad20955caf6cbe8a" gracePeriod=10
Jul 24 17:39:18.509401 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:39:18.509377    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fppqv"
Jul 24 17:39:18.583541 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:39:18.583518    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wk9b2\" (UniqueName: \"kubernetes.io/projected/aa3a665c-675d-419d-8577-94075f56e37e-kube-api-access-wk9b2\") pod \"aa3a665c-675d-419d-8577-94075f56e37e\" (UID: \"aa3a665c-675d-419d-8577-94075f56e37e\") "
Jul 24 17:39:18.583709 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:39:18.583566    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa3a665c-675d-419d-8577-94075f56e37e-extensions-socket-volume\") pod \"aa3a665c-675d-419d-8577-94075f56e37e\" (UID: \"aa3a665c-675d-419d-8577-94075f56e37e\") "
Jul 24 17:39:18.583992 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:39:18.583968    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa3a665c-675d-419d-8577-94075f56e37e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "aa3a665c-675d-419d-8577-94075f56e37e" (UID: "aa3a665c-675d-419d-8577-94075f56e37e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:39:18.585660 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:39:18.585635    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa3a665c-675d-419d-8577-94075f56e37e-kube-api-access-wk9b2" (OuterVolumeSpecName: "kube-api-access-wk9b2") pod "aa3a665c-675d-419d-8577-94075f56e37e" (UID: "aa3a665c-675d-419d-8577-94075f56e37e"). InnerVolumeSpecName "kube-api-access-wk9b2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:39:18.618045 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:39:18.618011    2562 generic.go:356] "Generic (PLEG): container finished" podID="aa3a665c-675d-419d-8577-94075f56e37e" containerID="c2a53ace6d06d6abeaa2211c2f396fd03ab70c492bf1d2fcad20955caf6cbe8a" exitCode=0
Jul 24 17:39:18.618238 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:39:18.618054    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fppqv"
Jul 24 17:39:18.618238 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:39:18.618076    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fppqv" event={"ID":"aa3a665c-675d-419d-8577-94075f56e37e","Type":"ContainerDied","Data":"c2a53ace6d06d6abeaa2211c2f396fd03ab70c492bf1d2fcad20955caf6cbe8a"}
Jul 24 17:39:18.618238 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:39:18.618121    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fppqv" event={"ID":"aa3a665c-675d-419d-8577-94075f56e37e","Type":"ContainerDied","Data":"064f92839805d94b05b3208c0d8aa3021accba4aed1833228ef72a9e8d9ab58f"}
Jul 24 17:39:18.618238 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:39:18.618156    2562 scope.go:117] "RemoveContainer" containerID="c2a53ace6d06d6abeaa2211c2f396fd03ab70c492bf1d2fcad20955caf6cbe8a"
Jul 24 17:39:18.628579 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:39:18.628560    2562 scope.go:117] "RemoveContainer" containerID="c2a53ace6d06d6abeaa2211c2f396fd03ab70c492bf1d2fcad20955caf6cbe8a"
Jul 24 17:39:18.628857 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:39:18.628838    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2a53ace6d06d6abeaa2211c2f396fd03ab70c492bf1d2fcad20955caf6cbe8a\": container with ID starting with c2a53ace6d06d6abeaa2211c2f396fd03ab70c492bf1d2fcad20955caf6cbe8a not found: ID does not exist" containerID="c2a53ace6d06d6abeaa2211c2f396fd03ab70c492bf1d2fcad20955caf6cbe8a"
Jul 24 17:39:18.628895 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:39:18.628866    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2a53ace6d06d6abeaa2211c2f396fd03ab70c492bf1d2fcad20955caf6cbe8a"} err="failed to get container status \"c2a53ace6d06d6abeaa2211c2f396fd03ab70c492bf1d2fcad20955caf6cbe8a\": rpc error: code = NotFound desc = could not find container \"c2a53ace6d06d6abeaa2211c2f396fd03ab70c492bf1d2fcad20955caf6cbe8a\": container with ID starting with c2a53ace6d06d6abeaa2211c2f396fd03ab70c492bf1d2fcad20955caf6cbe8a not found: ID does not exist"
Jul 24 17:39:18.656093 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:39:18.656064    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fppqv"]
Jul 24 17:39:18.672373 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:39:18.668916    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fppqv"]
Jul 24 17:39:18.684454 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:39:18.684422    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wk9b2\" (UniqueName: \"kubernetes.io/projected/aa3a665c-675d-419d-8577-94075f56e37e-kube-api-access-wk9b2\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:39:18.684454 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:39:18.684449    2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa3a665c-675d-419d-8577-94075f56e37e-extensions-socket-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:39:19.824739 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:39:19.824704    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa3a665c-675d-419d-8577-94075f56e37e" path="/var/lib/kubelet/pods/aa3a665c-675d-419d-8577-94075f56e37e/volumes"
Jul 24 17:40:16.385309 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:40:16.385272    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45xlb"]
Jul 24 17:40:16.385919 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:40:16.385906    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa3a665c-675d-419d-8577-94075f56e37e" containerName="manager"
Jul 24 17:40:16.385996 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:40:16.385925    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa3a665c-675d-419d-8577-94075f56e37e" containerName="manager"
Jul 24 17:40:16.386048 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:40:16.386031    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa3a665c-675d-419d-8577-94075f56e37e" containerName="manager"
Jul 24 17:40:16.389405 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:40:16.389383    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45xlb"
Jul 24 17:40:16.392195 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:40:16.392169    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 24 17:40:16.392350 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:40:16.392170    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 24 17:40:16.392350 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:40:16.392168    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-csc7m\""
Jul 24 17:40:16.401665 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:40:16.401638    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45xlb"]
Jul 24 17:40:16.494903 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:40:16.494851    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsq6p\" (UniqueName: \"kubernetes.io/projected/b7cfe6b0-acc6-47c6-8da0-a2d0463dee29-kube-api-access-gsq6p\") pod \"kuadrant-operator-controller-manager-86878c769f-45xlb\" (UID: \"b7cfe6b0-acc6-47c6-8da0-a2d0463dee29\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45xlb"
Jul 24 17:40:16.495088 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:40:16.494931    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7cfe6b0-acc6-47c6-8da0-a2d0463dee29-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-45xlb\" (UID: \"b7cfe6b0-acc6-47c6-8da0-a2d0463dee29\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45xlb"
Jul 24 17:40:16.596294 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:40:16.596255    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsq6p\" (UniqueName: \"kubernetes.io/projected/b7cfe6b0-acc6-47c6-8da0-a2d0463dee29-kube-api-access-gsq6p\") pod \"kuadrant-operator-controller-manager-86878c769f-45xlb\" (UID: \"b7cfe6b0-acc6-47c6-8da0-a2d0463dee29\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45xlb"
Jul 24 17:40:16.596495 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:40:16.596328    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7cfe6b0-acc6-47c6-8da0-a2d0463dee29-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-45xlb\" (UID: \"b7cfe6b0-acc6-47c6-8da0-a2d0463dee29\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45xlb"
Jul 24 17:40:16.596716 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:40:16.596695    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7cfe6b0-acc6-47c6-8da0-a2d0463dee29-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-45xlb\" (UID: \"b7cfe6b0-acc6-47c6-8da0-a2d0463dee29\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45xlb"
Jul 24 17:40:16.606599 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:40:16.606572    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsq6p\" (UniqueName: \"kubernetes.io/projected/b7cfe6b0-acc6-47c6-8da0-a2d0463dee29-kube-api-access-gsq6p\") pod \"kuadrant-operator-controller-manager-86878c769f-45xlb\" (UID: \"b7cfe6b0-acc6-47c6-8da0-a2d0463dee29\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45xlb"
Jul 24 17:40:16.700235 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:40:16.700161    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45xlb"
Jul 24 17:40:16.841007 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:40:16.840980    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45xlb"]
Jul 24 17:40:16.843313 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:40:16.843287    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7cfe6b0_acc6_47c6_8da0_a2d0463dee29.slice/crio-ee226e32429ca4efeaf74c3d6d18812c30c27029e9f6949ec5102edae9389e59 WatchSource:0}: Error finding container ee226e32429ca4efeaf74c3d6d18812c30c27029e9f6949ec5102edae9389e59: Status 404 returned error can't find the container with id ee226e32429ca4efeaf74c3d6d18812c30c27029e9f6949ec5102edae9389e59
Jul 24 17:40:16.892903 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:40:16.892874    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45xlb" event={"ID":"b7cfe6b0-acc6-47c6-8da0-a2d0463dee29","Type":"ContainerStarted","Data":"ee226e32429ca4efeaf74c3d6d18812c30c27029e9f6949ec5102edae9389e59"}
Jul 24 17:40:17.899342 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:40:17.899300    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45xlb" event={"ID":"b7cfe6b0-acc6-47c6-8da0-a2d0463dee29","Type":"ContainerStarted","Data":"1bee44a0447e6c61c0306104a7f1f7cda4b234a40219158b731db105a45f5614"}
Jul 24 17:40:17.899703 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:40:17.899397    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45xlb"
Jul 24 17:40:17.931541 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:40:17.931490    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45xlb" podStartSLOduration=1.9314751430000001 podStartE2EDuration="1.931475143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:40:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:17.928685416 +0000 UTC m=+1380.655841626" watchObservedRunningTime="2026-07-24 17:40:17.931475143 +0000 UTC m=+1380.658631356"
Jul 24 17:40:28.905462 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:40:28.905428    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45xlb"
Jul 24 17:47:23.602084 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:23.602053    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp"]
Jul 24 17:47:23.605386 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:23.605358    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp"
Jul 24 17:47:23.607991 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:23.607959    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-be21bfbc-serving-cert\""
Jul 24 17:47:23.616894 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:23.616870    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp"]
Jul 24 17:47:23.694937 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:23.694906    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d26de71-fab8-4eda-acda-b4ede65a5a3d-maas-api-tls\") pod \"maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp\" (UID: \"1d26de71-fab8-4eda-acda-b4ede65a5a3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp"
Jul 24 17:47:23.695100 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:23.694958    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d26de71-fab8-4eda-acda-b4ede65a5a3d-openshift-service-ca\") pod \"maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp\" (UID: \"1d26de71-fab8-4eda-acda-b4ede65a5a3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp"
Jul 24 17:47:23.695100 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:23.695063    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsc44\" (UniqueName: \"kubernetes.io/projected/1d26de71-fab8-4eda-acda-b4ede65a5a3d-kube-api-access-fsc44\") pod \"maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp\" (UID: \"1d26de71-fab8-4eda-acda-b4ede65a5a3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp"
Jul 24 17:47:23.795825 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:23.795779    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d26de71-fab8-4eda-acda-b4ede65a5a3d-maas-api-tls\") pod \"maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp\" (UID: \"1d26de71-fab8-4eda-acda-b4ede65a5a3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp"
Jul 24 17:47:23.796005 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:23.795848    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d26de71-fab8-4eda-acda-b4ede65a5a3d-openshift-service-ca\") pod \"maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp\" (UID: \"1d26de71-fab8-4eda-acda-b4ede65a5a3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp"
Jul 24 17:47:23.796005 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:23.795882    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsc44\" (UniqueName: \"kubernetes.io/projected/1d26de71-fab8-4eda-acda-b4ede65a5a3d-kube-api-access-fsc44\") pod \"maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp\" (UID: \"1d26de71-fab8-4eda-acda-b4ede65a5a3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp"
Jul 24 17:47:23.796519 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:23.796498    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d26de71-fab8-4eda-acda-b4ede65a5a3d-openshift-service-ca\") pod \"maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp\" (UID: \"1d26de71-fab8-4eda-acda-b4ede65a5a3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp"
Jul 24 17:47:23.798274 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:23.798251    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d26de71-fab8-4eda-acda-b4ede65a5a3d-maas-api-tls\") pod \"maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp\" (UID: \"1d26de71-fab8-4eda-acda-b4ede65a5a3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp"
Jul 24 17:47:23.804526 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:23.804501    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsc44\" (UniqueName: \"kubernetes.io/projected/1d26de71-fab8-4eda-acda-b4ede65a5a3d-kube-api-access-fsc44\") pod \"maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp\" (UID: \"1d26de71-fab8-4eda-acda-b4ede65a5a3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp"
Jul 24 17:47:23.917628 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:23.917557    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp"
Jul 24 17:47:24.055465 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:24.055339    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp"]
Jul 24 17:47:24.057687 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:47:24.057657    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d26de71_fab8_4eda_acda_b4ede65a5a3d.slice/crio-459e915956b5a2bd1997233a2b920a7e2a61f9354855f62caa42140128137f13 WatchSource:0}: Error finding container 459e915956b5a2bd1997233a2b920a7e2a61f9354855f62caa42140128137f13: Status 404 returned error can't find the container with id 459e915956b5a2bd1997233a2b920a7e2a61f9354855f62caa42140128137f13
Jul 24 17:47:24.058999 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:24.058981    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 17:47:24.345647 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:24.345614    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p"]
Jul 24 17:47:24.350285 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:24.350250    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p"
Jul 24 17:47:24.353956 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:24.353932    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-be21bfbc\""
Jul 24 17:47:24.354039 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:24.354008    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-be21bfbc-dockercfg-v7rmq\""
Jul 24 17:47:24.357750 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:24.357726    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p"]
Jul 24 17:47:24.380009 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:24.379984    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7"]
Jul 24 17:47:24.384122 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:24.384101    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7"
Jul 24 17:47:24.395198 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:24.395177    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7"]
Jul 24 17:47:24.400970 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:24.400947    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a736293a-da10-46b8-bb72-98faa41b6e6b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p\" (UID: \"a736293a-da10-46b8-bb72-98faa41b6e6b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p"
Jul 24 17:47:24.401071 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:24.400987    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qlkp\" (UniqueName: \"kubernetes.io/projected/a736293a-da10-46b8-bb72-98faa41b6e6b-kube-api-access-8qlkp\") pod \"payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p\" (UID: \"a736293a-da10-46b8-bb72-98faa41b6e6b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p"
Jul 24 17:47:24.501889 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:24.501858    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcktb\" (UniqueName: \"kubernetes.io/projected/b5f31834-2bf6-4354-8808-afb5a6b0be4c-kube-api-access-lcktb\") pod \"payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7\" (UID: \"b5f31834-2bf6-4354-8808-afb5a6b0be4c\") " pod="openshift-ingress/payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7"
Jul 24 17:47:24.502046 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:24.501922    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5f31834-2bf6-4354-8808-afb5a6b0be4c-plugins-config-volume\") pod \"payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7\" (UID: \"b5f31834-2bf6-4354-8808-afb5a6b0be4c\") " pod="openshift-ingress/payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7"
Jul 24 17:47:24.502046 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:24.501999    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a736293a-da10-46b8-bb72-98faa41b6e6b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p\" (UID: \"a736293a-da10-46b8-bb72-98faa41b6e6b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p"
Jul 24 17:47:24.502046 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:24.502033    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qlkp\" (UniqueName: \"kubernetes.io/projected/a736293a-da10-46b8-bb72-98faa41b6e6b-kube-api-access-8qlkp\") pod \"payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p\" (UID: \"a736293a-da10-46b8-bb72-98faa41b6e6b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p"
Jul 24 17:47:24.502699 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:24.502680    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a736293a-da10-46b8-bb72-98faa41b6e6b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p\" (UID: \"a736293a-da10-46b8-bb72-98faa41b6e6b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p"
Jul 24 17:47:24.510674 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:24.510644    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qlkp\" (UniqueName: \"kubernetes.io/projected/a736293a-da10-46b8-bb72-98faa41b6e6b-kube-api-access-8qlkp\") pod \"payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p\" (UID: \"a736293a-da10-46b8-bb72-98faa41b6e6b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p"
Jul 24 17:47:24.603526 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:24.603500    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcktb\" (UniqueName: \"kubernetes.io/projected/b5f31834-2bf6-4354-8808-afb5a6b0be4c-kube-api-access-lcktb\") pod \"payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7\" (UID: \"b5f31834-2bf6-4354-8808-afb5a6b0be4c\") " pod="openshift-ingress/payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7"
Jul 24 17:47:24.603894 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:24.603582    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5f31834-2bf6-4354-8808-afb5a6b0be4c-plugins-config-volume\") pod \"payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7\" (UID: \"b5f31834-2bf6-4354-8808-afb5a6b0be4c\") " pod="openshift-ingress/payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7"
Jul 24 17:47:24.604531 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:24.604469    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5f31834-2bf6-4354-8808-afb5a6b0be4c-plugins-config-volume\") pod \"payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7\" (UID: \"b5f31834-2bf6-4354-8808-afb5a6b0be4c\") " pod="openshift-ingress/payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7"
Jul 24 17:47:24.612989 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:24.612957    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcktb\" (UniqueName: \"kubernetes.io/projected/b5f31834-2bf6-4354-8808-afb5a6b0be4c-kube-api-access-lcktb\") pod \"payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7\" (UID: \"b5f31834-2bf6-4354-8808-afb5a6b0be4c\") " pod="openshift-ingress/payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7"
Jul 24 17:47:24.662656 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:24.662631    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p"
Jul 24 17:47:24.694707 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:24.694679    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7"
Jul 24 17:47:24.807244 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:24.806197    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p"]
Jul 24 17:47:24.810488 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:47:24.810226    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda736293a_da10_46b8_bb72_98faa41b6e6b.slice/crio-f87178ccfd1a7befbb4ab4e237a5fcdae7a0fdec012cdb79e95bc201b8f2654a WatchSource:0}: Error finding container f87178ccfd1a7befbb4ab4e237a5fcdae7a0fdec012cdb79e95bc201b8f2654a: Status 404 returned error can't find the container with id f87178ccfd1a7befbb4ab4e237a5fcdae7a0fdec012cdb79e95bc201b8f2654a
Jul 24 17:47:24.846534 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:24.846508    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7"]
Jul 24 17:47:24.857386 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:47:24.857357    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5f31834_2bf6_4354_8808_afb5a6b0be4c.slice/crio-1b585d7dbfcba307ee7e72aa14d34f5cea5f2a3529982ef103e8cb05b7c652f1 WatchSource:0}: Error finding container 1b585d7dbfcba307ee7e72aa14d34f5cea5f2a3529982ef103e8cb05b7c652f1: Status 404 returned error can't find the container with id 1b585d7dbfcba307ee7e72aa14d34f5cea5f2a3529982ef103e8cb05b7c652f1
Jul 24 17:47:25.026631 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:25.026591    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp" event={"ID":"1d26de71-fab8-4eda-acda-b4ede65a5a3d","Type":"ContainerStarted","Data":"29c391b37f355490624d6d896b68240656a86495917ee667ad583535aad7a32a"}
Jul 24 17:47:25.026818 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:25.026657    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp" event={"ID":"1d26de71-fab8-4eda-acda-b4ede65a5a3d","Type":"ContainerStarted","Data":"459e915956b5a2bd1997233a2b920a7e2a61f9354855f62caa42140128137f13"}
Jul 24 17:47:25.026915 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:25.026821    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp"
Jul 24 17:47:25.028587 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:25.028558    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p" event={"ID":"a736293a-da10-46b8-bb72-98faa41b6e6b","Type":"ContainerStarted","Data":"d3843589feb73fd8a6efed4e62fad887aa34919b6bf11dfce1e0e16be12115a5"}
Jul 24 17:47:25.028710 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:25.028600    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p" event={"ID":"a736293a-da10-46b8-bb72-98faa41b6e6b","Type":"ContainerStarted","Data":"f87178ccfd1a7befbb4ab4e237a5fcdae7a0fdec012cdb79e95bc201b8f2654a"}
Jul 24 17:47:25.028710 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:25.028656    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p"
Jul 24 17:47:25.030372 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:25.030348    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7" event={"ID":"b5f31834-2bf6-4354-8808-afb5a6b0be4c","Type":"ContainerStarted","Data":"19e06f15633b11aa1b842a287987dce4f1182c244491ac7dd965b7b0a9e8624e"}
Jul 24 17:47:25.030471 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:25.030395    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7" event={"ID":"b5f31834-2bf6-4354-8808-afb5a6b0be4c","Type":"ContainerStarted","Data":"1b585d7dbfcba307ee7e72aa14d34f5cea5f2a3529982ef103e8cb05b7c652f1"}
Jul 24 17:47:25.030471 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:25.030424    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7"
Jul 24 17:47:25.045778 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:25.045739    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp" podStartSLOduration=1.5590673910000001 podStartE2EDuration="2.045726634s" totalImagesPullingTime="486.659243ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:47:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:47:25.043838604 +0000 UTC m=+1807.770994821" watchObservedRunningTime="2026-07-24 17:47:25.045726634 +0000 UTC m=+1807.772882847"
Jul 24 17:47:25.062703 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:25.062662    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7" podStartSLOduration=1.062639429 podStartE2EDuration="1.062639429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:47:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:47:25.061813466 +0000 UTC m=+1807.788969682" watchObservedRunningTime="2026-07-24 17:47:25.062639429 +0000 UTC m=+1807.789795620"
Jul 24 17:47:25.078115 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:25.078072    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p" podStartSLOduration=1.078061052 podStartE2EDuration="1.078061052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:47:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:47:25.076234677 +0000 UTC m=+1807.803390916" watchObservedRunningTime="2026-07-24 17:47:25.078061052 +0000 UTC m=+1807.805217265"
Jul 24 17:47:31.041573 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:31.041543    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp"
Jul 24 17:47:36.038114 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:36.038083    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7"
Jul 24 17:47:36.038590 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:36.038275    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p"
Jul 24 17:47:59.558916 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:59.558876    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp"]
Jul 24 17:47:59.559497 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:59.559128    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp" podUID="1d26de71-fab8-4eda-acda-b4ede65a5a3d" containerName="maas-api" containerID="cri-o://29c391b37f355490624d6d896b68240656a86495917ee667ad583535aad7a32a" gracePeriod=30
Jul 24 17:47:59.617772 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:59.617733    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7"]
Jul 24 17:47:59.618031 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:59.617983    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7" podUID="b5f31834-2bf6-4354-8808-afb5a6b0be4c" containerName="payload-processing" containerID="cri-o://19e06f15633b11aa1b842a287987dce4f1182c244491ac7dd965b7b0a9e8624e" gracePeriod=30
Jul 24 17:47:59.626970 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:59.626944    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p"]
Jul 24 17:47:59.627194 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:59.627172    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p" podUID="a736293a-da10-46b8-bb72-98faa41b6e6b" containerName="payload-pre-processing" containerID="cri-o://d3843589feb73fd8a6efed4e62fad887aa34919b6bf11dfce1e0e16be12115a5" gracePeriod=30
Jul 24 17:47:59.910696 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:59.910669    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp"
Jul 24 17:47:59.912845 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:59.912824    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p"
Jul 24 17:47:59.936937 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:47:59.936855    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7"
Jul 24 17:48:00.033913 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.033875    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a736293a-da10-46b8-bb72-98faa41b6e6b-plugins-config-volume\") pod \"a736293a-da10-46b8-bb72-98faa41b6e6b\" (UID: \"a736293a-da10-46b8-bb72-98faa41b6e6b\") "
Jul 24 17:48:00.034093 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.033941    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lcktb\" (UniqueName: \"kubernetes.io/projected/b5f31834-2bf6-4354-8808-afb5a6b0be4c-kube-api-access-lcktb\") pod \"b5f31834-2bf6-4354-8808-afb5a6b0be4c\" (UID: \"b5f31834-2bf6-4354-8808-afb5a6b0be4c\") "
Jul 24 17:48:00.034093 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.033989    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d26de71-fab8-4eda-acda-b4ede65a5a3d-maas-api-tls\") pod \"1d26de71-fab8-4eda-acda-b4ede65a5a3d\" (UID: \"1d26de71-fab8-4eda-acda-b4ede65a5a3d\") "
Jul 24 17:48:00.034093 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.034024    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qlkp\" (UniqueName: \"kubernetes.io/projected/a736293a-da10-46b8-bb72-98faa41b6e6b-kube-api-access-8qlkp\") pod \"a736293a-da10-46b8-bb72-98faa41b6e6b\" (UID: \"a736293a-da10-46b8-bb72-98faa41b6e6b\") "
Jul 24 17:48:00.034093 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.034039    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5f31834-2bf6-4354-8808-afb5a6b0be4c-plugins-config-volume\") pod \"b5f31834-2bf6-4354-8808-afb5a6b0be4c\" (UID: \"b5f31834-2bf6-4354-8808-afb5a6b0be4c\") "
Jul 24 17:48:00.034093 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.034086    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fsc44\" (UniqueName: \"kubernetes.io/projected/1d26de71-fab8-4eda-acda-b4ede65a5a3d-kube-api-access-fsc44\") pod \"1d26de71-fab8-4eda-acda-b4ede65a5a3d\" (UID: \"1d26de71-fab8-4eda-acda-b4ede65a5a3d\") "
Jul 24 17:48:00.034419 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.034133    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d26de71-fab8-4eda-acda-b4ede65a5a3d-openshift-service-ca\") pod \"1d26de71-fab8-4eda-acda-b4ede65a5a3d\" (UID: \"1d26de71-fab8-4eda-acda-b4ede65a5a3d\") "
Jul 24 17:48:00.034512 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.034486    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5f31834-2bf6-4354-8808-afb5a6b0be4c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b5f31834-2bf6-4354-8808-afb5a6b0be4c" (UID: "b5f31834-2bf6-4354-8808-afb5a6b0be4c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:48:00.034580 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.034549    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d26de71-fab8-4eda-acda-b4ede65a5a3d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1d26de71-fab8-4eda-acda-b4ede65a5a3d" (UID: "1d26de71-fab8-4eda-acda-b4ede65a5a3d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:48:00.034766 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.034745    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a736293a-da10-46b8-bb72-98faa41b6e6b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a736293a-da10-46b8-bb72-98faa41b6e6b" (UID: "a736293a-da10-46b8-bb72-98faa41b6e6b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:48:00.036651 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.036630    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5f31834-2bf6-4354-8808-afb5a6b0be4c-kube-api-access-lcktb" (OuterVolumeSpecName: "kube-api-access-lcktb") pod "b5f31834-2bf6-4354-8808-afb5a6b0be4c" (UID: "b5f31834-2bf6-4354-8808-afb5a6b0be4c"). InnerVolumeSpecName "kube-api-access-lcktb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:48:00.036756 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.036729    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d26de71-fab8-4eda-acda-b4ede65a5a3d-kube-api-access-fsc44" (OuterVolumeSpecName: "kube-api-access-fsc44") pod "1d26de71-fab8-4eda-acda-b4ede65a5a3d" (UID: "1d26de71-fab8-4eda-acda-b4ede65a5a3d"). InnerVolumeSpecName "kube-api-access-fsc44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:48:00.036756 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.036738    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d26de71-fab8-4eda-acda-b4ede65a5a3d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1d26de71-fab8-4eda-acda-b4ede65a5a3d" (UID: "1d26de71-fab8-4eda-acda-b4ede65a5a3d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:48:00.036934 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.036915    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a736293a-da10-46b8-bb72-98faa41b6e6b-kube-api-access-8qlkp" (OuterVolumeSpecName: "kube-api-access-8qlkp") pod "a736293a-da10-46b8-bb72-98faa41b6e6b" (UID: "a736293a-da10-46b8-bb72-98faa41b6e6b"). InnerVolumeSpecName "kube-api-access-8qlkp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:48:00.135011 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.134925    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcktb\" (UniqueName: \"kubernetes.io/projected/b5f31834-2bf6-4354-8808-afb5a6b0be4c-kube-api-access-lcktb\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:48:00.135011 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.134957    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d26de71-fab8-4eda-acda-b4ede65a5a3d-maas-api-tls\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:48:00.135011 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.134971    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qlkp\" (UniqueName: \"kubernetes.io/projected/a736293a-da10-46b8-bb72-98faa41b6e6b-kube-api-access-8qlkp\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:48:00.135011 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.134984    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5f31834-2bf6-4354-8808-afb5a6b0be4c-plugins-config-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:48:00.135011 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.134996    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fsc44\" (UniqueName: \"kubernetes.io/projected/1d26de71-fab8-4eda-acda-b4ede65a5a3d-kube-api-access-fsc44\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:48:00.135011 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.135009    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d26de71-fab8-4eda-acda-b4ede65a5a3d-openshift-service-ca\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:48:00.135332 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.135021    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a736293a-da10-46b8-bb72-98faa41b6e6b-plugins-config-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:48:00.210492 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.210456    2562 generic.go:356] "Generic (PLEG): container finished" podID="1d26de71-fab8-4eda-acda-b4ede65a5a3d" containerID="29c391b37f355490624d6d896b68240656a86495917ee667ad583535aad7a32a" exitCode=0
Jul 24 17:48:00.210692 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.210527    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp"
Jul 24 17:48:00.210692 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.210536    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp" event={"ID":"1d26de71-fab8-4eda-acda-b4ede65a5a3d","Type":"ContainerDied","Data":"29c391b37f355490624d6d896b68240656a86495917ee667ad583535aad7a32a"}
Jul 24 17:48:00.210692 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.210632    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp" event={"ID":"1d26de71-fab8-4eda-acda-b4ede65a5a3d","Type":"ContainerDied","Data":"459e915956b5a2bd1997233a2b920a7e2a61f9354855f62caa42140128137f13"}
Jul 24 17:48:00.210692 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.210668    2562 scope.go:117] "RemoveContainer" containerID="29c391b37f355490624d6d896b68240656a86495917ee667ad583535aad7a32a"
Jul 24 17:48:00.212119 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.212098    2562 generic.go:356] "Generic (PLEG): container finished" podID="a736293a-da10-46b8-bb72-98faa41b6e6b" containerID="d3843589feb73fd8a6efed4e62fad887aa34919b6bf11dfce1e0e16be12115a5" exitCode=0
Jul 24 17:48:00.212249 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.212183    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p"
Jul 24 17:48:00.212249 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.212180    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p" event={"ID":"a736293a-da10-46b8-bb72-98faa41b6e6b","Type":"ContainerDied","Data":"d3843589feb73fd8a6efed4e62fad887aa34919b6bf11dfce1e0e16be12115a5"}
Jul 24 17:48:00.212348 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.212263    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p" event={"ID":"a736293a-da10-46b8-bb72-98faa41b6e6b","Type":"ContainerDied","Data":"f87178ccfd1a7befbb4ab4e237a5fcdae7a0fdec012cdb79e95bc201b8f2654a"}
Jul 24 17:48:00.213655 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.213634    2562 generic.go:356] "Generic (PLEG): container finished" podID="b5f31834-2bf6-4354-8808-afb5a6b0be4c" containerID="19e06f15633b11aa1b842a287987dce4f1182c244491ac7dd965b7b0a9e8624e" exitCode=0
Jul 24 17:48:00.213792 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.213665    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7" event={"ID":"b5f31834-2bf6-4354-8808-afb5a6b0be4c","Type":"ContainerDied","Data":"19e06f15633b11aa1b842a287987dce4f1182c244491ac7dd965b7b0a9e8624e"}
Jul 24 17:48:00.213882 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.213820    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7" event={"ID":"b5f31834-2bf6-4354-8808-afb5a6b0be4c","Type":"ContainerDied","Data":"1b585d7dbfcba307ee7e72aa14d34f5cea5f2a3529982ef103e8cb05b7c652f1"}
Jul 24 17:48:00.213882 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.213716    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7"
Jul 24 17:48:00.220775 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.220751    2562 scope.go:117] "RemoveContainer" containerID="29c391b37f355490624d6d896b68240656a86495917ee667ad583535aad7a32a"
Jul 24 17:48:00.221026 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:48:00.221006    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29c391b37f355490624d6d896b68240656a86495917ee667ad583535aad7a32a\": container with ID starting with 29c391b37f355490624d6d896b68240656a86495917ee667ad583535aad7a32a not found: ID does not exist" containerID="29c391b37f355490624d6d896b68240656a86495917ee667ad583535aad7a32a"
Jul 24 17:48:00.221109 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.221031    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29c391b37f355490624d6d896b68240656a86495917ee667ad583535aad7a32a"} err="failed to get container status \"29c391b37f355490624d6d896b68240656a86495917ee667ad583535aad7a32a\": rpc error: code = NotFound desc = could not find container \"29c391b37f355490624d6d896b68240656a86495917ee667ad583535aad7a32a\": container with ID starting with 29c391b37f355490624d6d896b68240656a86495917ee667ad583535aad7a32a not found: ID does not exist"
Jul 24 17:48:00.221109 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.221046    2562 scope.go:117] "RemoveContainer" containerID="d3843589feb73fd8a6efed4e62fad887aa34919b6bf11dfce1e0e16be12115a5"
Jul 24 17:48:00.229486 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.229471    2562 scope.go:117] "RemoveContainer" containerID="d3843589feb73fd8a6efed4e62fad887aa34919b6bf11dfce1e0e16be12115a5"
Jul 24 17:48:00.229728 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:48:00.229710    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3843589feb73fd8a6efed4e62fad887aa34919b6bf11dfce1e0e16be12115a5\": container with ID starting with d3843589feb73fd8a6efed4e62fad887aa34919b6bf11dfce1e0e16be12115a5 not found: ID does not exist" containerID="d3843589feb73fd8a6efed4e62fad887aa34919b6bf11dfce1e0e16be12115a5"
Jul 24 17:48:00.229771 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.229733    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3843589feb73fd8a6efed4e62fad887aa34919b6bf11dfce1e0e16be12115a5"} err="failed to get container status \"d3843589feb73fd8a6efed4e62fad887aa34919b6bf11dfce1e0e16be12115a5\": rpc error: code = NotFound desc = could not find container \"d3843589feb73fd8a6efed4e62fad887aa34919b6bf11dfce1e0e16be12115a5\": container with ID starting with d3843589feb73fd8a6efed4e62fad887aa34919b6bf11dfce1e0e16be12115a5 not found: ID does not exist"
Jul 24 17:48:00.229771 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.229750    2562 scope.go:117] "RemoveContainer" containerID="19e06f15633b11aa1b842a287987dce4f1182c244491ac7dd965b7b0a9e8624e"
Jul 24 17:48:00.237680 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.237664    2562 scope.go:117] "RemoveContainer" containerID="19e06f15633b11aa1b842a287987dce4f1182c244491ac7dd965b7b0a9e8624e"
Jul 24 17:48:00.237897 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:48:00.237881    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19e06f15633b11aa1b842a287987dce4f1182c244491ac7dd965b7b0a9e8624e\": container with ID starting with 19e06f15633b11aa1b842a287987dce4f1182c244491ac7dd965b7b0a9e8624e not found: ID does not exist" containerID="19e06f15633b11aa1b842a287987dce4f1182c244491ac7dd965b7b0a9e8624e"
Jul 24 17:48:00.237949 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.237901    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19e06f15633b11aa1b842a287987dce4f1182c244491ac7dd965b7b0a9e8624e"} err="failed to get container status \"19e06f15633b11aa1b842a287987dce4f1182c244491ac7dd965b7b0a9e8624e\": rpc error: code = NotFound desc = could not find container \"19e06f15633b11aa1b842a287987dce4f1182c244491ac7dd965b7b0a9e8624e\": container with ID starting with 19e06f15633b11aa1b842a287987dce4f1182c244491ac7dd965b7b0a9e8624e not found: ID does not exist"
Jul 24 17:48:00.238758 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.238738    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p"]
Jul 24 17:48:00.248991 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.248969    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-be21bfbc-94794bbdd-fc46p"]
Jul 24 17:48:00.261477 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.261453    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7"]
Jul 24 17:48:00.265353 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.265330    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-be21bfbc-66bd757c54-rgrf7"]
Jul 24 17:48:00.275788 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.275766    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp"]
Jul 24 17:48:00.282451 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:00.282432    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-be21bfbc-75c6bf559-rfztp"]
Jul 24 17:48:01.824680 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:01.824639    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d26de71-fab8-4eda-acda-b4ede65a5a3d" path="/var/lib/kubelet/pods/1d26de71-fab8-4eda-acda-b4ede65a5a3d/volumes"
Jul 24 17:48:01.825183 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:01.825169    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a736293a-da10-46b8-bb72-98faa41b6e6b" path="/var/lib/kubelet/pods/a736293a-da10-46b8-bb72-98faa41b6e6b/volumes"
Jul 24 17:48:01.825558 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:01.825538    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5f31834-2bf6-4354-8808-afb5a6b0be4c" path="/var/lib/kubelet/pods/b5f31834-2bf6-4354-8808-afb5a6b0be4c/volumes"
Jul 24 17:48:07.871729 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:07.871696    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt"]
Jul 24 17:48:07.872211 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:07.872066    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d26de71-fab8-4eda-acda-b4ede65a5a3d" containerName="maas-api"
Jul 24 17:48:07.872211 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:07.872076    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d26de71-fab8-4eda-acda-b4ede65a5a3d" containerName="maas-api"
Jul 24 17:48:07.872211 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:07.872097    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5f31834-2bf6-4354-8808-afb5a6b0be4c" containerName="payload-processing"
Jul 24 17:48:07.872211 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:07.872103    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5f31834-2bf6-4354-8808-afb5a6b0be4c" containerName="payload-processing"
Jul 24 17:48:07.872211 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:07.872112    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a736293a-da10-46b8-bb72-98faa41b6e6b" containerName="payload-pre-processing"
Jul 24 17:48:07.872211 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:07.872117    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a736293a-da10-46b8-bb72-98faa41b6e6b" containerName="payload-pre-processing"
Jul 24 17:48:07.872211 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:07.872197    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5f31834-2bf6-4354-8808-afb5a6b0be4c" containerName="payload-processing"
Jul 24 17:48:07.872211 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:07.872206    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d26de71-fab8-4eda-acda-b4ede65a5a3d" containerName="maas-api"
Jul 24 17:48:07.872211 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:07.872213    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a736293a-da10-46b8-bb72-98faa41b6e6b" containerName="payload-pre-processing"
Jul 24 17:48:07.877049 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:07.877028    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt"
Jul 24 17:48:07.880346 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:07.880324    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-ad794cce-serving-cert\""
Jul 24 17:48:07.894231 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:07.894206    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt"]
Jul 24 17:48:08.013981 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.013950    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwsjf\" (UniqueName: \"kubernetes.io/projected/fb7e375b-1876-4f19-9452-81110c893f22-kube-api-access-mwsjf\") pod \"maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt\" (UID: \"fb7e375b-1876-4f19-9452-81110c893f22\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt"
Jul 24 17:48:08.014121 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.014003    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb7e375b-1876-4f19-9452-81110c893f22-openshift-service-ca\") pod \"maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt\" (UID: \"fb7e375b-1876-4f19-9452-81110c893f22\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt"
Jul 24 17:48:08.014121 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.014077    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb7e375b-1876-4f19-9452-81110c893f22-maas-api-tls\") pod \"maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt\" (UID: \"fb7e375b-1876-4f19-9452-81110c893f22\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt"
Jul 24 17:48:08.115506 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.115476    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb7e375b-1876-4f19-9452-81110c893f22-openshift-service-ca\") pod \"maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt\" (UID: \"fb7e375b-1876-4f19-9452-81110c893f22\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt"
Jul 24 17:48:08.115661 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.115520    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb7e375b-1876-4f19-9452-81110c893f22-maas-api-tls\") pod \"maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt\" (UID: \"fb7e375b-1876-4f19-9452-81110c893f22\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt"
Jul 24 17:48:08.115661 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.115597    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwsjf\" (UniqueName: \"kubernetes.io/projected/fb7e375b-1876-4f19-9452-81110c893f22-kube-api-access-mwsjf\") pod \"maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt\" (UID: \"fb7e375b-1876-4f19-9452-81110c893f22\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt"
Jul 24 17:48:08.116319 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.116295    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb7e375b-1876-4f19-9452-81110c893f22-openshift-service-ca\") pod \"maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt\" (UID: \"fb7e375b-1876-4f19-9452-81110c893f22\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt"
Jul 24 17:48:08.117869 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.117843    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb7e375b-1876-4f19-9452-81110c893f22-maas-api-tls\") pod \"maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt\" (UID: \"fb7e375b-1876-4f19-9452-81110c893f22\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt"
Jul 24 17:48:08.125501 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.125442    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwsjf\" (UniqueName: \"kubernetes.io/projected/fb7e375b-1876-4f19-9452-81110c893f22-kube-api-access-mwsjf\") pod \"maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt\" (UID: \"fb7e375b-1876-4f19-9452-81110c893f22\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt"
Jul 24 17:48:08.187888 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.187869    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt"
Jul 24 17:48:08.538786 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.538760    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt"]
Jul 24 17:48:08.541682 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:48:08.541655    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb7e375b_1876_4f19_9452_81110c893f22.slice/crio-9a91bcba2235b8448904b2610180acab650a0b3bbd9641625d8d554441ba466b WatchSource:0}: Error finding container 9a91bcba2235b8448904b2610180acab650a0b3bbd9641625d8d554441ba466b: Status 404 returned error can't find the container with id 9a91bcba2235b8448904b2610180acab650a0b3bbd9641625d8d554441ba466b
Jul 24 17:48:08.619544 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.619508    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb"]
Jul 24 17:48:08.624469 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.624451    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb"
Jul 24 17:48:08.631886 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.631865    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-ad794cce\""
Jul 24 17:48:08.637067 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.637044    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb"]
Jul 24 17:48:08.660674 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.660647    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2"]
Jul 24 17:48:08.664852 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.664830    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2"
Jul 24 17:48:08.672268 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.672246    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-ad794cce-dockercfg-ngksb\""
Jul 24 17:48:08.690513 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.690491    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2"]
Jul 24 17:48:08.722401 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.722376    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znprm\" (UniqueName: \"kubernetes.io/projected/0cd65b67-c9c9-4494-9ec7-be27849cafc0-kube-api-access-znprm\") pod \"payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2\" (UID: \"0cd65b67-c9c9-4494-9ec7-be27849cafc0\") " pod="openshift-ingress/payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2"
Jul 24 17:48:08.722520 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.722491    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cd65b67-c9c9-4494-9ec7-be27849cafc0-plugins-config-volume\") pod \"payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2\" (UID: \"0cd65b67-c9c9-4494-9ec7-be27849cafc0\") " pod="openshift-ingress/payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2"
Jul 24 17:48:08.722587 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.722533    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvgkk\" (UniqueName: \"kubernetes.io/projected/90e86b31-364a-4741-b4b4-7440849a0814-kube-api-access-tvgkk\") pod \"payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb\" (UID: \"90e86b31-364a-4741-b4b4-7440849a0814\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb"
Jul 24 17:48:08.722646 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.722613    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90e86b31-364a-4741-b4b4-7440849a0814-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb\" (UID: \"90e86b31-364a-4741-b4b4-7440849a0814\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb"
Jul 24 17:48:08.823911 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.823802    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cd65b67-c9c9-4494-9ec7-be27849cafc0-plugins-config-volume\") pod \"payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2\" (UID: \"0cd65b67-c9c9-4494-9ec7-be27849cafc0\") " pod="openshift-ingress/payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2"
Jul 24 17:48:08.823911 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.823861    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvgkk\" (UniqueName: \"kubernetes.io/projected/90e86b31-364a-4741-b4b4-7440849a0814-kube-api-access-tvgkk\") pod \"payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb\" (UID: \"90e86b31-364a-4741-b4b4-7440849a0814\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb"
Jul 24 17:48:08.823911 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.823913    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90e86b31-364a-4741-b4b4-7440849a0814-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb\" (UID: \"90e86b31-364a-4741-b4b4-7440849a0814\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb"
Jul 24 17:48:08.824256 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.823965    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znprm\" (UniqueName: \"kubernetes.io/projected/0cd65b67-c9c9-4494-9ec7-be27849cafc0-kube-api-access-znprm\") pod \"payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2\" (UID: \"0cd65b67-c9c9-4494-9ec7-be27849cafc0\") " pod="openshift-ingress/payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2"
Jul 24 17:48:08.824619 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.824596    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cd65b67-c9c9-4494-9ec7-be27849cafc0-plugins-config-volume\") pod \"payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2\" (UID: \"0cd65b67-c9c9-4494-9ec7-be27849cafc0\") " pod="openshift-ingress/payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2"
Jul 24 17:48:08.824666 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.824646    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90e86b31-364a-4741-b4b4-7440849a0814-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb\" (UID: \"90e86b31-364a-4741-b4b4-7440849a0814\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb"
Jul 24 17:48:08.845046 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.845019    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvgkk\" (UniqueName: \"kubernetes.io/projected/90e86b31-364a-4741-b4b4-7440849a0814-kube-api-access-tvgkk\") pod \"payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb\" (UID: \"90e86b31-364a-4741-b4b4-7440849a0814\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb"
Jul 24 17:48:08.845046 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.845053    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znprm\" (UniqueName: \"kubernetes.io/projected/0cd65b67-c9c9-4494-9ec7-be27849cafc0-kube-api-access-znprm\") pod \"payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2\" (UID: \"0cd65b67-c9c9-4494-9ec7-be27849cafc0\") " pod="openshift-ingress/payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2"
Jul 24 17:48:08.935617 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.935582    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb"
Jul 24 17:48:08.974821 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:08.974706    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2"
Jul 24 17:48:09.090092 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:09.090060    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb"]
Jul 24 17:48:09.091370 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:48:09.091343    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90e86b31_364a_4741_b4b4_7440849a0814.slice/crio-4981991abebbeda5181fb005b8b55e9dac9778daa78aaaa78d3f781789fe54ee WatchSource:0}: Error finding container 4981991abebbeda5181fb005b8b55e9dac9778daa78aaaa78d3f781789fe54ee: Status 404 returned error can't find the container with id 4981991abebbeda5181fb005b8b55e9dac9778daa78aaaa78d3f781789fe54ee
Jul 24 17:48:09.142063 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:09.142040    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2"]
Jul 24 17:48:09.146333 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:48:09.146307    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cd65b67_c9c9_4494_9ec7_be27849cafc0.slice/crio-c7d8c47a4f254330808b516bb0ffb1f69a9b3c68db4ba29c105bc8b4fcbb2ae5 WatchSource:0}: Error finding container c7d8c47a4f254330808b516bb0ffb1f69a9b3c68db4ba29c105bc8b4fcbb2ae5: Status 404 returned error can't find the container with id c7d8c47a4f254330808b516bb0ffb1f69a9b3c68db4ba29c105bc8b4fcbb2ae5
Jul 24 17:48:09.262084 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:09.262053    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2" event={"ID":"0cd65b67-c9c9-4494-9ec7-be27849cafc0","Type":"ContainerStarted","Data":"aa00944fc682e7c434ab2e4b103500ceea7662e3e7c8ff778a3f1f7b3e47b983"}
Jul 24 17:48:09.262279 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:09.262109    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2" event={"ID":"0cd65b67-c9c9-4494-9ec7-be27849cafc0","Type":"ContainerStarted","Data":"c7d8c47a4f254330808b516bb0ffb1f69a9b3c68db4ba29c105bc8b4fcbb2ae5"}
Jul 24 17:48:09.262279 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:09.262169    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2"
Jul 24 17:48:09.263822 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:09.263792    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt" event={"ID":"fb7e375b-1876-4f19-9452-81110c893f22","Type":"ContainerStarted","Data":"55f6d89f8a89d8d361bf2e47bca7807d5e87bd2f24afba8e5592c7772c84ab8d"}
Jul 24 17:48:09.263945 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:09.263845    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt" event={"ID":"fb7e375b-1876-4f19-9452-81110c893f22","Type":"ContainerStarted","Data":"9a91bcba2235b8448904b2610180acab650a0b3bbd9641625d8d554441ba466b"}
Jul 24 17:48:09.263945 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:09.263879    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt"
Jul 24 17:48:09.265815 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:09.265733    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb" event={"ID":"90e86b31-364a-4741-b4b4-7440849a0814","Type":"ContainerStarted","Data":"943a88c824ad614b0bf20690a5bfd8171d8a77b8880abeebc940e6de47532acc"}
Jul 24 17:48:09.265960 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:09.265816    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb" event={"ID":"90e86b31-364a-4741-b4b4-7440849a0814","Type":"ContainerStarted","Data":"4981991abebbeda5181fb005b8b55e9dac9778daa78aaaa78d3f781789fe54ee"}
Jul 24 17:48:09.265960 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:09.265928    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb"
Jul 24 17:48:09.280221 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:09.280165    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2" podStartSLOduration=1.28012962 podStartE2EDuration="1.28012962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:48:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:48:09.278574621 +0000 UTC m=+1852.005730835" watchObservedRunningTime="2026-07-24 17:48:09.28012962 +0000 UTC m=+1852.007285834"
Jul 24 17:48:09.294910 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:09.294850    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb" podStartSLOduration=1.294836581 podStartE2EDuration="1.294836581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:48:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:48:09.294075438 +0000 UTC m=+1852.021231655" watchObservedRunningTime="2026-07-24 17:48:09.294836581 +0000 UTC m=+1852.021992794"
Jul 24 17:48:09.313342 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:09.313283    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt" podStartSLOduration=1.9199942330000002 podStartE2EDuration="2.313262835s" totalImagesPullingTime="393.268602ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:48:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:48:09.311318005 +0000 UTC m=+1852.038474244" watchObservedRunningTime="2026-07-24 17:48:09.313262835 +0000 UTC m=+1852.040419051"
Jul 24 17:48:15.276505 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:15.276474    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt"
Jul 24 17:48:20.273290 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:20.273256    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2"
Jul 24 17:48:20.273676 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:20.273337    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb"
Jul 24 17:48:43.533484 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:43.533401    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt"]
Jul 24 17:48:43.533887 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:43.533668    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt" podUID="fb7e375b-1876-4f19-9452-81110c893f22" containerName="maas-api" containerID="cri-o://55f6d89f8a89d8d361bf2e47bca7807d5e87bd2f24afba8e5592c7772c84ab8d" gracePeriod=30
Jul 24 17:48:43.589868 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:43.589834    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2"]
Jul 24 17:48:43.590135 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:43.590090    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2" podUID="0cd65b67-c9c9-4494-9ec7-be27849cafc0" containerName="payload-processing" containerID="cri-o://aa00944fc682e7c434ab2e4b103500ceea7662e3e7c8ff778a3f1f7b3e47b983" gracePeriod=30
Jul 24 17:48:43.633530 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:43.633494    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb"]
Jul 24 17:48:43.633765 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:43.633741    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb" podUID="90e86b31-364a-4741-b4b4-7440849a0814" containerName="payload-pre-processing" containerID="cri-o://943a88c824ad614b0bf20690a5bfd8171d8a77b8880abeebc940e6de47532acc" gracePeriod=30
Jul 24 17:48:43.849946 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:43.849778    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt"
Jul 24 17:48:43.919055 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:43.919030    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2"
Jul 24 17:48:43.922126 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:43.922108    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb"
Jul 24 17:48:43.946785 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:43.946760    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwsjf\" (UniqueName: \"kubernetes.io/projected/fb7e375b-1876-4f19-9452-81110c893f22-kube-api-access-mwsjf\") pod \"fb7e375b-1876-4f19-9452-81110c893f22\" (UID: \"fb7e375b-1876-4f19-9452-81110c893f22\") "
Jul 24 17:48:43.946890 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:43.946866    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb7e375b-1876-4f19-9452-81110c893f22-openshift-service-ca\") pod \"fb7e375b-1876-4f19-9452-81110c893f22\" (UID: \"fb7e375b-1876-4f19-9452-81110c893f22\") "
Jul 24 17:48:43.946940 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:43.946892    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb7e375b-1876-4f19-9452-81110c893f22-maas-api-tls\") pod \"fb7e375b-1876-4f19-9452-81110c893f22\" (UID: \"fb7e375b-1876-4f19-9452-81110c893f22\") "
Jul 24 17:48:43.947300 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:43.947261    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb7e375b-1876-4f19-9452-81110c893f22-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fb7e375b-1876-4f19-9452-81110c893f22" (UID: "fb7e375b-1876-4f19-9452-81110c893f22"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:48:43.948960 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:43.948930    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb7e375b-1876-4f19-9452-81110c893f22-kube-api-access-mwsjf" (OuterVolumeSpecName: "kube-api-access-mwsjf") pod "fb7e375b-1876-4f19-9452-81110c893f22" (UID: "fb7e375b-1876-4f19-9452-81110c893f22"). InnerVolumeSpecName "kube-api-access-mwsjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:48:43.949088 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:43.948979    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb7e375b-1876-4f19-9452-81110c893f22-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fb7e375b-1876-4f19-9452-81110c893f22" (UID: "fb7e375b-1876-4f19-9452-81110c893f22"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:48:44.047792 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.047761    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvgkk\" (UniqueName: \"kubernetes.io/projected/90e86b31-364a-4741-b4b4-7440849a0814-kube-api-access-tvgkk\") pod \"90e86b31-364a-4741-b4b4-7440849a0814\" (UID: \"90e86b31-364a-4741-b4b4-7440849a0814\") "
Jul 24 17:48:44.047937 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.047823    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90e86b31-364a-4741-b4b4-7440849a0814-plugins-config-volume\") pod \"90e86b31-364a-4741-b4b4-7440849a0814\" (UID: \"90e86b31-364a-4741-b4b4-7440849a0814\") "
Jul 24 17:48:44.047937 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.047859    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-znprm\" (UniqueName: \"kubernetes.io/projected/0cd65b67-c9c9-4494-9ec7-be27849cafc0-kube-api-access-znprm\") pod \"0cd65b67-c9c9-4494-9ec7-be27849cafc0\" (UID: \"0cd65b67-c9c9-4494-9ec7-be27849cafc0\") "
Jul 24 17:48:44.047937 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.047922    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cd65b67-c9c9-4494-9ec7-be27849cafc0-plugins-config-volume\") pod \"0cd65b67-c9c9-4494-9ec7-be27849cafc0\" (UID: \"0cd65b67-c9c9-4494-9ec7-be27849cafc0\") "
Jul 24 17:48:44.048190 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.048169    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwsjf\" (UniqueName: \"kubernetes.io/projected/fb7e375b-1876-4f19-9452-81110c893f22-kube-api-access-mwsjf\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:48:44.048279 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.048195    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb7e375b-1876-4f19-9452-81110c893f22-openshift-service-ca\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:48:44.048279 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.048216    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb7e375b-1876-4f19-9452-81110c893f22-maas-api-tls\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:48:44.048279 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.048189    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90e86b31-364a-4741-b4b4-7440849a0814-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "90e86b31-364a-4741-b4b4-7440849a0814" (UID: "90e86b31-364a-4741-b4b4-7440849a0814"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:48:44.048466 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.048436    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cd65b67-c9c9-4494-9ec7-be27849cafc0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0cd65b67-c9c9-4494-9ec7-be27849cafc0" (UID: "0cd65b67-c9c9-4494-9ec7-be27849cafc0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:48:44.049788 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.049768    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90e86b31-364a-4741-b4b4-7440849a0814-kube-api-access-tvgkk" (OuterVolumeSpecName: "kube-api-access-tvgkk") pod "90e86b31-364a-4741-b4b4-7440849a0814" (UID: "90e86b31-364a-4741-b4b4-7440849a0814"). InnerVolumeSpecName "kube-api-access-tvgkk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:48:44.049869 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.049849    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cd65b67-c9c9-4494-9ec7-be27849cafc0-kube-api-access-znprm" (OuterVolumeSpecName: "kube-api-access-znprm") pod "0cd65b67-c9c9-4494-9ec7-be27849cafc0" (UID: "0cd65b67-c9c9-4494-9ec7-be27849cafc0"). InnerVolumeSpecName "kube-api-access-znprm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:48:44.149418 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.149390    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90e86b31-364a-4741-b4b4-7440849a0814-plugins-config-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:48:44.149418 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.149414    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-znprm\" (UniqueName: \"kubernetes.io/projected/0cd65b67-c9c9-4494-9ec7-be27849cafc0-kube-api-access-znprm\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:48:44.149418 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.149424    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cd65b67-c9c9-4494-9ec7-be27849cafc0-plugins-config-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:48:44.149635 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.149433    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvgkk\" (UniqueName: \"kubernetes.io/projected/90e86b31-364a-4741-b4b4-7440849a0814-kube-api-access-tvgkk\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:48:44.443823 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.443737    2562 generic.go:356] "Generic (PLEG): container finished" podID="0cd65b67-c9c9-4494-9ec7-be27849cafc0" containerID="aa00944fc682e7c434ab2e4b103500ceea7662e3e7c8ff778a3f1f7b3e47b983" exitCode=0
Jul 24 17:48:44.443823 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.443787    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2"
Jul 24 17:48:44.444080 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.443823    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2" event={"ID":"0cd65b67-c9c9-4494-9ec7-be27849cafc0","Type":"ContainerDied","Data":"aa00944fc682e7c434ab2e4b103500ceea7662e3e7c8ff778a3f1f7b3e47b983"}
Jul 24 17:48:44.444080 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.443890    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2" event={"ID":"0cd65b67-c9c9-4494-9ec7-be27849cafc0","Type":"ContainerDied","Data":"c7d8c47a4f254330808b516bb0ffb1f69a9b3c68db4ba29c105bc8b4fcbb2ae5"}
Jul 24 17:48:44.444080 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.443925    2562 scope.go:117] "RemoveContainer" containerID="aa00944fc682e7c434ab2e4b103500ceea7662e3e7c8ff778a3f1f7b3e47b983"
Jul 24 17:48:44.445440 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.445420    2562 generic.go:356] "Generic (PLEG): container finished" podID="fb7e375b-1876-4f19-9452-81110c893f22" containerID="55f6d89f8a89d8d361bf2e47bca7807d5e87bd2f24afba8e5592c7772c84ab8d" exitCode=0
Jul 24 17:48:44.445535 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.445474    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt"
Jul 24 17:48:44.445535 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.445504    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt" event={"ID":"fb7e375b-1876-4f19-9452-81110c893f22","Type":"ContainerDied","Data":"55f6d89f8a89d8d361bf2e47bca7807d5e87bd2f24afba8e5592c7772c84ab8d"}
Jul 24 17:48:44.445653 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.445556    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt" event={"ID":"fb7e375b-1876-4f19-9452-81110c893f22","Type":"ContainerDied","Data":"9a91bcba2235b8448904b2610180acab650a0b3bbd9641625d8d554441ba466b"}
Jul 24 17:48:44.446840 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.446811    2562 generic.go:356] "Generic (PLEG): container finished" podID="90e86b31-364a-4741-b4b4-7440849a0814" containerID="943a88c824ad614b0bf20690a5bfd8171d8a77b8880abeebc940e6de47532acc" exitCode=0
Jul 24 17:48:44.446961 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.446835    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb" event={"ID":"90e86b31-364a-4741-b4b4-7440849a0814","Type":"ContainerDied","Data":"943a88c824ad614b0bf20690a5bfd8171d8a77b8880abeebc940e6de47532acc"}
Jul 24 17:48:44.446961 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.446860    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb"
Jul 24 17:48:44.446961 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.446877    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb" event={"ID":"90e86b31-364a-4741-b4b4-7440849a0814","Type":"ContainerDied","Data":"4981991abebbeda5181fb005b8b55e9dac9778daa78aaaa78d3f781789fe54ee"}
Jul 24 17:48:44.453454 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.453420    2562 scope.go:117] "RemoveContainer" containerID="aa00944fc682e7c434ab2e4b103500ceea7662e3e7c8ff778a3f1f7b3e47b983"
Jul 24 17:48:44.453719 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:48:44.453697    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa00944fc682e7c434ab2e4b103500ceea7662e3e7c8ff778a3f1f7b3e47b983\": container with ID starting with aa00944fc682e7c434ab2e4b103500ceea7662e3e7c8ff778a3f1f7b3e47b983 not found: ID does not exist" containerID="aa00944fc682e7c434ab2e4b103500ceea7662e3e7c8ff778a3f1f7b3e47b983"
Jul 24 17:48:44.453812 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.453730    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa00944fc682e7c434ab2e4b103500ceea7662e3e7c8ff778a3f1f7b3e47b983"} err="failed to get container status \"aa00944fc682e7c434ab2e4b103500ceea7662e3e7c8ff778a3f1f7b3e47b983\": rpc error: code = NotFound desc = could not find container \"aa00944fc682e7c434ab2e4b103500ceea7662e3e7c8ff778a3f1f7b3e47b983\": container with ID starting with aa00944fc682e7c434ab2e4b103500ceea7662e3e7c8ff778a3f1f7b3e47b983 not found: ID does not exist"
Jul 24 17:48:44.453812 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.453751    2562 scope.go:117] "RemoveContainer" containerID="55f6d89f8a89d8d361bf2e47bca7807d5e87bd2f24afba8e5592c7772c84ab8d"
Jul 24 17:48:44.463565 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.463544    2562 scope.go:117] "RemoveContainer" containerID="55f6d89f8a89d8d361bf2e47bca7807d5e87bd2f24afba8e5592c7772c84ab8d"
Jul 24 17:48:44.463843 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:48:44.463825    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55f6d89f8a89d8d361bf2e47bca7807d5e87bd2f24afba8e5592c7772c84ab8d\": container with ID starting with 55f6d89f8a89d8d361bf2e47bca7807d5e87bd2f24afba8e5592c7772c84ab8d not found: ID does not exist" containerID="55f6d89f8a89d8d361bf2e47bca7807d5e87bd2f24afba8e5592c7772c84ab8d"
Jul 24 17:48:44.463897 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.463848    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55f6d89f8a89d8d361bf2e47bca7807d5e87bd2f24afba8e5592c7772c84ab8d"} err="failed to get container status \"55f6d89f8a89d8d361bf2e47bca7807d5e87bd2f24afba8e5592c7772c84ab8d\": rpc error: code = NotFound desc = could not find container \"55f6d89f8a89d8d361bf2e47bca7807d5e87bd2f24afba8e5592c7772c84ab8d\": container with ID starting with 55f6d89f8a89d8d361bf2e47bca7807d5e87bd2f24afba8e5592c7772c84ab8d not found: ID does not exist"
Jul 24 17:48:44.463897 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.463864    2562 scope.go:117] "RemoveContainer" containerID="943a88c824ad614b0bf20690a5bfd8171d8a77b8880abeebc940e6de47532acc"
Jul 24 17:48:44.472132 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.472116    2562 scope.go:117] "RemoveContainer" containerID="943a88c824ad614b0bf20690a5bfd8171d8a77b8880abeebc940e6de47532acc"
Jul 24 17:48:44.472486 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:48:44.472466    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"943a88c824ad614b0bf20690a5bfd8171d8a77b8880abeebc940e6de47532acc\": container with ID starting with 943a88c824ad614b0bf20690a5bfd8171d8a77b8880abeebc940e6de47532acc not found: ID does not exist" containerID="943a88c824ad614b0bf20690a5bfd8171d8a77b8880abeebc940e6de47532acc"
Jul 24 17:48:44.472558 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.472494    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"943a88c824ad614b0bf20690a5bfd8171d8a77b8880abeebc940e6de47532acc"} err="failed to get container status \"943a88c824ad614b0bf20690a5bfd8171d8a77b8880abeebc940e6de47532acc\": rpc error: code = NotFound desc = could not find container \"943a88c824ad614b0bf20690a5bfd8171d8a77b8880abeebc940e6de47532acc\": container with ID starting with 943a88c824ad614b0bf20690a5bfd8171d8a77b8880abeebc940e6de47532acc not found: ID does not exist"
Jul 24 17:48:44.473298 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.473278    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb"]
Jul 24 17:48:44.477427 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.477403    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-ad794cce-7f59cbcb56-wztsb"]
Jul 24 17:48:44.490407 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.490387    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2"]
Jul 24 17:48:44.496976 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.496955    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ad794cce-74479c9b8f-b78n2"]
Jul 24 17:48:44.508867 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.508847    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt"]
Jul 24 17:48:44.513055 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:44.513033    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ad794cce-78f7849759-hx4mt"]
Jul 24 17:48:45.825052 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:45.825020    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cd65b67-c9c9-4494-9ec7-be27849cafc0" path="/var/lib/kubelet/pods/0cd65b67-c9c9-4494-9ec7-be27849cafc0/volumes"
Jul 24 17:48:45.825430 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:45.825417    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90e86b31-364a-4741-b4b4-7440849a0814" path="/var/lib/kubelet/pods/90e86b31-364a-4741-b4b4-7440849a0814/volumes"
Jul 24 17:48:45.825767 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:45.825754    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb7e375b-1876-4f19-9452-81110c893f22" path="/var/lib/kubelet/pods/fb7e375b-1876-4f19-9452-81110c893f22/volumes"
Jul 24 17:48:52.234402 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:52.234367    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-647df330-7df554ccf7-x5pj2"]
Jul 24 17:48:52.234771 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:52.234752    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cd65b67-c9c9-4494-9ec7-be27849cafc0" containerName="payload-processing"
Jul 24 17:48:52.234771 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:52.234763    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cd65b67-c9c9-4494-9ec7-be27849cafc0" containerName="payload-processing"
Jul 24 17:48:52.234843 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:52.234782    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90e86b31-364a-4741-b4b4-7440849a0814" containerName="payload-pre-processing"
Jul 24 17:48:52.234843 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:52.234788    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="90e86b31-364a-4741-b4b4-7440849a0814" containerName="payload-pre-processing"
Jul 24 17:48:52.234843 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:52.234804    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb7e375b-1876-4f19-9452-81110c893f22" containerName="maas-api"
Jul 24 17:48:52.234843 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:52.234810    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb7e375b-1876-4f19-9452-81110c893f22" containerName="maas-api"
Jul 24 17:48:52.234978 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:52.234864    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb7e375b-1876-4f19-9452-81110c893f22" containerName="maas-api"
Jul 24 17:48:52.234978 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:52.234871    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cd65b67-c9c9-4494-9ec7-be27849cafc0" containerName="payload-processing"
Jul 24 17:48:52.234978 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:52.234888    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="90e86b31-364a-4741-b4b4-7440849a0814" containerName="payload-pre-processing"
Jul 24 17:48:52.237826 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:52.237805    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-647df330-7df554ccf7-x5pj2"
Jul 24 17:48:52.242538 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:52.242516    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-647df330-serving-cert\""
Jul 24 17:48:52.260805 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:52.260771    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-647df330-7df554ccf7-x5pj2"]
Jul 24 17:48:52.318247 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:52.318197    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2-maas-api-tls\") pod \"maas-api-e2e-ait-647df330-7df554ccf7-x5pj2\" (UID: \"f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-647df330-7df554ccf7-x5pj2"
Jul 24 17:48:52.318427 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:52.318255    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf8sw\" (UniqueName: \"kubernetes.io/projected/f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2-kube-api-access-rf8sw\") pod \"maas-api-e2e-ait-647df330-7df554ccf7-x5pj2\" (UID: \"f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-647df330-7df554ccf7-x5pj2"
Jul 24 17:48:52.318427 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:52.318380    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2-openshift-service-ca\") pod \"maas-api-e2e-ait-647df330-7df554ccf7-x5pj2\" (UID: \"f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-647df330-7df554ccf7-x5pj2"
Jul 24 17:48:52.419199 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:52.419162    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2-openshift-service-ca\") pod \"maas-api-e2e-ait-647df330-7df554ccf7-x5pj2\" (UID: \"f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-647df330-7df554ccf7-x5pj2"
Jul 24 17:48:52.419406 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:52.419312    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2-maas-api-tls\") pod \"maas-api-e2e-ait-647df330-7df554ccf7-x5pj2\" (UID: \"f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-647df330-7df554ccf7-x5pj2"
Jul 24 17:48:52.419406 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:52.419341    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf8sw\" (UniqueName: \"kubernetes.io/projected/f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2-kube-api-access-rf8sw\") pod \"maas-api-e2e-ait-647df330-7df554ccf7-x5pj2\" (UID: \"f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-647df330-7df554ccf7-x5pj2"
Jul 24 17:48:52.419901 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:52.419873    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2-openshift-service-ca\") pod \"maas-api-e2e-ait-647df330-7df554ccf7-x5pj2\" (UID: \"f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-647df330-7df554ccf7-x5pj2"
Jul 24 17:48:52.421757 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:52.421734    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2-maas-api-tls\") pod \"maas-api-e2e-ait-647df330-7df554ccf7-x5pj2\" (UID: \"f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-647df330-7df554ccf7-x5pj2"
Jul 24 17:48:52.436312 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:52.436292    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf8sw\" (UniqueName: \"kubernetes.io/projected/f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2-kube-api-access-rf8sw\") pod \"maas-api-e2e-ait-647df330-7df554ccf7-x5pj2\" (UID: \"f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-647df330-7df554ccf7-x5pj2"
Jul 24 17:48:52.547839 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:52.547715    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-647df330-7df554ccf7-x5pj2"
Jul 24 17:48:52.688970 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:52.688944    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-647df330-7df554ccf7-x5pj2"]
Jul 24 17:48:52.691417 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:48:52.691380    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5fe599c_c0fa_41fe_8b7f_433e6c60d0b2.slice/crio-ceb1ef00ffbd7e4df5a48ecd07bf1d8232690ea405b26bd041711a186076ec4c WatchSource:0}: Error finding container ceb1ef00ffbd7e4df5a48ecd07bf1d8232690ea405b26bd041711a186076ec4c: Status 404 returned error can't find the container with id ceb1ef00ffbd7e4df5a48ecd07bf1d8232690ea405b26bd041711a186076ec4c
Jul 24 17:48:53.291908 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:53.291879    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6"]
Jul 24 17:48:53.295489 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:53.295473    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6"
Jul 24 17:48:53.299473 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:53.299449    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-647df330-dockercfg-4hswc\""
Jul 24 17:48:53.299647 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:53.299626    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-647df330\""
Jul 24 17:48:53.314946 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:53.314921    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6"]
Jul 24 17:48:53.379383 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:53.379356    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-647df330-795b59b65d-99bm6"]
Jul 24 17:48:53.382863 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:53.382846    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-647df330-795b59b65d-99bm6"
Jul 24 17:48:53.400040 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:53.400017    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-647df330-795b59b65d-99bm6"]
Jul 24 17:48:53.430291 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:53.430257    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rz97z\" (UniqueName: \"kubernetes.io/projected/a70777a8-1897-490d-8561-df9374c4030a-kube-api-access-rz97z\") pod \"payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6\" (UID: \"a70777a8-1897-490d-8561-df9374c4030a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6"
Jul 24 17:48:53.430443 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:53.430365    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a70777a8-1897-490d-8561-df9374c4030a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6\" (UID: \"a70777a8-1897-490d-8561-df9374c4030a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6"
Jul 24 17:48:53.496810 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:53.496774    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-647df330-7df554ccf7-x5pj2" event={"ID":"f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2","Type":"ContainerStarted","Data":"0e9c8601f2f12d526bf31ec0dd613090f0d04f09e51f0a2022f2992125a57aa0"}
Jul 24 17:48:53.496950 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:53.496837    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-647df330-7df554ccf7-x5pj2" event={"ID":"f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2","Type":"ContainerStarted","Data":"ceb1ef00ffbd7e4df5a48ecd07bf1d8232690ea405b26bd041711a186076ec4c"}
Jul 24 17:48:53.496950 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:53.496928    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-647df330-7df554ccf7-x5pj2"
Jul 24 17:48:53.524298 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:53.524237    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-647df330-7df554ccf7-x5pj2" podStartSLOduration=1.152205961 podStartE2EDuration="1.524217077s" totalImagesPullingTime="372.011116ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:48:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:48:53.521683731 +0000 UTC m=+1896.248839944" watchObservedRunningTime="2026-07-24 17:48:53.524217077 +0000 UTC m=+1896.251373292"
Jul 24 17:48:53.531045 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:53.531017    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rz97z\" (UniqueName: \"kubernetes.io/projected/a70777a8-1897-490d-8561-df9374c4030a-kube-api-access-rz97z\") pod \"payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6\" (UID: \"a70777a8-1897-490d-8561-df9374c4030a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6"
Jul 24 17:48:53.531204 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:53.531051    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51a89701-db56-4d0a-95f5-b0ec15469e8b-plugins-config-volume\") pod \"payload-processing-e2e-ait-647df330-795b59b65d-99bm6\" (UID: \"51a89701-db56-4d0a-95f5-b0ec15469e8b\") " pod="openshift-ingress/payload-processing-e2e-ait-647df330-795b59b65d-99bm6"
Jul 24 17:48:53.531204 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:53.531100    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flqgv\" (UniqueName: \"kubernetes.io/projected/51a89701-db56-4d0a-95f5-b0ec15469e8b-kube-api-access-flqgv\") pod \"payload-processing-e2e-ait-647df330-795b59b65d-99bm6\" (UID: \"51a89701-db56-4d0a-95f5-b0ec15469e8b\") " pod="openshift-ingress/payload-processing-e2e-ait-647df330-795b59b65d-99bm6"
Jul 24 17:48:53.531204 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:53.531130    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a70777a8-1897-490d-8561-df9374c4030a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6\" (UID: \"a70777a8-1897-490d-8561-df9374c4030a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6"
Jul 24 17:48:53.531798 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:53.531781    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a70777a8-1897-490d-8561-df9374c4030a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6\" (UID: \"a70777a8-1897-490d-8561-df9374c4030a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6"
Jul 24 17:48:53.542437 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:53.542411    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rz97z\" (UniqueName: \"kubernetes.io/projected/a70777a8-1897-490d-8561-df9374c4030a-kube-api-access-rz97z\") pod \"payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6\" (UID: \"a70777a8-1897-490d-8561-df9374c4030a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6"
Jul 24 17:48:53.605363 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:53.605330    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6"
Jul 24 17:48:53.632576 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:53.632514    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51a89701-db56-4d0a-95f5-b0ec15469e8b-plugins-config-volume\") pod \"payload-processing-e2e-ait-647df330-795b59b65d-99bm6\" (UID: \"51a89701-db56-4d0a-95f5-b0ec15469e8b\") " pod="openshift-ingress/payload-processing-e2e-ait-647df330-795b59b65d-99bm6"
Jul 24 17:48:53.632682 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:53.632626    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flqgv\" (UniqueName: \"kubernetes.io/projected/51a89701-db56-4d0a-95f5-b0ec15469e8b-kube-api-access-flqgv\") pod \"payload-processing-e2e-ait-647df330-795b59b65d-99bm6\" (UID: \"51a89701-db56-4d0a-95f5-b0ec15469e8b\") " pod="openshift-ingress/payload-processing-e2e-ait-647df330-795b59b65d-99bm6"
Jul 24 17:48:53.633353 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:53.633332    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51a89701-db56-4d0a-95f5-b0ec15469e8b-plugins-config-volume\") pod \"payload-processing-e2e-ait-647df330-795b59b65d-99bm6\" (UID: \"51a89701-db56-4d0a-95f5-b0ec15469e8b\") " pod="openshift-ingress/payload-processing-e2e-ait-647df330-795b59b65d-99bm6"
Jul 24 17:48:53.643274 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:53.643249    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flqgv\" (UniqueName: \"kubernetes.io/projected/51a89701-db56-4d0a-95f5-b0ec15469e8b-kube-api-access-flqgv\") pod \"payload-processing-e2e-ait-647df330-795b59b65d-99bm6\" (UID: \"51a89701-db56-4d0a-95f5-b0ec15469e8b\") " pod="openshift-ingress/payload-processing-e2e-ait-647df330-795b59b65d-99bm6"
Jul 24 17:48:53.693495 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:53.693452    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-647df330-795b59b65d-99bm6"
Jul 24 17:48:53.752098 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:53.752033    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6"]
Jul 24 17:48:53.755830 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:48:53.754916    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda70777a8_1897_490d_8561_df9374c4030a.slice/crio-c1de6e7e482e829012b510f7ebd2e7b2bf64e2bf1e651428c45386e0831f3a01 WatchSource:0}: Error finding container c1de6e7e482e829012b510f7ebd2e7b2bf64e2bf1e651428c45386e0831f3a01: Status 404 returned error can't find the container with id c1de6e7e482e829012b510f7ebd2e7b2bf64e2bf1e651428c45386e0831f3a01
Jul 24 17:48:53.863626 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:53.863594    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-647df330-795b59b65d-99bm6"]
Jul 24 17:48:53.868023 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:48:53.867995    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51a89701_db56_4d0a_95f5_b0ec15469e8b.slice/crio-51e8437e83a2dd11ffde40f0c28a33d323e1adde7ea807c8a158bda1de51c1ab WatchSource:0}: Error finding container 51e8437e83a2dd11ffde40f0c28a33d323e1adde7ea807c8a158bda1de51c1ab: Status 404 returned error can't find the container with id 51e8437e83a2dd11ffde40f0c28a33d323e1adde7ea807c8a158bda1de51c1ab
Jul 24 17:48:54.503164 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:54.503115    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6" event={"ID":"a70777a8-1897-490d-8561-df9374c4030a","Type":"ContainerStarted","Data":"d25c04a71f766bb7e571b2f0e0ad937d806536959f7c4fdf26f080b14e3140fa"}
Jul 24 17:48:54.503586 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:54.503313    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6"
Jul 24 17:48:54.503586 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:54.503365    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6" event={"ID":"a70777a8-1897-490d-8561-df9374c4030a","Type":"ContainerStarted","Data":"c1de6e7e482e829012b510f7ebd2e7b2bf64e2bf1e651428c45386e0831f3a01"}
Jul 24 17:48:54.505189 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:54.505155    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-647df330-795b59b65d-99bm6" event={"ID":"51a89701-db56-4d0a-95f5-b0ec15469e8b","Type":"ContainerStarted","Data":"ec10745e2265e8a4118649a7fb252c7e11495e9e39172a8529d47ffe9c4b58ad"}
Jul 24 17:48:54.505349 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:54.505210    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-647df330-795b59b65d-99bm6" event={"ID":"51a89701-db56-4d0a-95f5-b0ec15469e8b","Type":"ContainerStarted","Data":"51e8437e83a2dd11ffde40f0c28a33d323e1adde7ea807c8a158bda1de51c1ab"}
Jul 24 17:48:54.505442 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:54.505423    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-647df330-795b59b65d-99bm6"
Jul 24 17:48:54.524801 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:54.524760    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6" podStartSLOduration=1.524741082 podStartE2EDuration="1.524741082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:48:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:48:54.524392327 +0000 UTC m=+1897.251548543" watchObservedRunningTime="2026-07-24 17:48:54.524741082 +0000 UTC m=+1897.251897298"
Jul 24 17:48:54.562032 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:54.561988    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-647df330-795b59b65d-99bm6" podStartSLOduration=1.56197273 podStartE2EDuration="1.56197273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:48:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:48:54.561073988 +0000 UTC m=+1897.288230215" watchObservedRunningTime="2026-07-24 17:48:54.56197273 +0000 UTC m=+1897.289128943"
Jul 24 17:48:59.512834 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:48:59.512803    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-647df330-7df554ccf7-x5pj2"
Jul 24 17:49:05.513541 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:05.513502    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6"
Jul 24 17:49:05.514049 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:05.513792    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-647df330-795b59b65d-99bm6"
Jul 24 17:49:28.490415 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:28.490384    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-647df330-7df554ccf7-x5pj2"]
Jul 24 17:49:28.490851 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:28.490592    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-647df330-7df554ccf7-x5pj2" podUID="f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2" containerName="maas-api" containerID="cri-o://0e9c8601f2f12d526bf31ec0dd613090f0d04f09e51f0a2022f2992125a57aa0" gracePeriod=30
Jul 24 17:49:28.565176 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:28.565125    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-647df330-795b59b65d-99bm6"]
Jul 24 17:49:28.565469 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:28.565397    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-647df330-795b59b65d-99bm6" podUID="51a89701-db56-4d0a-95f5-b0ec15469e8b" containerName="payload-processing" containerID="cri-o://ec10745e2265e8a4118649a7fb252c7e11495e9e39172a8529d47ffe9c4b58ad" gracePeriod=30
Jul 24 17:49:28.574827 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:28.574798    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6"]
Jul 24 17:49:28.575077 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:28.575030    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6" podUID="a70777a8-1897-490d-8561-df9374c4030a" containerName="payload-pre-processing" containerID="cri-o://d25c04a71f766bb7e571b2f0e0ad937d806536959f7c4fdf26f080b14e3140fa" gracePeriod=30
Jul 24 17:49:28.686531 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:28.686496    2562 generic.go:356] "Generic (PLEG): container finished" podID="51a89701-db56-4d0a-95f5-b0ec15469e8b" containerID="ec10745e2265e8a4118649a7fb252c7e11495e9e39172a8529d47ffe9c4b58ad" exitCode=0
Jul 24 17:49:28.686648 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:28.686619    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-647df330-795b59b65d-99bm6" event={"ID":"51a89701-db56-4d0a-95f5-b0ec15469e8b","Type":"ContainerDied","Data":"ec10745e2265e8a4118649a7fb252c7e11495e9e39172a8529d47ffe9c4b58ad"}
Jul 24 17:49:28.688756 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:28.688734    2562 generic.go:356] "Generic (PLEG): container finished" podID="f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2" containerID="0e9c8601f2f12d526bf31ec0dd613090f0d04f09e51f0a2022f2992125a57aa0" exitCode=0
Jul 24 17:49:28.688879 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:28.688802    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-647df330-7df554ccf7-x5pj2" event={"ID":"f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2","Type":"ContainerDied","Data":"0e9c8601f2f12d526bf31ec0dd613090f0d04f09e51f0a2022f2992125a57aa0"}
Jul 24 17:49:28.835074 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:28.834780    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-647df330-795b59b65d-99bm6"
Jul 24 17:49:28.850467 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:28.850442    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-647df330-7df554ccf7-x5pj2"
Jul 24 17:49:28.875875 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:28.875722    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6"
Jul 24 17:49:28.950517 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:28.950472    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rz97z\" (UniqueName: \"kubernetes.io/projected/a70777a8-1897-490d-8561-df9374c4030a-kube-api-access-rz97z\") pod \"a70777a8-1897-490d-8561-df9374c4030a\" (UID: \"a70777a8-1897-490d-8561-df9374c4030a\") "
Jul 24 17:49:28.950696 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:28.950538    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rf8sw\" (UniqueName: \"kubernetes.io/projected/f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2-kube-api-access-rf8sw\") pod \"f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2\" (UID: \"f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2\") "
Jul 24 17:49:28.950696 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:28.950598    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51a89701-db56-4d0a-95f5-b0ec15469e8b-plugins-config-volume\") pod \"51a89701-db56-4d0a-95f5-b0ec15469e8b\" (UID: \"51a89701-db56-4d0a-95f5-b0ec15469e8b\") "
Jul 24 17:49:28.950696 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:28.950628    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-flqgv\" (UniqueName: \"kubernetes.io/projected/51a89701-db56-4d0a-95f5-b0ec15469e8b-kube-api-access-flqgv\") pod \"51a89701-db56-4d0a-95f5-b0ec15469e8b\" (UID: \"51a89701-db56-4d0a-95f5-b0ec15469e8b\") "
Jul 24 17:49:28.950696 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:28.950666    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2-maas-api-tls\") pod \"f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2\" (UID: \"f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2\") "
Jul 24 17:49:28.950939 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:28.950775    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2-openshift-service-ca\") pod \"f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2\" (UID: \"f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2\") "
Jul 24 17:49:28.950939 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:28.950809    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a70777a8-1897-490d-8561-df9374c4030a-plugins-config-volume\") pod \"a70777a8-1897-490d-8561-df9374c4030a\" (UID: \"a70777a8-1897-490d-8561-df9374c4030a\") "
Jul 24 17:49:28.951187 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:28.951008    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51a89701-db56-4d0a-95f5-b0ec15469e8b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "51a89701-db56-4d0a-95f5-b0ec15469e8b" (UID: "51a89701-db56-4d0a-95f5-b0ec15469e8b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:49:28.951290 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:28.951272    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51a89701-db56-4d0a-95f5-b0ec15469e8b-plugins-config-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:49:28.951357 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:28.951329    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a70777a8-1897-490d-8561-df9374c4030a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a70777a8-1897-490d-8561-df9374c4030a" (UID: "a70777a8-1897-490d-8561-df9374c4030a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:49:28.951357 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:28.951344    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2" (UID: "f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:49:28.952794 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:28.952772    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51a89701-db56-4d0a-95f5-b0ec15469e8b-kube-api-access-flqgv" (OuterVolumeSpecName: "kube-api-access-flqgv") pod "51a89701-db56-4d0a-95f5-b0ec15469e8b" (UID: "51a89701-db56-4d0a-95f5-b0ec15469e8b"). InnerVolumeSpecName "kube-api-access-flqgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:49:28.952902 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:28.952798    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a70777a8-1897-490d-8561-df9374c4030a-kube-api-access-rz97z" (OuterVolumeSpecName: "kube-api-access-rz97z") pod "a70777a8-1897-490d-8561-df9374c4030a" (UID: "a70777a8-1897-490d-8561-df9374c4030a"). InnerVolumeSpecName "kube-api-access-rz97z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:49:28.953163 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:28.953123    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2-kube-api-access-rf8sw" (OuterVolumeSpecName: "kube-api-access-rf8sw") pod "f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2" (UID: "f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2"). InnerVolumeSpecName "kube-api-access-rf8sw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:49:28.953269 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:28.953129    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2" (UID: "f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:49:29.052020 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:29.051929    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rz97z\" (UniqueName: \"kubernetes.io/projected/a70777a8-1897-490d-8561-df9374c4030a-kube-api-access-rz97z\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:49:29.052020 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:29.051961    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rf8sw\" (UniqueName: \"kubernetes.io/projected/f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2-kube-api-access-rf8sw\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:49:29.052020 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:29.051973    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flqgv\" (UniqueName: \"kubernetes.io/projected/51a89701-db56-4d0a-95f5-b0ec15469e8b-kube-api-access-flqgv\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:49:29.052020 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:29.051985    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2-maas-api-tls\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:49:29.052020 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:29.051994    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2-openshift-service-ca\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:49:29.052020 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:29.052003    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a70777a8-1897-490d-8561-df9374c4030a-plugins-config-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:49:29.695500 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:29.695465    2562 generic.go:356] "Generic (PLEG): container finished" podID="a70777a8-1897-490d-8561-df9374c4030a" containerID="d25c04a71f766bb7e571b2f0e0ad937d806536959f7c4fdf26f080b14e3140fa" exitCode=0
Jul 24 17:49:29.695939 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:29.695511    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6"
Jul 24 17:49:29.695939 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:29.695548    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6" event={"ID":"a70777a8-1897-490d-8561-df9374c4030a","Type":"ContainerDied","Data":"d25c04a71f766bb7e571b2f0e0ad937d806536959f7c4fdf26f080b14e3140fa"}
Jul 24 17:49:29.695939 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:29.695603    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6" event={"ID":"a70777a8-1897-490d-8561-df9374c4030a","Type":"ContainerDied","Data":"c1de6e7e482e829012b510f7ebd2e7b2bf64e2bf1e651428c45386e0831f3a01"}
Jul 24 17:49:29.695939 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:29.695630    2562 scope.go:117] "RemoveContainer" containerID="d25c04a71f766bb7e571b2f0e0ad937d806536959f7c4fdf26f080b14e3140fa"
Jul 24 17:49:29.696839 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:29.696822    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-647df330-795b59b65d-99bm6"
Jul 24 17:49:29.696912 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:29.696830    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-647df330-795b59b65d-99bm6" event={"ID":"51a89701-db56-4d0a-95f5-b0ec15469e8b","Type":"ContainerDied","Data":"51e8437e83a2dd11ffde40f0c28a33d323e1adde7ea807c8a158bda1de51c1ab"}
Jul 24 17:49:29.698163 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:29.698125    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-647df330-7df554ccf7-x5pj2"
Jul 24 17:49:29.698248 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:29.698135    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-647df330-7df554ccf7-x5pj2" event={"ID":"f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2","Type":"ContainerDied","Data":"ceb1ef00ffbd7e4df5a48ecd07bf1d8232690ea405b26bd041711a186076ec4c"}
Jul 24 17:49:29.707206 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:29.707024    2562 scope.go:117] "RemoveContainer" containerID="d25c04a71f766bb7e571b2f0e0ad937d806536959f7c4fdf26f080b14e3140fa"
Jul 24 17:49:29.707520 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:49:29.707491    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d25c04a71f766bb7e571b2f0e0ad937d806536959f7c4fdf26f080b14e3140fa\": container with ID starting with d25c04a71f766bb7e571b2f0e0ad937d806536959f7c4fdf26f080b14e3140fa not found: ID does not exist" containerID="d25c04a71f766bb7e571b2f0e0ad937d806536959f7c4fdf26f080b14e3140fa"
Jul 24 17:49:29.707520 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:29.707529    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d25c04a71f766bb7e571b2f0e0ad937d806536959f7c4fdf26f080b14e3140fa"} err="failed to get container status \"d25c04a71f766bb7e571b2f0e0ad937d806536959f7c4fdf26f080b14e3140fa\": rpc error: code = NotFound desc = could not find container \"d25c04a71f766bb7e571b2f0e0ad937d806536959f7c4fdf26f080b14e3140fa\": container with ID starting with d25c04a71f766bb7e571b2f0e0ad937d806536959f7c4fdf26f080b14e3140fa not found: ID does not exist"
Jul 24 17:49:29.707520 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:29.707553    2562 scope.go:117] "RemoveContainer" containerID="ec10745e2265e8a4118649a7fb252c7e11495e9e39172a8529d47ffe9c4b58ad"
Jul 24 17:49:29.717893 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:29.717875    2562 scope.go:117] "RemoveContainer" containerID="0e9c8601f2f12d526bf31ec0dd613090f0d04f09e51f0a2022f2992125a57aa0"
Jul 24 17:49:29.724380 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:29.724354    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6"]
Jul 24 17:49:29.726805 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:29.726788    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-647df330-7b5c66778d-2fgv6"]
Jul 24 17:49:29.737469 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:29.737449    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-647df330-795b59b65d-99bm6"]
Jul 24 17:49:29.743064 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:29.743044    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-647df330-795b59b65d-99bm6"]
Jul 24 17:49:29.754346 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:29.754327    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-647df330-7df554ccf7-x5pj2"]
Jul 24 17:49:29.758740 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:29.758720    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-647df330-7df554ccf7-x5pj2"]
Jul 24 17:49:29.824997 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:29.824915    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51a89701-db56-4d0a-95f5-b0ec15469e8b" path="/var/lib/kubelet/pods/51a89701-db56-4d0a-95f5-b0ec15469e8b/volumes"
Jul 24 17:49:29.825323 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:29.825310    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a70777a8-1897-490d-8561-df9374c4030a" path="/var/lib/kubelet/pods/a70777a8-1897-490d-8561-df9374c4030a/volumes"
Jul 24 17:49:29.825673 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:29.825654    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2" path="/var/lib/kubelet/pods/f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2/volumes"
Jul 24 17:49:37.308910 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:37.308872    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf"]
Jul 24 17:49:37.309956 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:37.309931    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51a89701-db56-4d0a-95f5-b0ec15469e8b" containerName="payload-processing"
Jul 24 17:49:37.309956 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:37.309958    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="51a89701-db56-4d0a-95f5-b0ec15469e8b" containerName="payload-processing"
Jul 24 17:49:37.310116 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:37.309978    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a70777a8-1897-490d-8561-df9374c4030a" containerName="payload-pre-processing"
Jul 24 17:49:37.310116 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:37.309984    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a70777a8-1897-490d-8561-df9374c4030a" containerName="payload-pre-processing"
Jul 24 17:49:37.310116 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:37.309994    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2" containerName="maas-api"
Jul 24 17:49:37.310116 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:37.309999    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2" containerName="maas-api"
Jul 24 17:49:37.310116 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:37.310104    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="51a89701-db56-4d0a-95f5-b0ec15469e8b" containerName="payload-processing"
Jul 24 17:49:37.310116 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:37.310117    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5fe599c-c0fa-41fe-8b7f-433e6c60d0b2" containerName="maas-api"
Jul 24 17:49:37.310421 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:37.310126    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a70777a8-1897-490d-8561-df9374c4030a" containerName="payload-pre-processing"
Jul 24 17:49:37.315821 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:37.315318    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf"
Jul 24 17:49:37.318383 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:37.318361    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-e7442c1c-serving-cert\""
Jul 24 17:49:37.330242 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:37.330214    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf"]
Jul 24 17:49:37.433538 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:37.433499    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5049dff8-5dfa-417c-9c95-144350a42388-maas-api-tls\") pod \"maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf\" (UID: \"5049dff8-5dfa-417c-9c95-144350a42388\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf"
Jul 24 17:49:37.433724 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:37.433580    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8r95\" (UniqueName: \"kubernetes.io/projected/5049dff8-5dfa-417c-9c95-144350a42388-kube-api-access-h8r95\") pod \"maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf\" (UID: \"5049dff8-5dfa-417c-9c95-144350a42388\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf"
Jul 24 17:49:37.433724 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:37.433640    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5049dff8-5dfa-417c-9c95-144350a42388-openshift-service-ca\") pod \"maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf\" (UID: \"5049dff8-5dfa-417c-9c95-144350a42388\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf"
Jul 24 17:49:37.534831 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:37.534783    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8r95\" (UniqueName: \"kubernetes.io/projected/5049dff8-5dfa-417c-9c95-144350a42388-kube-api-access-h8r95\") pod \"maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf\" (UID: \"5049dff8-5dfa-417c-9c95-144350a42388\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf"
Jul 24 17:49:37.535020 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:37.534897    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5049dff8-5dfa-417c-9c95-144350a42388-openshift-service-ca\") pod \"maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf\" (UID: \"5049dff8-5dfa-417c-9c95-144350a42388\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf"
Jul 24 17:49:37.535020 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:37.534933    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5049dff8-5dfa-417c-9c95-144350a42388-maas-api-tls\") pod \"maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf\" (UID: \"5049dff8-5dfa-417c-9c95-144350a42388\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf"
Jul 24 17:49:37.535188 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:49:37.535102    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-serving-cert: secret "maas-api-e2e-derive-e7442c1c-serving-cert" not found
Jul 24 17:49:37.535250 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:49:37.535209    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5049dff8-5dfa-417c-9c95-144350a42388-maas-api-tls podName:5049dff8-5dfa-417c-9c95-144350a42388 nodeName:}" failed. No retries permitted until 2026-07-24 17:49:38.035185406 +0000 UTC m=+1940.762341608 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5049dff8-5dfa-417c-9c95-144350a42388-maas-api-tls") pod "maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf" (UID: "5049dff8-5dfa-417c-9c95-144350a42388") : secret "maas-api-e2e-derive-e7442c1c-serving-cert" not found
Jul 24 17:49:37.535554 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:37.535535    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5049dff8-5dfa-417c-9c95-144350a42388-openshift-service-ca\") pod \"maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf\" (UID: \"5049dff8-5dfa-417c-9c95-144350a42388\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf"
Jul 24 17:49:37.546099 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:37.546071    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8r95\" (UniqueName: \"kubernetes.io/projected/5049dff8-5dfa-417c-9c95-144350a42388-kube-api-access-h8r95\") pod \"maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf\" (UID: \"5049dff8-5dfa-417c-9c95-144350a42388\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf"
Jul 24 17:49:38.040266 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.040225    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5049dff8-5dfa-417c-9c95-144350a42388-maas-api-tls\") pod \"maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf\" (UID: \"5049dff8-5dfa-417c-9c95-144350a42388\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf"
Jul 24 17:49:38.040459 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:49:38.040382    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-serving-cert: secret "maas-api-e2e-derive-e7442c1c-serving-cert" not found
Jul 24 17:49:38.040505 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:49:38.040461    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5049dff8-5dfa-417c-9c95-144350a42388-maas-api-tls podName:5049dff8-5dfa-417c-9c95-144350a42388 nodeName:}" failed. No retries permitted until 2026-07-24 17:49:39.040444039 +0000 UTC m=+1941.767600236 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5049dff8-5dfa-417c-9c95-144350a42388-maas-api-tls") pod "maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf" (UID: "5049dff8-5dfa-417c-9c95-144350a42388") : secret "maas-api-e2e-derive-e7442c1c-serving-cert" not found
Jul 24 17:49:38.058232 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.058184    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr"]
Jul 24 17:49:38.062040 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.062015    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj"]
Jul 24 17:49:38.062210 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.062164    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr"
Jul 24 17:49:38.065514 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.065492    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-e7442c1c\""
Jul 24 17:49:38.065630 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.065608    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj"
Jul 24 17:49:38.078614 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.078587    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr"]
Jul 24 17:49:38.087822 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.087789    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj"]
Jul 24 17:49:38.141528 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.141487    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bqvl\" (UniqueName: \"kubernetes.io/projected/c15fca4f-c471-4bd6-b620-2d34bd7946e9-kube-api-access-6bqvl\") pod \"payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr\" (UID: \"c15fca4f-c471-4bd6-b620-2d34bd7946e9\") " pod="openshift-ingress/payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr"
Jul 24 17:49:38.141701 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.141606    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c15fca4f-c471-4bd6-b620-2d34bd7946e9-plugins-config-volume\") pod \"payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr\" (UID: \"c15fca4f-c471-4bd6-b620-2d34bd7946e9\") " pod="openshift-ingress/payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr"
Jul 24 17:49:38.242339 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.242301    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c15fca4f-c471-4bd6-b620-2d34bd7946e9-plugins-config-volume\") pod \"payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr\" (UID: \"c15fca4f-c471-4bd6-b620-2d34bd7946e9\") " pod="openshift-ingress/payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr"
Jul 24 17:49:38.242516 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.242393    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bqvl\" (UniqueName: \"kubernetes.io/projected/c15fca4f-c471-4bd6-b620-2d34bd7946e9-kube-api-access-6bqvl\") pod \"payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr\" (UID: \"c15fca4f-c471-4bd6-b620-2d34bd7946e9\") " pod="openshift-ingress/payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr"
Jul 24 17:49:38.242516 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.242435    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2tct\" (UniqueName: \"kubernetes.io/projected/61a62783-6b79-4662-b4ec-9fcc334a41e5-kube-api-access-k2tct\") pod \"payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj\" (UID: \"61a62783-6b79-4662-b4ec-9fcc334a41e5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj"
Jul 24 17:49:38.242516 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.242484    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61a62783-6b79-4662-b4ec-9fcc334a41e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj\" (UID: \"61a62783-6b79-4662-b4ec-9fcc334a41e5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj"
Jul 24 17:49:38.243056 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.243036    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c15fca4f-c471-4bd6-b620-2d34bd7946e9-plugins-config-volume\") pod \"payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr\" (UID: \"c15fca4f-c471-4bd6-b620-2d34bd7946e9\") " pod="openshift-ingress/payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr"
Jul 24 17:49:38.254268 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.254236    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bqvl\" (UniqueName: \"kubernetes.io/projected/c15fca4f-c471-4bd6-b620-2d34bd7946e9-kube-api-access-6bqvl\") pod \"payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr\" (UID: \"c15fca4f-c471-4bd6-b620-2d34bd7946e9\") " pod="openshift-ingress/payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr"
Jul 24 17:49:38.343597 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.343561    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2tct\" (UniqueName: \"kubernetes.io/projected/61a62783-6b79-4662-b4ec-9fcc334a41e5-kube-api-access-k2tct\") pod \"payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj\" (UID: \"61a62783-6b79-4662-b4ec-9fcc334a41e5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj"
Jul 24 17:49:38.344027 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.343617    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61a62783-6b79-4662-b4ec-9fcc334a41e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj\" (UID: \"61a62783-6b79-4662-b4ec-9fcc334a41e5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj"
Jul 24 17:49:38.344367 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.344347    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61a62783-6b79-4662-b4ec-9fcc334a41e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj\" (UID: \"61a62783-6b79-4662-b4ec-9fcc334a41e5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj"
Jul 24 17:49:38.361360 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.361331    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2tct\" (UniqueName: \"kubernetes.io/projected/61a62783-6b79-4662-b4ec-9fcc334a41e5-kube-api-access-k2tct\") pod \"payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj\" (UID: \"61a62783-6b79-4662-b4ec-9fcc334a41e5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj"
Jul 24 17:49:38.374212 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.374185    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr"
Jul 24 17:49:38.382039 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.382017    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj"
Jul 24 17:49:38.559961 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.559930    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr"]
Jul 24 17:49:38.563546 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:49:38.563495    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc15fca4f_c471_4bd6_b620_2d34bd7946e9.slice/crio-3e861a85d3e61647fb7683706b41ac63749d8443c23407c14a62e46445572a1b WatchSource:0}: Error finding container 3e861a85d3e61647fb7683706b41ac63749d8443c23407c14a62e46445572a1b: Status 404 returned error can't find the container with id 3e861a85d3e61647fb7683706b41ac63749d8443c23407c14a62e46445572a1b
Jul 24 17:49:38.592222 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.592189    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj"]
Jul 24 17:49:38.594237 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:49:38.594179    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61a62783_6b79_4662_b4ec_9fcc334a41e5.slice/crio-e5c647abc066db0edfb96a89fdd489a235577b6d7bf58f4530cae189b560c183 WatchSource:0}: Error finding container e5c647abc066db0edfb96a89fdd489a235577b6d7bf58f4530cae189b560c183: Status 404 returned error can't find the container with id e5c647abc066db0edfb96a89fdd489a235577b6d7bf58f4530cae189b560c183
Jul 24 17:49:38.748540 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.748503    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr" event={"ID":"c15fca4f-c471-4bd6-b620-2d34bd7946e9","Type":"ContainerStarted","Data":"d48da0be0377905a2cf081234e34481f0d3bd62d542839c3bdc16919c6cd2fe7"}
Jul 24 17:49:38.748724 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.748562    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr" event={"ID":"c15fca4f-c471-4bd6-b620-2d34bd7946e9","Type":"ContainerStarted","Data":"3e861a85d3e61647fb7683706b41ac63749d8443c23407c14a62e46445572a1b"}
Jul 24 17:49:38.748724 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.748614    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr"
Jul 24 17:49:38.750376 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.750343    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj" event={"ID":"61a62783-6b79-4662-b4ec-9fcc334a41e5","Type":"ContainerStarted","Data":"eef6e05547034b2dd34d2cbf87c814a79a4994332b444083591e1f48b143c12e"}
Jul 24 17:49:38.750499 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.750402    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj" event={"ID":"61a62783-6b79-4662-b4ec-9fcc334a41e5","Type":"ContainerStarted","Data":"e5c647abc066db0edfb96a89fdd489a235577b6d7bf58f4530cae189b560c183"}
Jul 24 17:49:38.750499 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.750434    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj"
Jul 24 17:49:38.767204 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.767157    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr" podStartSLOduration=0.767131619 podStartE2EDuration="767.131619ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:49:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:49:38.764772921 +0000 UTC m=+1941.491929160" watchObservedRunningTime="2026-07-24 17:49:38.767131619 +0000 UTC m=+1941.494287867"
Jul 24 17:49:38.782392 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:38.782343    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj" podStartSLOduration=0.782330548 podStartE2EDuration="782.330548ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:49:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:49:38.782221873 +0000 UTC m=+1941.509378091" watchObservedRunningTime="2026-07-24 17:49:38.782330548 +0000 UTC m=+1941.509486763"
Jul 24 17:49:39.055802 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:39.055699    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5049dff8-5dfa-417c-9c95-144350a42388-maas-api-tls\") pod \"maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf\" (UID: \"5049dff8-5dfa-417c-9c95-144350a42388\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf"
Jul 24 17:49:39.058109 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:39.058084    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5049dff8-5dfa-417c-9c95-144350a42388-maas-api-tls\") pod \"maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf\" (UID: \"5049dff8-5dfa-417c-9c95-144350a42388\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf"
Jul 24 17:49:39.126489 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:39.126453    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf"
Jul 24 17:49:39.264371 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:39.264346    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf"]
Jul 24 17:49:39.265915 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:49:39.265894    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5049dff8_5dfa_417c_9c95_144350a42388.slice/crio-8c744b04c4eb34142e03e6f0fec61899b15f770a5484a1d7f11716f1f20632bd WatchSource:0}: Error finding container 8c744b04c4eb34142e03e6f0fec61899b15f770a5484a1d7f11716f1f20632bd: Status 404 returned error can't find the container with id 8c744b04c4eb34142e03e6f0fec61899b15f770a5484a1d7f11716f1f20632bd
Jul 24 17:49:39.757465 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:39.757429    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf" event={"ID":"5049dff8-5dfa-417c-9c95-144350a42388","Type":"ContainerStarted","Data":"8c744b04c4eb34142e03e6f0fec61899b15f770a5484a1d7f11716f1f20632bd"}
Jul 24 17:49:40.763984 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:40.763946    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf" event={"ID":"5049dff8-5dfa-417c-9c95-144350a42388","Type":"ContainerStarted","Data":"ba7a52ef00078ad512b7bf8fec151d23b81c6bcf5c06facfb3ce2b52742a56a9"}
Jul 24 17:49:40.764432 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:40.764009    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf"
Jul 24 17:49:40.783266 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:40.783218    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf" podStartSLOduration=3.344013664 podStartE2EDuration="3.783203017s" totalImagesPullingTime="439.189353ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:49:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:49:40.781311523 +0000 UTC m=+1943.508467737" watchObservedRunningTime="2026-07-24 17:49:40.783203017 +0000 UTC m=+1943.510359231"
Jul 24 17:49:46.775400 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:46.775369    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf"
Jul 24 17:49:49.760009 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:49.759980    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr"
Jul 24 17:49:49.760480 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:49:49.760169    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj"
Jul 24 17:50:12.907393 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:12.907316    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf"]
Jul 24 17:50:12.907945 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:12.907536    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf" podUID="5049dff8-5dfa-417c-9c95-144350a42388" containerName="maas-api" containerID="cri-o://ba7a52ef00078ad512b7bf8fec151d23b81c6bcf5c06facfb3ce2b52742a56a9" gracePeriod=30
Jul 24 17:50:13.004988 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.004951    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr"]
Jul 24 17:50:13.005255 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.005210    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr" podUID="c15fca4f-c471-4bd6-b620-2d34bd7946e9" containerName="payload-processing" containerID="cri-o://d48da0be0377905a2cf081234e34481f0d3bd62d542839c3bdc16919c6cd2fe7" gracePeriod=30
Jul 24 17:50:13.030235 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.030210    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj"]
Jul 24 17:50:13.030564 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.030537    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj" podUID="61a62783-6b79-4662-b4ec-9fcc334a41e5" containerName="payload-pre-processing" containerID="cri-o://eef6e05547034b2dd34d2cbf87c814a79a4994332b444083591e1f48b143c12e" gracePeriod=30
Jul 24 17:50:13.169859 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.169765    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf"
Jul 24 17:50:13.249721 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.249686    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5049dff8-5dfa-417c-9c95-144350a42388-openshift-service-ca\") pod \"5049dff8-5dfa-417c-9c95-144350a42388\" (UID: \"5049dff8-5dfa-417c-9c95-144350a42388\") "
Jul 24 17:50:13.249887 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.249730    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8r95\" (UniqueName: \"kubernetes.io/projected/5049dff8-5dfa-417c-9c95-144350a42388-kube-api-access-h8r95\") pod \"5049dff8-5dfa-417c-9c95-144350a42388\" (UID: \"5049dff8-5dfa-417c-9c95-144350a42388\") "
Jul 24 17:50:13.249887 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.249767    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5049dff8-5dfa-417c-9c95-144350a42388-maas-api-tls\") pod \"5049dff8-5dfa-417c-9c95-144350a42388\" (UID: \"5049dff8-5dfa-417c-9c95-144350a42388\") "
Jul 24 17:50:13.250208 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.250115    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5049dff8-5dfa-417c-9c95-144350a42388-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5049dff8-5dfa-417c-9c95-144350a42388" (UID: "5049dff8-5dfa-417c-9c95-144350a42388"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:50:13.252327 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.252297    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5049dff8-5dfa-417c-9c95-144350a42388-kube-api-access-h8r95" (OuterVolumeSpecName: "kube-api-access-h8r95") pod "5049dff8-5dfa-417c-9c95-144350a42388" (UID: "5049dff8-5dfa-417c-9c95-144350a42388"). InnerVolumeSpecName "kube-api-access-h8r95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:50:13.254042 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.253998    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5049dff8-5dfa-417c-9c95-144350a42388-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5049dff8-5dfa-417c-9c95-144350a42388" (UID: "5049dff8-5dfa-417c-9c95-144350a42388"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:50:13.290597 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.290564    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr"
Jul 24 17:50:13.300310 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.300290    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj"
Jul 24 17:50:13.350418 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.350389    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c15fca4f-c471-4bd6-b620-2d34bd7946e9-plugins-config-volume\") pod \"c15fca4f-c471-4bd6-b620-2d34bd7946e9\" (UID: \"c15fca4f-c471-4bd6-b620-2d34bd7946e9\") "
Jul 24 17:50:13.350638 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.350485    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6bqvl\" (UniqueName: \"kubernetes.io/projected/c15fca4f-c471-4bd6-b620-2d34bd7946e9-kube-api-access-6bqvl\") pod \"c15fca4f-c471-4bd6-b620-2d34bd7946e9\" (UID: \"c15fca4f-c471-4bd6-b620-2d34bd7946e9\") "
Jul 24 17:50:13.350852 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.350826    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c15fca4f-c471-4bd6-b620-2d34bd7946e9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c15fca4f-c471-4bd6-b620-2d34bd7946e9" (UID: "c15fca4f-c471-4bd6-b620-2d34bd7946e9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:50:13.350954 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.350832    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5049dff8-5dfa-417c-9c95-144350a42388-openshift-service-ca\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:50:13.350954 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.350881    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8r95\" (UniqueName: \"kubernetes.io/projected/5049dff8-5dfa-417c-9c95-144350a42388-kube-api-access-h8r95\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:50:13.350954 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.350902    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5049dff8-5dfa-417c-9c95-144350a42388-maas-api-tls\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:50:13.353196 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.353162    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c15fca4f-c471-4bd6-b620-2d34bd7946e9-kube-api-access-6bqvl" (OuterVolumeSpecName: "kube-api-access-6bqvl") pod "c15fca4f-c471-4bd6-b620-2d34bd7946e9" (UID: "c15fca4f-c471-4bd6-b620-2d34bd7946e9"). InnerVolumeSpecName "kube-api-access-6bqvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:50:13.451380 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.451306    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61a62783-6b79-4662-b4ec-9fcc334a41e5-plugins-config-volume\") pod \"61a62783-6b79-4662-b4ec-9fcc334a41e5\" (UID: \"61a62783-6b79-4662-b4ec-9fcc334a41e5\") "
Jul 24 17:50:13.451380 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.451361    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2tct\" (UniqueName: \"kubernetes.io/projected/61a62783-6b79-4662-b4ec-9fcc334a41e5-kube-api-access-k2tct\") pod \"61a62783-6b79-4662-b4ec-9fcc334a41e5\" (UID: \"61a62783-6b79-4662-b4ec-9fcc334a41e5\") "
Jul 24 17:50:13.451649 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.451630    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c15fca4f-c471-4bd6-b620-2d34bd7946e9-plugins-config-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:50:13.451702 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.451657    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6bqvl\" (UniqueName: \"kubernetes.io/projected/c15fca4f-c471-4bd6-b620-2d34bd7946e9-kube-api-access-6bqvl\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:50:13.451749 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.451731    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61a62783-6b79-4662-b4ec-9fcc334a41e5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "61a62783-6b79-4662-b4ec-9fcc334a41e5" (UID: "61a62783-6b79-4662-b4ec-9fcc334a41e5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:50:13.453476 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.453450    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61a62783-6b79-4662-b4ec-9fcc334a41e5-kube-api-access-k2tct" (OuterVolumeSpecName: "kube-api-access-k2tct") pod "61a62783-6b79-4662-b4ec-9fcc334a41e5" (UID: "61a62783-6b79-4662-b4ec-9fcc334a41e5"). InnerVolumeSpecName "kube-api-access-k2tct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:50:13.552578 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.552542    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61a62783-6b79-4662-b4ec-9fcc334a41e5-plugins-config-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:50:13.552578 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.552570    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2tct\" (UniqueName: \"kubernetes.io/projected/61a62783-6b79-4662-b4ec-9fcc334a41e5-kube-api-access-k2tct\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:50:13.935899 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.935862    2562 generic.go:356] "Generic (PLEG): container finished" podID="5049dff8-5dfa-417c-9c95-144350a42388" containerID="ba7a52ef00078ad512b7bf8fec151d23b81c6bcf5c06facfb3ce2b52742a56a9" exitCode=0
Jul 24 17:50:13.936344 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.935907    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf" event={"ID":"5049dff8-5dfa-417c-9c95-144350a42388","Type":"ContainerDied","Data":"ba7a52ef00078ad512b7bf8fec151d23b81c6bcf5c06facfb3ce2b52742a56a9"}
Jul 24 17:50:13.936344 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.935928    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf"
Jul 24 17:50:13.936344 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.935972    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf" event={"ID":"5049dff8-5dfa-417c-9c95-144350a42388","Type":"ContainerDied","Data":"8c744b04c4eb34142e03e6f0fec61899b15f770a5484a1d7f11716f1f20632bd"}
Jul 24 17:50:13.936344 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.936005    2562 scope.go:117] "RemoveContainer" containerID="ba7a52ef00078ad512b7bf8fec151d23b81c6bcf5c06facfb3ce2b52742a56a9"
Jul 24 17:50:13.937463 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.937442    2562 generic.go:356] "Generic (PLEG): container finished" podID="c15fca4f-c471-4bd6-b620-2d34bd7946e9" containerID="d48da0be0377905a2cf081234e34481f0d3bd62d542839c3bdc16919c6cd2fe7" exitCode=0
Jul 24 17:50:13.937539 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.937505    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr"
Jul 24 17:50:13.937539 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.937506    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr" event={"ID":"c15fca4f-c471-4bd6-b620-2d34bd7946e9","Type":"ContainerDied","Data":"d48da0be0377905a2cf081234e34481f0d3bd62d542839c3bdc16919c6cd2fe7"}
Jul 24 17:50:13.937645 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.937592    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr" event={"ID":"c15fca4f-c471-4bd6-b620-2d34bd7946e9","Type":"ContainerDied","Data":"3e861a85d3e61647fb7683706b41ac63749d8443c23407c14a62e46445572a1b"}
Jul 24 17:50:13.939001 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.938980    2562 generic.go:356] "Generic (PLEG): container finished" podID="61a62783-6b79-4662-b4ec-9fcc334a41e5" containerID="eef6e05547034b2dd34d2cbf87c814a79a4994332b444083591e1f48b143c12e" exitCode=0
Jul 24 17:50:13.939097 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.939011    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj" event={"ID":"61a62783-6b79-4662-b4ec-9fcc334a41e5","Type":"ContainerDied","Data":"eef6e05547034b2dd34d2cbf87c814a79a4994332b444083591e1f48b143c12e"}
Jul 24 17:50:13.939097 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.939035    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj"
Jul 24 17:50:13.939097 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.939068    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj" event={"ID":"61a62783-6b79-4662-b4ec-9fcc334a41e5","Type":"ContainerDied","Data":"e5c647abc066db0edfb96a89fdd489a235577b6d7bf58f4530cae189b560c183"}
Jul 24 17:50:13.946208 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.946177    2562 scope.go:117] "RemoveContainer" containerID="ba7a52ef00078ad512b7bf8fec151d23b81c6bcf5c06facfb3ce2b52742a56a9"
Jul 24 17:50:13.946539 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:50:13.946513    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba7a52ef00078ad512b7bf8fec151d23b81c6bcf5c06facfb3ce2b52742a56a9\": container with ID starting with ba7a52ef00078ad512b7bf8fec151d23b81c6bcf5c06facfb3ce2b52742a56a9 not found: ID does not exist" containerID="ba7a52ef00078ad512b7bf8fec151d23b81c6bcf5c06facfb3ce2b52742a56a9"
Jul 24 17:50:13.946642 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.946549    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba7a52ef00078ad512b7bf8fec151d23b81c6bcf5c06facfb3ce2b52742a56a9"} err="failed to get container status \"ba7a52ef00078ad512b7bf8fec151d23b81c6bcf5c06facfb3ce2b52742a56a9\": rpc error: code = NotFound desc = could not find container \"ba7a52ef00078ad512b7bf8fec151d23b81c6bcf5c06facfb3ce2b52742a56a9\": container with ID starting with ba7a52ef00078ad512b7bf8fec151d23b81c6bcf5c06facfb3ce2b52742a56a9 not found: ID does not exist"
Jul 24 17:50:13.946642 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.946570    2562 scope.go:117] "RemoveContainer" containerID="d48da0be0377905a2cf081234e34481f0d3bd62d542839c3bdc16919c6cd2fe7"
Jul 24 17:50:13.954496 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.954477    2562 scope.go:117] "RemoveContainer" containerID="d48da0be0377905a2cf081234e34481f0d3bd62d542839c3bdc16919c6cd2fe7"
Jul 24 17:50:13.954749 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:50:13.954729    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d48da0be0377905a2cf081234e34481f0d3bd62d542839c3bdc16919c6cd2fe7\": container with ID starting with d48da0be0377905a2cf081234e34481f0d3bd62d542839c3bdc16919c6cd2fe7 not found: ID does not exist" containerID="d48da0be0377905a2cf081234e34481f0d3bd62d542839c3bdc16919c6cd2fe7"
Jul 24 17:50:13.954808 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.954757    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d48da0be0377905a2cf081234e34481f0d3bd62d542839c3bdc16919c6cd2fe7"} err="failed to get container status \"d48da0be0377905a2cf081234e34481f0d3bd62d542839c3bdc16919c6cd2fe7\": rpc error: code = NotFound desc = could not find container \"d48da0be0377905a2cf081234e34481f0d3bd62d542839c3bdc16919c6cd2fe7\": container with ID starting with d48da0be0377905a2cf081234e34481f0d3bd62d542839c3bdc16919c6cd2fe7 not found: ID does not exist"
Jul 24 17:50:13.954808 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.954773    2562 scope.go:117] "RemoveContainer" containerID="eef6e05547034b2dd34d2cbf87c814a79a4994332b444083591e1f48b143c12e"
Jul 24 17:50:13.962095 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.962080    2562 scope.go:117] "RemoveContainer" containerID="eef6e05547034b2dd34d2cbf87c814a79a4994332b444083591e1f48b143c12e"
Jul 24 17:50:13.962382 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:50:13.962363    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eef6e05547034b2dd34d2cbf87c814a79a4994332b444083591e1f48b143c12e\": container with ID starting with eef6e05547034b2dd34d2cbf87c814a79a4994332b444083591e1f48b143c12e not found: ID does not exist" containerID="eef6e05547034b2dd34d2cbf87c814a79a4994332b444083591e1f48b143c12e"
Jul 24 17:50:13.962444 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.962389    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eef6e05547034b2dd34d2cbf87c814a79a4994332b444083591e1f48b143c12e"} err="failed to get container status \"eef6e05547034b2dd34d2cbf87c814a79a4994332b444083591e1f48b143c12e\": rpc error: code = NotFound desc = could not find container \"eef6e05547034b2dd34d2cbf87c814a79a4994332b444083591e1f48b143c12e\": container with ID starting with eef6e05547034b2dd34d2cbf87c814a79a4994332b444083591e1f48b143c12e not found: ID does not exist"
Jul 24 17:50:13.969524 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.969505    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf"]
Jul 24 17:50:13.976421 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.976395    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e7442c1c-55bd965446-r7cgf"]
Jul 24 17:50:13.990735 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.990707    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj"]
Jul 24 17:50:13.997930 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:13.997910    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-e7442c1c-76679c6cf6-c4sdj"]
Jul 24 17:50:14.011527 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:14.011505    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr"]
Jul 24 17:50:14.013020 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:14.013004    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-e7442c1c-7dfdb788bb-46sgr"]
Jul 24 17:50:15.824929 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:15.824898    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5049dff8-5dfa-417c-9c95-144350a42388" path="/var/lib/kubelet/pods/5049dff8-5dfa-417c-9c95-144350a42388/volumes"
Jul 24 17:50:15.827370 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:15.825273    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61a62783-6b79-4662-b4ec-9fcc334a41e5" path="/var/lib/kubelet/pods/61a62783-6b79-4662-b4ec-9fcc334a41e5/volumes"
Jul 24 17:50:15.827370 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:50:15.825580    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c15fca4f-c471-4bd6-b620-2d34bd7946e9" path="/var/lib/kubelet/pods/c15fca4f-c471-4bd6-b620-2d34bd7946e9/volumes"
Jul 24 17:52:08.132766 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:08.132673    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-8557fb8557-brbdk"]
Jul 24 17:52:08.133374 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:08.133302    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c15fca4f-c471-4bd6-b620-2d34bd7946e9" containerName="payload-processing"
Jul 24 17:52:08.133374 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:08.133323    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c15fca4f-c471-4bd6-b620-2d34bd7946e9" containerName="payload-processing"
Jul 24 17:52:08.133374 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:08.133335    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61a62783-6b79-4662-b4ec-9fcc334a41e5" containerName="payload-pre-processing"
Jul 24 17:52:08.133374 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:08.133344    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="61a62783-6b79-4662-b4ec-9fcc334a41e5" containerName="payload-pre-processing"
Jul 24 17:52:08.133596 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:08.133382    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5049dff8-5dfa-417c-9c95-144350a42388" containerName="maas-api"
Jul 24 17:52:08.133596 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:08.133391    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5049dff8-5dfa-417c-9c95-144350a42388" containerName="maas-api"
Jul 24 17:52:08.133596 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:08.133507    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c15fca4f-c471-4bd6-b620-2d34bd7946e9" containerName="payload-processing"
Jul 24 17:52:08.133596 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:08.133526    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="61a62783-6b79-4662-b4ec-9fcc334a41e5" containerName="payload-pre-processing"
Jul 24 17:52:08.133596 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:08.133536    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5049dff8-5dfa-417c-9c95-144350a42388" containerName="maas-api"
Jul 24 17:52:08.141707 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:08.141666    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-8557fb8557-brbdk"
Jul 24 17:52:08.144713 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:08.144680    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-117b3f04-serving-cert\""
Jul 24 17:52:08.170768 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:08.170739    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-8557fb8557-brbdk"]
Jul 24 17:52:08.180785 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:08.180755    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6892fd83-cd5e-41ee-a993-d62ccdd323ee-openshift-service-ca\") pod \"maas-api-e2e-mt-117b3f04-8557fb8557-brbdk\" (UID: \"6892fd83-cd5e-41ee-a993-d62ccdd323ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-8557fb8557-brbdk"
Jul 24 17:52:08.180948 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:08.180809    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6892fd83-cd5e-41ee-a993-d62ccdd323ee-maas-api-tls\") pod \"maas-api-e2e-mt-117b3f04-8557fb8557-brbdk\" (UID: \"6892fd83-cd5e-41ee-a993-d62ccdd323ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-8557fb8557-brbdk"
Jul 24 17:52:08.180948 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:08.180869    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-588q8\" (UniqueName: \"kubernetes.io/projected/6892fd83-cd5e-41ee-a993-d62ccdd323ee-kube-api-access-588q8\") pod \"maas-api-e2e-mt-117b3f04-8557fb8557-brbdk\" (UID: \"6892fd83-cd5e-41ee-a993-d62ccdd323ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-8557fb8557-brbdk"
Jul 24 17:52:08.281804 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:08.281767    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-588q8\" (UniqueName: \"kubernetes.io/projected/6892fd83-cd5e-41ee-a993-d62ccdd323ee-kube-api-access-588q8\") pod \"maas-api-e2e-mt-117b3f04-8557fb8557-brbdk\" (UID: \"6892fd83-cd5e-41ee-a993-d62ccdd323ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-8557fb8557-brbdk"
Jul 24 17:52:08.282045 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:08.281855    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6892fd83-cd5e-41ee-a993-d62ccdd323ee-openshift-service-ca\") pod \"maas-api-e2e-mt-117b3f04-8557fb8557-brbdk\" (UID: \"6892fd83-cd5e-41ee-a993-d62ccdd323ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-8557fb8557-brbdk"
Jul 24 17:52:08.282045 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:08.281922    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6892fd83-cd5e-41ee-a993-d62ccdd323ee-maas-api-tls\") pod \"maas-api-e2e-mt-117b3f04-8557fb8557-brbdk\" (UID: \"6892fd83-cd5e-41ee-a993-d62ccdd323ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-8557fb8557-brbdk"
Jul 24 17:52:08.282135 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:52:08.282097    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-serving-cert: secret "maas-api-e2e-mt-117b3f04-serving-cert" not found
Jul 24 17:52:08.282201 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:52:08.282195    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6892fd83-cd5e-41ee-a993-d62ccdd323ee-maas-api-tls podName:6892fd83-cd5e-41ee-a993-d62ccdd323ee nodeName:}" failed. No retries permitted until 2026-07-24 17:52:08.782173554 +0000 UTC m=+2091.509329749 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6892fd83-cd5e-41ee-a993-d62ccdd323ee-maas-api-tls") pod "maas-api-e2e-mt-117b3f04-8557fb8557-brbdk" (UID: "6892fd83-cd5e-41ee-a993-d62ccdd323ee") : secret "maas-api-e2e-mt-117b3f04-serving-cert" not found
Jul 24 17:52:08.282566 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:08.282544    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6892fd83-cd5e-41ee-a993-d62ccdd323ee-openshift-service-ca\") pod \"maas-api-e2e-mt-117b3f04-8557fb8557-brbdk\" (UID: \"6892fd83-cd5e-41ee-a993-d62ccdd323ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-8557fb8557-brbdk"
Jul 24 17:52:08.303680 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:08.303643    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-588q8\" (UniqueName: \"kubernetes.io/projected/6892fd83-cd5e-41ee-a993-d62ccdd323ee-kube-api-access-588q8\") pod \"maas-api-e2e-mt-117b3f04-8557fb8557-brbdk\" (UID: \"6892fd83-cd5e-41ee-a993-d62ccdd323ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-8557fb8557-brbdk"
Jul 24 17:52:08.786900 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:08.786849    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6892fd83-cd5e-41ee-a993-d62ccdd323ee-maas-api-tls\") pod \"maas-api-e2e-mt-117b3f04-8557fb8557-brbdk\" (UID: \"6892fd83-cd5e-41ee-a993-d62ccdd323ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-8557fb8557-brbdk"
Jul 24 17:52:08.789475 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:08.789450    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6892fd83-cd5e-41ee-a993-d62ccdd323ee-maas-api-tls\") pod \"maas-api-e2e-mt-117b3f04-8557fb8557-brbdk\" (UID: \"6892fd83-cd5e-41ee-a993-d62ccdd323ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-8557fb8557-brbdk"
Jul 24 17:52:09.053865 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:09.053763    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-8557fb8557-brbdk"
Jul 24 17:52:09.217008 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:09.216968    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-8557fb8557-brbdk"]
Jul 24 17:52:09.225743 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:52:09.225704    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6892fd83_cd5e_41ee_a993_d62ccdd323ee.slice/crio-aac836327c156332bd4e2f292f594d51d770a6f07f98839df1dcbac2014f07c9 WatchSource:0}: Error finding container aac836327c156332bd4e2f292f594d51d770a6f07f98839df1dcbac2014f07c9: Status 404 returned error can't find the container with id aac836327c156332bd4e2f292f594d51d770a6f07f98839df1dcbac2014f07c9
Jul 24 17:52:09.501535 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:09.501495    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-8557fb8557-brbdk" event={"ID":"6892fd83-cd5e-41ee-a993-d62ccdd323ee","Type":"ContainerStarted","Data":"aac836327c156332bd4e2f292f594d51d770a6f07f98839df1dcbac2014f07c9"}
Jul 24 17:52:09.525923 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:09.525880    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6"]
Jul 24 17:52:09.530720 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:09.530689    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6"
Jul 24 17:52:09.533407 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:09.533378    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-117b3f04-dockercfg-7l2jd\""
Jul 24 17:52:09.536721 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:09.536701    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-117b3f04\""
Jul 24 17:52:09.548400 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:09.548366    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6"]
Jul 24 17:52:09.586907 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:09.586854    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l"]
Jul 24 17:52:09.590654 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:09.590634    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l"
Jul 24 17:52:09.595208 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:09.595177    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6\" (UID: \"a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6"
Jul 24 17:52:09.595377 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:09.595220    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twhfz\" (UniqueName: \"kubernetes.io/projected/a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5-kube-api-access-twhfz\") pod \"payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6\" (UID: \"a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6"
Jul 24 17:52:09.615035 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:09.615005    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l"]
Jul 24 17:52:09.696030 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:09.695995    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95ee5d00-72b8-43cf-bf55-1c73c964827b-plugins-config-volume\") pod \"payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l\" (UID: \"95ee5d00-72b8-43cf-bf55-1c73c964827b\") " pod="openshift-ingress/payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l"
Jul 24 17:52:09.696272 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:09.696045    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrfxt\" (UniqueName: \"kubernetes.io/projected/95ee5d00-72b8-43cf-bf55-1c73c964827b-kube-api-access-hrfxt\") pod \"payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l\" (UID: \"95ee5d00-72b8-43cf-bf55-1c73c964827b\") " pod="openshift-ingress/payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l"
Jul 24 17:52:09.696272 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:09.696083    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6\" (UID: \"a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6"
Jul 24 17:52:09.696272 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:09.696136    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twhfz\" (UniqueName: \"kubernetes.io/projected/a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5-kube-api-access-twhfz\") pod \"payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6\" (UID: \"a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6"
Jul 24 17:52:09.696812 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:09.696791    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6\" (UID: \"a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6"
Jul 24 17:52:09.709269 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:09.709240    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twhfz\" (UniqueName: \"kubernetes.io/projected/a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5-kube-api-access-twhfz\") pod \"payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6\" (UID: \"a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6"
Jul 24 17:52:09.797468 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:09.797437    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95ee5d00-72b8-43cf-bf55-1c73c964827b-plugins-config-volume\") pod \"payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l\" (UID: \"95ee5d00-72b8-43cf-bf55-1c73c964827b\") " pod="openshift-ingress/payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l"
Jul 24 17:52:09.797617 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:09.797483    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrfxt\" (UniqueName: \"kubernetes.io/projected/95ee5d00-72b8-43cf-bf55-1c73c964827b-kube-api-access-hrfxt\") pod \"payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l\" (UID: \"95ee5d00-72b8-43cf-bf55-1c73c964827b\") " pod="openshift-ingress/payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l"
Jul 24 17:52:09.798309 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:09.798169    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95ee5d00-72b8-43cf-bf55-1c73c964827b-plugins-config-volume\") pod \"payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l\" (UID: \"95ee5d00-72b8-43cf-bf55-1c73c964827b\") " pod="openshift-ingress/payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l"
Jul 24 17:52:09.809480 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:09.809448    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrfxt\" (UniqueName: \"kubernetes.io/projected/95ee5d00-72b8-43cf-bf55-1c73c964827b-kube-api-access-hrfxt\") pod \"payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l\" (UID: \"95ee5d00-72b8-43cf-bf55-1c73c964827b\") " pod="openshift-ingress/payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l"
Jul 24 17:52:09.842115 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:09.842061    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6"
Jul 24 17:52:09.903183 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:09.903044    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l"
Jul 24 17:52:10.075118 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:52:10.075088    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95ee5d00_72b8_43cf_bf55_1c73c964827b.slice/crio-bf6b72c28494ffbf6400d403b3e08fec5349102f05dc3b910f1dbb18f1fb63e7 WatchSource:0}: Error finding container bf6b72c28494ffbf6400d403b3e08fec5349102f05dc3b910f1dbb18f1fb63e7: Status 404 returned error can't find the container with id bf6b72c28494ffbf6400d403b3e08fec5349102f05dc3b910f1dbb18f1fb63e7
Jul 24 17:52:10.075726 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:10.075702    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l"]
Jul 24 17:52:10.245927 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:10.245890    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6"]
Jul 24 17:52:10.248680 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:52:10.248644    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8ff7a7a_a4ac_4563_b4c8_8ee63280b5e5.slice/crio-020e6acb2239596286f1822694115913fa7e0ee7fcca2fec6e8a3fb194488933 WatchSource:0}: Error finding container 020e6acb2239596286f1822694115913fa7e0ee7fcca2fec6e8a3fb194488933: Status 404 returned error can't find the container with id 020e6acb2239596286f1822694115913fa7e0ee7fcca2fec6e8a3fb194488933
Jul 24 17:52:10.509987 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:10.509862    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-8557fb8557-brbdk" event={"ID":"6892fd83-cd5e-41ee-a993-d62ccdd323ee","Type":"ContainerStarted","Data":"d14528e193e55801eab5af5db6468455a2108af74981a150d0ef70e8d673a5b8"}
Jul 24 17:52:10.510218 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:10.510073    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-8557fb8557-brbdk"
Jul 24 17:52:10.511755 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:10.511721    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l" event={"ID":"95ee5d00-72b8-43cf-bf55-1c73c964827b","Type":"ContainerStarted","Data":"3a8ed0faf36c3e779379f72966e4dcc85ff8d030c5aab2ff5e81efe86d85df45"}
Jul 24 17:52:10.511893 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:10.511774    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l" event={"ID":"95ee5d00-72b8-43cf-bf55-1c73c964827b","Type":"ContainerStarted","Data":"bf6b72c28494ffbf6400d403b3e08fec5349102f05dc3b910f1dbb18f1fb63e7"}
Jul 24 17:52:10.511893 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:10.511802    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l"
Jul 24 17:52:10.517220 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:10.517186    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6" event={"ID":"a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5","Type":"ContainerStarted","Data":"90ca5c4fcef870ed7a73b2d4f6e89be0689369da773332acdbf6b8d13fabb17f"}
Jul 24 17:52:10.517413 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:10.517249    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6" event={"ID":"a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5","Type":"ContainerStarted","Data":"020e6acb2239596286f1822694115913fa7e0ee7fcca2fec6e8a3fb194488933"}
Jul 24 17:52:10.517413 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:10.517303    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6"
Jul 24 17:52:10.590867 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:10.590807    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-8557fb8557-brbdk" podStartSLOduration=2.024091216 podStartE2EDuration="2.590790991s" totalImagesPullingTime="566.699775ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:52:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:52:10.588462161 +0000 UTC m=+2093.315618384" watchObservedRunningTime="2026-07-24 17:52:10.590790991 +0000 UTC m=+2093.317947205"
Jul 24 17:52:10.689213 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:10.689123    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l" podStartSLOduration=1.6891048560000002 podStartE2EDuration="1.689104856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:52:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:52:10.681219687 +0000 UTC m=+2093.408375901" watchObservedRunningTime="2026-07-24 17:52:10.689104856 +0000 UTC m=+2093.416261070"
Jul 24 17:52:10.690130 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:10.690097    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6" podStartSLOduration=1.690088153 podStartE2EDuration="1.690088153s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:52:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:52:10.632328996 +0000 UTC m=+2093.359485210" watchObservedRunningTime="2026-07-24 17:52:10.690088153 +0000 UTC m=+2093.417244369"
Jul 24 17:52:16.527940 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:16.527907    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-8557fb8557-brbdk"
Jul 24 17:52:21.524666 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:21.524628    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l"
Jul 24 17:52:21.525054 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:21.524939    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6"
Jul 24 17:52:31.683411 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:31.683371    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-8557fb8557-brbdk"]
Jul 24 17:52:31.683875 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:31.683639    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-8557fb8557-brbdk" podUID="6892fd83-cd5e-41ee-a993-d62ccdd323ee" containerName="maas-api" containerID="cri-o://d14528e193e55801eab5af5db6468455a2108af74981a150d0ef70e8d673a5b8" gracePeriod=30
Jul 24 17:52:31.760873 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:31.760841    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l"]
Jul 24 17:52:31.761118 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:31.761093    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l" podUID="95ee5d00-72b8-43cf-bf55-1c73c964827b" containerName="payload-processing" containerID="cri-o://3a8ed0faf36c3e779379f72966e4dcc85ff8d030c5aab2ff5e81efe86d85df45" gracePeriod=30
Jul 24 17:52:31.770248 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:31.770223    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6"]
Jul 24 17:52:31.770478 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:31.770428    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6" podUID="a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5" containerName="payload-pre-processing" containerID="cri-o://90ca5c4fcef870ed7a73b2d4f6e89be0689369da773332acdbf6b8d13fabb17f" gracePeriod=30
Jul 24 17:52:32.047360 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.047331    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l"
Jul 24 17:52:32.060580 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.060557    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6"
Jul 24 17:52:32.070690 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.070661    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-8557fb8557-brbdk"
Jul 24 17:52:32.195748 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.195659    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6892fd83-cd5e-41ee-a993-d62ccdd323ee-openshift-service-ca\") pod \"6892fd83-cd5e-41ee-a993-d62ccdd323ee\" (UID: \"6892fd83-cd5e-41ee-a993-d62ccdd323ee\") "
Jul 24 17:52:32.195748 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.195714    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hrfxt\" (UniqueName: \"kubernetes.io/projected/95ee5d00-72b8-43cf-bf55-1c73c964827b-kube-api-access-hrfxt\") pod \"95ee5d00-72b8-43cf-bf55-1c73c964827b\" (UID: \"95ee5d00-72b8-43cf-bf55-1c73c964827b\") "
Jul 24 17:52:32.195748 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.195747    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-588q8\" (UniqueName: \"kubernetes.io/projected/6892fd83-cd5e-41ee-a993-d62ccdd323ee-kube-api-access-588q8\") pod \"6892fd83-cd5e-41ee-a993-d62ccdd323ee\" (UID: \"6892fd83-cd5e-41ee-a993-d62ccdd323ee\") "
Jul 24 17:52:32.196022 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.195813    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5-plugins-config-volume\") pod \"a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5\" (UID: \"a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5\") "
Jul 24 17:52:32.196022 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.195841    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twhfz\" (UniqueName: \"kubernetes.io/projected/a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5-kube-api-access-twhfz\") pod \"a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5\" (UID: \"a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5\") "
Jul 24 17:52:32.196022 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.195859    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95ee5d00-72b8-43cf-bf55-1c73c964827b-plugins-config-volume\") pod \"95ee5d00-72b8-43cf-bf55-1c73c964827b\" (UID: \"95ee5d00-72b8-43cf-bf55-1c73c964827b\") "
Jul 24 17:52:32.196022 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.195902    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6892fd83-cd5e-41ee-a993-d62ccdd323ee-maas-api-tls\") pod \"6892fd83-cd5e-41ee-a993-d62ccdd323ee\" (UID: \"6892fd83-cd5e-41ee-a993-d62ccdd323ee\") "
Jul 24 17:52:32.196345 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.196301    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95ee5d00-72b8-43cf-bf55-1c73c964827b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "95ee5d00-72b8-43cf-bf55-1c73c964827b" (UID: "95ee5d00-72b8-43cf-bf55-1c73c964827b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:52:32.196345 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.196317    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5" (UID: "a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:52:32.196477 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.196419    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6892fd83-cd5e-41ee-a993-d62ccdd323ee-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6892fd83-cd5e-41ee-a993-d62ccdd323ee" (UID: "6892fd83-cd5e-41ee-a993-d62ccdd323ee"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:52:32.198074 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.198053    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6892fd83-cd5e-41ee-a993-d62ccdd323ee-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6892fd83-cd5e-41ee-a993-d62ccdd323ee" (UID: "6892fd83-cd5e-41ee-a993-d62ccdd323ee"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:52:32.198074 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.198063    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95ee5d00-72b8-43cf-bf55-1c73c964827b-kube-api-access-hrfxt" (OuterVolumeSpecName: "kube-api-access-hrfxt") pod "95ee5d00-72b8-43cf-bf55-1c73c964827b" (UID: "95ee5d00-72b8-43cf-bf55-1c73c964827b"). InnerVolumeSpecName "kube-api-access-hrfxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:52:32.198371 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.198340    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5-kube-api-access-twhfz" (OuterVolumeSpecName: "kube-api-access-twhfz") pod "a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5" (UID: "a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5"). InnerVolumeSpecName "kube-api-access-twhfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:52:32.198371 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.198344    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6892fd83-cd5e-41ee-a993-d62ccdd323ee-kube-api-access-588q8" (OuterVolumeSpecName: "kube-api-access-588q8") pod "6892fd83-cd5e-41ee-a993-d62ccdd323ee" (UID: "6892fd83-cd5e-41ee-a993-d62ccdd323ee"). InnerVolumeSpecName "kube-api-access-588q8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:52:32.297446 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.297402    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5-plugins-config-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:52:32.297446 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.297435    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twhfz\" (UniqueName: \"kubernetes.io/projected/a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5-kube-api-access-twhfz\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:52:32.297446 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.297445    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95ee5d00-72b8-43cf-bf55-1c73c964827b-plugins-config-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:52:32.297446 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.297455    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6892fd83-cd5e-41ee-a993-d62ccdd323ee-maas-api-tls\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:52:32.297711 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.297465    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6892fd83-cd5e-41ee-a993-d62ccdd323ee-openshift-service-ca\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:52:32.297711 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.297473    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrfxt\" (UniqueName: \"kubernetes.io/projected/95ee5d00-72b8-43cf-bf55-1c73c964827b-kube-api-access-hrfxt\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:52:32.297711 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.297483    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-588q8\" (UniqueName: \"kubernetes.io/projected/6892fd83-cd5e-41ee-a993-d62ccdd323ee-kube-api-access-588q8\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:52:32.627944 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.627913    2562 generic.go:356] "Generic (PLEG): container finished" podID="6892fd83-cd5e-41ee-a993-d62ccdd323ee" containerID="d14528e193e55801eab5af5db6468455a2108af74981a150d0ef70e8d673a5b8" exitCode=0
Jul 24 17:52:32.628107 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.627957    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-8557fb8557-brbdk"
Jul 24 17:52:32.628107 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.627994    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-8557fb8557-brbdk" event={"ID":"6892fd83-cd5e-41ee-a993-d62ccdd323ee","Type":"ContainerDied","Data":"d14528e193e55801eab5af5db6468455a2108af74981a150d0ef70e8d673a5b8"}
Jul 24 17:52:32.628107 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.628056    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-8557fb8557-brbdk" event={"ID":"6892fd83-cd5e-41ee-a993-d62ccdd323ee","Type":"ContainerDied","Data":"aac836327c156332bd4e2f292f594d51d770a6f07f98839df1dcbac2014f07c9"}
Jul 24 17:52:32.628107 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.628085    2562 scope.go:117] "RemoveContainer" containerID="d14528e193e55801eab5af5db6468455a2108af74981a150d0ef70e8d673a5b8"
Jul 24 17:52:32.629377 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.629351    2562 generic.go:356] "Generic (PLEG): container finished" podID="95ee5d00-72b8-43cf-bf55-1c73c964827b" containerID="3a8ed0faf36c3e779379f72966e4dcc85ff8d030c5aab2ff5e81efe86d85df45" exitCode=0
Jul 24 17:52:32.629452 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.629385    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l"
Jul 24 17:52:32.629513 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.629441    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l" event={"ID":"95ee5d00-72b8-43cf-bf55-1c73c964827b","Type":"ContainerDied","Data":"3a8ed0faf36c3e779379f72966e4dcc85ff8d030c5aab2ff5e81efe86d85df45"}
Jul 24 17:52:32.629513 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.629490    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l" event={"ID":"95ee5d00-72b8-43cf-bf55-1c73c964827b","Type":"ContainerDied","Data":"bf6b72c28494ffbf6400d403b3e08fec5349102f05dc3b910f1dbb18f1fb63e7"}
Jul 24 17:52:32.630916 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.630894    2562 generic.go:356] "Generic (PLEG): container finished" podID="a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5" containerID="90ca5c4fcef870ed7a73b2d4f6e89be0689369da773332acdbf6b8d13fabb17f" exitCode=0
Jul 24 17:52:32.631003 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.630941    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6"
Jul 24 17:52:32.631003 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.630980    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6" event={"ID":"a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5","Type":"ContainerDied","Data":"90ca5c4fcef870ed7a73b2d4f6e89be0689369da773332acdbf6b8d13fabb17f"}
Jul 24 17:52:32.631085 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.631032    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6" event={"ID":"a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5","Type":"ContainerDied","Data":"020e6acb2239596286f1822694115913fa7e0ee7fcca2fec6e8a3fb194488933"}
Jul 24 17:52:32.637573 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.637542    2562 scope.go:117] "RemoveContainer" containerID="d14528e193e55801eab5af5db6468455a2108af74981a150d0ef70e8d673a5b8"
Jul 24 17:52:32.637867 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:52:32.637844    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d14528e193e55801eab5af5db6468455a2108af74981a150d0ef70e8d673a5b8\": container with ID starting with d14528e193e55801eab5af5db6468455a2108af74981a150d0ef70e8d673a5b8 not found: ID does not exist" containerID="d14528e193e55801eab5af5db6468455a2108af74981a150d0ef70e8d673a5b8"
Jul 24 17:52:32.637972 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.637928    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d14528e193e55801eab5af5db6468455a2108af74981a150d0ef70e8d673a5b8"} err="failed to get container status \"d14528e193e55801eab5af5db6468455a2108af74981a150d0ef70e8d673a5b8\": rpc error: code = NotFound desc = could not find container \"d14528e193e55801eab5af5db6468455a2108af74981a150d0ef70e8d673a5b8\": container with ID starting with d14528e193e55801eab5af5db6468455a2108af74981a150d0ef70e8d673a5b8 not found: ID does not exist"
Jul 24 17:52:32.637972 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.637968    2562 scope.go:117] "RemoveContainer" containerID="3a8ed0faf36c3e779379f72966e4dcc85ff8d030c5aab2ff5e81efe86d85df45"
Jul 24 17:52:32.647090 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.647073    2562 scope.go:117] "RemoveContainer" containerID="3a8ed0faf36c3e779379f72966e4dcc85ff8d030c5aab2ff5e81efe86d85df45"
Jul 24 17:52:32.647399 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:52:32.647375    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a8ed0faf36c3e779379f72966e4dcc85ff8d030c5aab2ff5e81efe86d85df45\": container with ID starting with 3a8ed0faf36c3e779379f72966e4dcc85ff8d030c5aab2ff5e81efe86d85df45 not found: ID does not exist" containerID="3a8ed0faf36c3e779379f72966e4dcc85ff8d030c5aab2ff5e81efe86d85df45"
Jul 24 17:52:32.647499 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.647403    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a8ed0faf36c3e779379f72966e4dcc85ff8d030c5aab2ff5e81efe86d85df45"} err="failed to get container status \"3a8ed0faf36c3e779379f72966e4dcc85ff8d030c5aab2ff5e81efe86d85df45\": rpc error: code = NotFound desc = could not find container \"3a8ed0faf36c3e779379f72966e4dcc85ff8d030c5aab2ff5e81efe86d85df45\": container with ID starting with 3a8ed0faf36c3e779379f72966e4dcc85ff8d030c5aab2ff5e81efe86d85df45 not found: ID does not exist"
Jul 24 17:52:32.647499 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.647418    2562 scope.go:117] "RemoveContainer" containerID="90ca5c4fcef870ed7a73b2d4f6e89be0689369da773332acdbf6b8d13fabb17f"
Jul 24 17:52:32.655229 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.655210    2562 scope.go:117] "RemoveContainer" containerID="90ca5c4fcef870ed7a73b2d4f6e89be0689369da773332acdbf6b8d13fabb17f"
Jul 24 17:52:32.655480 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:52:32.655461    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90ca5c4fcef870ed7a73b2d4f6e89be0689369da773332acdbf6b8d13fabb17f\": container with ID starting with 90ca5c4fcef870ed7a73b2d4f6e89be0689369da773332acdbf6b8d13fabb17f not found: ID does not exist" containerID="90ca5c4fcef870ed7a73b2d4f6e89be0689369da773332acdbf6b8d13fabb17f"
Jul 24 17:52:32.655546 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.655490    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90ca5c4fcef870ed7a73b2d4f6e89be0689369da773332acdbf6b8d13fabb17f"} err="failed to get container status \"90ca5c4fcef870ed7a73b2d4f6e89be0689369da773332acdbf6b8d13fabb17f\": rpc error: code = NotFound desc = could not find container \"90ca5c4fcef870ed7a73b2d4f6e89be0689369da773332acdbf6b8d13fabb17f\": container with ID starting with 90ca5c4fcef870ed7a73b2d4f6e89be0689369da773332acdbf6b8d13fabb17f not found: ID does not exist"
Jul 24 17:52:32.664137 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.664112    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l"]
Jul 24 17:52:32.676004 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.675967    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-117b3f04-7dd7c74f98-2gw2l"]
Jul 24 17:52:32.691396 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.691371    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-8557fb8557-brbdk"]
Jul 24 17:52:32.698874 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.698850    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-117b3f04-8557fb8557-brbdk"]
Jul 24 17:52:32.711034 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.711008    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6"]
Jul 24 17:52:32.722566 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:32.722540    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-117b3f04-f5b55d8d4-9gkn6"]
Jul 24 17:52:33.824661 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:33.824629    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6892fd83-cd5e-41ee-a993-d62ccdd323ee" path="/var/lib/kubelet/pods/6892fd83-cd5e-41ee-a993-d62ccdd323ee/volumes"
Jul 24 17:52:33.825043 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:33.824989    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95ee5d00-72b8-43cf-bf55-1c73c964827b" path="/var/lib/kubelet/pods/95ee5d00-72b8-43cf-bf55-1c73c964827b/volumes"
Jul 24 17:52:33.825357 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:52:33.825341    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5" path="/var/lib/kubelet/pods/a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5/volumes"
Jul 24 17:53:07.404757 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:07.404724    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64"]
Jul 24 17:53:07.405358 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:07.405118    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95ee5d00-72b8-43cf-bf55-1c73c964827b" containerName="payload-processing"
Jul 24 17:53:07.405358 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:07.405130    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="95ee5d00-72b8-43cf-bf55-1c73c964827b" containerName="payload-processing"
Jul 24 17:53:07.405358 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:07.405165    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6892fd83-cd5e-41ee-a993-d62ccdd323ee" containerName="maas-api"
Jul 24 17:53:07.405358 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:07.405174    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6892fd83-cd5e-41ee-a993-d62ccdd323ee" containerName="maas-api"
Jul 24 17:53:07.405358 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:07.405183    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5" containerName="payload-pre-processing"
Jul 24 17:53:07.405358 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:07.405189    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5" containerName="payload-pre-processing"
Jul 24 17:53:07.405358 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:07.405243    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8ff7a7a-a4ac-4563-b4c8-8ee63280b5e5" containerName="payload-pre-processing"
Jul 24 17:53:07.405358 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:07.405255    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6892fd83-cd5e-41ee-a993-d62ccdd323ee" containerName="maas-api"
Jul 24 17:53:07.405358 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:07.405263    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="95ee5d00-72b8-43cf-bf55-1c73c964827b" containerName="payload-processing"
Jul 24 17:53:07.408322 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:07.408301    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64"
Jul 24 17:53:07.413479 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:07.413453    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-9ae4eef5-serving-cert\""
Jul 24 17:53:07.424227 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:07.424200    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64"]
Jul 24 17:53:07.515819 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:07.515787    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca298aa0-ac5b-4146-a628-9bb8259fc6bd-openshift-service-ca\") pod \"maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64\" (UID: \"ca298aa0-ac5b-4146-a628-9bb8259fc6bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64"
Jul 24 17:53:07.515954 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:07.515833    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7c56h\" (UniqueName: \"kubernetes.io/projected/ca298aa0-ac5b-4146-a628-9bb8259fc6bd-kube-api-access-7c56h\") pod \"maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64\" (UID: \"ca298aa0-ac5b-4146-a628-9bb8259fc6bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64"
Jul 24 17:53:07.515954 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:07.515857    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca298aa0-ac5b-4146-a628-9bb8259fc6bd-maas-api-tls\") pod \"maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64\" (UID: \"ca298aa0-ac5b-4146-a628-9bb8259fc6bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64"
Jul 24 17:53:07.616500 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:07.616465    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7c56h\" (UniqueName: \"kubernetes.io/projected/ca298aa0-ac5b-4146-a628-9bb8259fc6bd-kube-api-access-7c56h\") pod \"maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64\" (UID: \"ca298aa0-ac5b-4146-a628-9bb8259fc6bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64"
Jul 24 17:53:07.616500 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:07.616502    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca298aa0-ac5b-4146-a628-9bb8259fc6bd-maas-api-tls\") pod \"maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64\" (UID: \"ca298aa0-ac5b-4146-a628-9bb8259fc6bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64"
Jul 24 17:53:07.616711 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:07.616566    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca298aa0-ac5b-4146-a628-9bb8259fc6bd-openshift-service-ca\") pod \"maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64\" (UID: \"ca298aa0-ac5b-4146-a628-9bb8259fc6bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64"
Jul 24 17:53:07.617135 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:07.617116    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca298aa0-ac5b-4146-a628-9bb8259fc6bd-openshift-service-ca\") pod \"maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64\" (UID: \"ca298aa0-ac5b-4146-a628-9bb8259fc6bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64"
Jul 24 17:53:07.619044 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:07.619026    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca298aa0-ac5b-4146-a628-9bb8259fc6bd-maas-api-tls\") pod \"maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64\" (UID: \"ca298aa0-ac5b-4146-a628-9bb8259fc6bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64"
Jul 24 17:53:07.626202 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:07.626180    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7c56h\" (UniqueName: \"kubernetes.io/projected/ca298aa0-ac5b-4146-a628-9bb8259fc6bd-kube-api-access-7c56h\") pod \"maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64\" (UID: \"ca298aa0-ac5b-4146-a628-9bb8259fc6bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64"
Jul 24 17:53:07.718586 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:07.718513    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64"
Jul 24 17:53:07.863906 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:07.863881    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64"]
Jul 24 17:53:07.865809 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:53:07.865776    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca298aa0_ac5b_4146_a628_9bb8259fc6bd.slice/crio-464a6a36d3cbb62c352df6ece8dc61b3936e6ca2bff10975f52c8142873b33ed WatchSource:0}: Error finding container 464a6a36d3cbb62c352df6ece8dc61b3936e6ca2bff10975f52c8142873b33ed: Status 404 returned error can't find the container with id 464a6a36d3cbb62c352df6ece8dc61b3936e6ca2bff10975f52c8142873b33ed
Jul 24 17:53:07.867229 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:07.867211    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 17:53:08.396636 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:08.396606    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8"]
Jul 24 17:53:08.400027 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:08.400004    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8"
Jul 24 17:53:08.403021 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:08.403005    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-9ae4eef5\""
Jul 24 17:53:08.404043 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:08.404023    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-9ae4eef5-dockercfg-8fv6x\""
Jul 24 17:53:08.422162 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:08.419358    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8"]
Jul 24 17:53:08.480459 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:08.480429    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt"]
Jul 24 17:53:08.484259 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:08.484239    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt"
Jul 24 17:53:08.505795 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:08.505728    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt"]
Jul 24 17:53:08.526599 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:08.526520    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5186e79-90a7-40e3-a439-5d39338d5326-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8\" (UID: \"c5186e79-90a7-40e3-a439-5d39338d5326\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8"
Jul 24 17:53:08.526599 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:08.526585    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2m57\" (UniqueName: \"kubernetes.io/projected/c5186e79-90a7-40e3-a439-5d39338d5326-kube-api-access-s2m57\") pod \"payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8\" (UID: \"c5186e79-90a7-40e3-a439-5d39338d5326\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8"
Jul 24 17:53:08.627104 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:08.627065    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5186e79-90a7-40e3-a439-5d39338d5326-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8\" (UID: \"c5186e79-90a7-40e3-a439-5d39338d5326\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8"
Jul 24 17:53:08.627262 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:08.627124    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3-plugins-config-volume\") pod \"payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt\" (UID: \"cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3\") " pod="openshift-ingress/payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt"
Jul 24 17:53:08.627262 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:08.627212    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2m57\" (UniqueName: \"kubernetes.io/projected/c5186e79-90a7-40e3-a439-5d39338d5326-kube-api-access-s2m57\") pod \"payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8\" (UID: \"c5186e79-90a7-40e3-a439-5d39338d5326\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8"
Jul 24 17:53:08.627405 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:08.627285    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrzdt\" (UniqueName: \"kubernetes.io/projected/cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3-kube-api-access-xrzdt\") pod \"payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt\" (UID: \"cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3\") " pod="openshift-ingress/payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt"
Jul 24 17:53:08.627864 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:08.627845    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5186e79-90a7-40e3-a439-5d39338d5326-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8\" (UID: \"c5186e79-90a7-40e3-a439-5d39338d5326\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8"
Jul 24 17:53:08.637813 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:08.637790    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2m57\" (UniqueName: \"kubernetes.io/projected/c5186e79-90a7-40e3-a439-5d39338d5326-kube-api-access-s2m57\") pod \"payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8\" (UID: \"c5186e79-90a7-40e3-a439-5d39338d5326\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8"
Jul 24 17:53:08.713108 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:08.713079    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8"
Jul 24 17:53:08.728528 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:08.728498    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3-plugins-config-volume\") pod \"payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt\" (UID: \"cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3\") " pod="openshift-ingress/payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt"
Jul 24 17:53:08.728645 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:08.728564    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrzdt\" (UniqueName: \"kubernetes.io/projected/cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3-kube-api-access-xrzdt\") pod \"payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt\" (UID: \"cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3\") " pod="openshift-ingress/payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt"
Jul 24 17:53:08.729206 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:08.729186    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3-plugins-config-volume\") pod \"payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt\" (UID: \"cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3\") " pod="openshift-ingress/payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt"
Jul 24 17:53:08.738232 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:08.738186    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrzdt\" (UniqueName: \"kubernetes.io/projected/cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3-kube-api-access-xrzdt\") pod \"payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt\" (UID: \"cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3\") " pod="openshift-ingress/payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt"
Jul 24 17:53:08.794082 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:08.793945    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt"
Jul 24 17:53:08.826927 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:08.826885    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64" event={"ID":"ca298aa0-ac5b-4146-a628-9bb8259fc6bd","Type":"ContainerStarted","Data":"b520c50e84b79edb889eb79fd4c6e575412ed248549adb2785ebf02d85b85ca5"}
Jul 24 17:53:08.827085 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:08.826950    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64" event={"ID":"ca298aa0-ac5b-4146-a628-9bb8259fc6bd","Type":"ContainerStarted","Data":"464a6a36d3cbb62c352df6ece8dc61b3936e6ca2bff10975f52c8142873b33ed"}
Jul 24 17:53:08.828190 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:08.827728    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64"
Jul 24 17:53:08.955110 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:08.955054    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64" podStartSLOduration=1.558198894 podStartE2EDuration="1.955036747s" totalImagesPullingTime="396.837853ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:53:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:53:08.86537321 +0000 UTC m=+2151.592529448" watchObservedRunningTime="2026-07-24 17:53:08.955036747 +0000 UTC m=+2151.682192969"
Jul 24 17:53:08.956448 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:08.956428    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt"]
Jul 24 17:53:08.958659 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:53:08.958637    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb2f2b69_f80e_4a5f_9d2a_873a9a1f2ca3.slice/crio-69b454eebe4ff3a3d52865920d5e585bf5ae1517c10f5815aa24e1d151d13218 WatchSource:0}: Error finding container 69b454eebe4ff3a3d52865920d5e585bf5ae1517c10f5815aa24e1d151d13218: Status 404 returned error can't find the container with id 69b454eebe4ff3a3d52865920d5e585bf5ae1517c10f5815aa24e1d151d13218
Jul 24 17:53:09.072793 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:09.072668    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8"]
Jul 24 17:53:09.075057 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:53:09.075034    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5186e79_90a7_40e3_a439_5d39338d5326.slice/crio-1eab8ef91f39e8a63aacc7ed837a84e1024d5da83e6336f771656708dd07d81e WatchSource:0}: Error finding container 1eab8ef91f39e8a63aacc7ed837a84e1024d5da83e6336f771656708dd07d81e: Status 404 returned error can't find the container with id 1eab8ef91f39e8a63aacc7ed837a84e1024d5da83e6336f771656708dd07d81e
Jul 24 17:53:09.833094 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:09.832998    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8" event={"ID":"c5186e79-90a7-40e3-a439-5d39338d5326","Type":"ContainerStarted","Data":"322b180ba5a025c7c676a4940ef6e0feba941e0451b78181f905e6b6ad5899d5"}
Jul 24 17:53:09.833094 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:09.833055    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8" event={"ID":"c5186e79-90a7-40e3-a439-5d39338d5326","Type":"ContainerStarted","Data":"1eab8ef91f39e8a63aacc7ed837a84e1024d5da83e6336f771656708dd07d81e"}
Jul 24 17:53:09.833541 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:09.833092    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8"
Jul 24 17:53:09.834710 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:09.834685    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt" event={"ID":"cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3","Type":"ContainerStarted","Data":"b60e0bfdb1daf08681ed7fa01f25f6140f7e2a00bc0802662f6686f98be95491"}
Jul 24 17:53:09.834848 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:09.834726    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt" event={"ID":"cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3","Type":"ContainerStarted","Data":"69b454eebe4ff3a3d52865920d5e585bf5ae1517c10f5815aa24e1d151d13218"}
Jul 24 17:53:09.834929 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:09.834903    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt"
Jul 24 17:53:09.894664 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:09.894621    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8" podStartSLOduration=1.8946093 podStartE2EDuration="1.8946093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:53:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:53:09.894233271 +0000 UTC m=+2152.621389497" watchObservedRunningTime="2026-07-24 17:53:09.8946093 +0000 UTC m=+2152.621765536"
Jul 24 17:53:09.926279 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:09.926231    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt" podStartSLOduration=1.926214666 podStartE2EDuration="1.926214666s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:53:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:53:09.924034585 +0000 UTC m=+2152.651190798" watchObservedRunningTime="2026-07-24 17:53:09.926214666 +0000 UTC m=+2152.653370898"
Jul 24 17:53:15.845929 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:15.845898    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64"
Jul 24 17:53:20.841961 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:20.841928    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8"
Jul 24 17:53:20.842462 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:53:20.842010    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt"
Jul 24 17:54:12.515054 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:12.515013    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64"]
Jul 24 17:54:12.515619 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:12.515288    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64" podUID="ca298aa0-ac5b-4146-a628-9bb8259fc6bd" containerName="maas-api" containerID="cri-o://b520c50e84b79edb889eb79fd4c6e575412ed248549adb2785ebf02d85b85ca5" gracePeriod=30
Jul 24 17:54:12.575120 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:12.575087    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt"]
Jul 24 17:54:12.575411 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:12.575304    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt" podUID="cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3" containerName="payload-processing" containerID="cri-o://b60e0bfdb1daf08681ed7fa01f25f6140f7e2a00bc0802662f6686f98be95491" gracePeriod=30
Jul 24 17:54:12.585467 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:12.585418    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8"]
Jul 24 17:54:12.586313 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:12.585698    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8" podUID="c5186e79-90a7-40e3-a439-5d39338d5326" containerName="payload-pre-processing" containerID="cri-o://322b180ba5a025c7c676a4940ef6e0feba941e0451b78181f905e6b6ad5899d5" gracePeriod=30
Jul 24 17:54:12.870360 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:12.870327    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt"
Jul 24 17:54:12.885526 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:12.885505    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64"
Jul 24 17:54:12.892660 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:12.892632    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8"
Jul 24 17:54:12.939214 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:12.939187    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3-plugins-config-volume\") pod \"cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3\" (UID: \"cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3\") "
Jul 24 17:54:12.939366 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:12.939250    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca298aa0-ac5b-4146-a628-9bb8259fc6bd-openshift-service-ca\") pod \"ca298aa0-ac5b-4146-a628-9bb8259fc6bd\" (UID: \"ca298aa0-ac5b-4146-a628-9bb8259fc6bd\") "
Jul 24 17:54:12.939366 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:12.939274    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5186e79-90a7-40e3-a439-5d39338d5326-plugins-config-volume\") pod \"c5186e79-90a7-40e3-a439-5d39338d5326\" (UID: \"c5186e79-90a7-40e3-a439-5d39338d5326\") "
Jul 24 17:54:12.939366 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:12.939297    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrzdt\" (UniqueName: \"kubernetes.io/projected/cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3-kube-api-access-xrzdt\") pod \"cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3\" (UID: \"cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3\") "
Jul 24 17:54:12.939580 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:12.939374    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7c56h\" (UniqueName: \"kubernetes.io/projected/ca298aa0-ac5b-4146-a628-9bb8259fc6bd-kube-api-access-7c56h\") pod \"ca298aa0-ac5b-4146-a628-9bb8259fc6bd\" (UID: \"ca298aa0-ac5b-4146-a628-9bb8259fc6bd\") "
Jul 24 17:54:12.939580 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:12.939399    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2m57\" (UniqueName: \"kubernetes.io/projected/c5186e79-90a7-40e3-a439-5d39338d5326-kube-api-access-s2m57\") pod \"c5186e79-90a7-40e3-a439-5d39338d5326\" (UID: \"c5186e79-90a7-40e3-a439-5d39338d5326\") "
Jul 24 17:54:12.939580 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:12.939458    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca298aa0-ac5b-4146-a628-9bb8259fc6bd-maas-api-tls\") pod \"ca298aa0-ac5b-4146-a628-9bb8259fc6bd\" (UID: \"ca298aa0-ac5b-4146-a628-9bb8259fc6bd\") "
Jul 24 17:54:12.939752 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:12.939609    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca298aa0-ac5b-4146-a628-9bb8259fc6bd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ca298aa0-ac5b-4146-a628-9bb8259fc6bd" (UID: "ca298aa0-ac5b-4146-a628-9bb8259fc6bd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:54:12.939752 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:12.939631    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3" (UID: "cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:54:12.939752 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:12.939668    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5186e79-90a7-40e3-a439-5d39338d5326-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c5186e79-90a7-40e3-a439-5d39338d5326" (UID: "c5186e79-90a7-40e3-a439-5d39338d5326"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:54:12.939920 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:12.939890    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3-plugins-config-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:54:12.939920 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:12.939911    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca298aa0-ac5b-4146-a628-9bb8259fc6bd-openshift-service-ca\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:54:12.940033 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:12.939927    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5186e79-90a7-40e3-a439-5d39338d5326-plugins-config-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:54:12.941663 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:12.941635    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca298aa0-ac5b-4146-a628-9bb8259fc6bd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ca298aa0-ac5b-4146-a628-9bb8259fc6bd" (UID: "ca298aa0-ac5b-4146-a628-9bb8259fc6bd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:54:12.941982 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:12.941956    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3-kube-api-access-xrzdt" (OuterVolumeSpecName: "kube-api-access-xrzdt") pod "cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3" (UID: "cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3"). InnerVolumeSpecName "kube-api-access-xrzdt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:54:12.942253 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:12.942230    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca298aa0-ac5b-4146-a628-9bb8259fc6bd-kube-api-access-7c56h" (OuterVolumeSpecName: "kube-api-access-7c56h") pod "ca298aa0-ac5b-4146-a628-9bb8259fc6bd" (UID: "ca298aa0-ac5b-4146-a628-9bb8259fc6bd"). InnerVolumeSpecName "kube-api-access-7c56h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:54:12.942386 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:12.942362    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5186e79-90a7-40e3-a439-5d39338d5326-kube-api-access-s2m57" (OuterVolumeSpecName: "kube-api-access-s2m57") pod "c5186e79-90a7-40e3-a439-5d39338d5326" (UID: "c5186e79-90a7-40e3-a439-5d39338d5326"). InnerVolumeSpecName "kube-api-access-s2m57". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:54:13.040886 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.040845    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrzdt\" (UniqueName: \"kubernetes.io/projected/cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3-kube-api-access-xrzdt\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:54:13.040886 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.040877    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7c56h\" (UniqueName: \"kubernetes.io/projected/ca298aa0-ac5b-4146-a628-9bb8259fc6bd-kube-api-access-7c56h\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:54:13.040886 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.040891    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2m57\" (UniqueName: \"kubernetes.io/projected/c5186e79-90a7-40e3-a439-5d39338d5326-kube-api-access-s2m57\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:54:13.041123 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.040904    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca298aa0-ac5b-4146-a628-9bb8259fc6bd-maas-api-tls\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:54:13.150034 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.149999    2562 generic.go:356] "Generic (PLEG): container finished" podID="cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3" containerID="b60e0bfdb1daf08681ed7fa01f25f6140f7e2a00bc0802662f6686f98be95491" exitCode=0
Jul 24 17:54:13.150263 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.150044    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt"
Jul 24 17:54:13.150263 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.150085    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt" event={"ID":"cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3","Type":"ContainerDied","Data":"b60e0bfdb1daf08681ed7fa01f25f6140f7e2a00bc0802662f6686f98be95491"}
Jul 24 17:54:13.150263 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.150209    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt" event={"ID":"cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3","Type":"ContainerDied","Data":"69b454eebe4ff3a3d52865920d5e585bf5ae1517c10f5815aa24e1d151d13218"}
Jul 24 17:54:13.150263 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.150243    2562 scope.go:117] "RemoveContainer" containerID="b60e0bfdb1daf08681ed7fa01f25f6140f7e2a00bc0802662f6686f98be95491"
Jul 24 17:54:13.151314 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.151299    2562 generic.go:356] "Generic (PLEG): container finished" podID="ca298aa0-ac5b-4146-a628-9bb8259fc6bd" containerID="b520c50e84b79edb889eb79fd4c6e575412ed248549adb2785ebf02d85b85ca5" exitCode=0
Jul 24 17:54:13.151383 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.151365    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64"
Jul 24 17:54:13.151433 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.151380    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64" event={"ID":"ca298aa0-ac5b-4146-a628-9bb8259fc6bd","Type":"ContainerDied","Data":"b520c50e84b79edb889eb79fd4c6e575412ed248549adb2785ebf02d85b85ca5"}
Jul 24 17:54:13.151433 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.151422    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64" event={"ID":"ca298aa0-ac5b-4146-a628-9bb8259fc6bd","Type":"ContainerDied","Data":"464a6a36d3cbb62c352df6ece8dc61b3936e6ca2bff10975f52c8142873b33ed"}
Jul 24 17:54:13.152783 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.152755    2562 generic.go:356] "Generic (PLEG): container finished" podID="c5186e79-90a7-40e3-a439-5d39338d5326" containerID="322b180ba5a025c7c676a4940ef6e0feba941e0451b78181f905e6b6ad5899d5" exitCode=0
Jul 24 17:54:13.152885 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.152785    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8" event={"ID":"c5186e79-90a7-40e3-a439-5d39338d5326","Type":"ContainerDied","Data":"322b180ba5a025c7c676a4940ef6e0feba941e0451b78181f905e6b6ad5899d5"}
Jul 24 17:54:13.152885 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.152811    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8"
Jul 24 17:54:13.152885 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.152821    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8" event={"ID":"c5186e79-90a7-40e3-a439-5d39338d5326","Type":"ContainerDied","Data":"1eab8ef91f39e8a63aacc7ed837a84e1024d5da83e6336f771656708dd07d81e"}
Jul 24 17:54:13.160589 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.160567    2562 scope.go:117] "RemoveContainer" containerID="b60e0bfdb1daf08681ed7fa01f25f6140f7e2a00bc0802662f6686f98be95491"
Jul 24 17:54:13.160867 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:54:13.160847    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b60e0bfdb1daf08681ed7fa01f25f6140f7e2a00bc0802662f6686f98be95491\": container with ID starting with b60e0bfdb1daf08681ed7fa01f25f6140f7e2a00bc0802662f6686f98be95491 not found: ID does not exist" containerID="b60e0bfdb1daf08681ed7fa01f25f6140f7e2a00bc0802662f6686f98be95491"
Jul 24 17:54:13.160945 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.160876    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b60e0bfdb1daf08681ed7fa01f25f6140f7e2a00bc0802662f6686f98be95491"} err="failed to get container status \"b60e0bfdb1daf08681ed7fa01f25f6140f7e2a00bc0802662f6686f98be95491\": rpc error: code = NotFound desc = could not find container \"b60e0bfdb1daf08681ed7fa01f25f6140f7e2a00bc0802662f6686f98be95491\": container with ID starting with b60e0bfdb1daf08681ed7fa01f25f6140f7e2a00bc0802662f6686f98be95491 not found: ID does not exist"
Jul 24 17:54:13.160945 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.160894    2562 scope.go:117] "RemoveContainer" containerID="b520c50e84b79edb889eb79fd4c6e575412ed248549adb2785ebf02d85b85ca5"
Jul 24 17:54:13.170119 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.170097    2562 scope.go:117] "RemoveContainer" containerID="b520c50e84b79edb889eb79fd4c6e575412ed248549adb2785ebf02d85b85ca5"
Jul 24 17:54:13.170526 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:54:13.170508    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b520c50e84b79edb889eb79fd4c6e575412ed248549adb2785ebf02d85b85ca5\": container with ID starting with b520c50e84b79edb889eb79fd4c6e575412ed248549adb2785ebf02d85b85ca5 not found: ID does not exist" containerID="b520c50e84b79edb889eb79fd4c6e575412ed248549adb2785ebf02d85b85ca5"
Jul 24 17:54:13.170587 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.170533    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b520c50e84b79edb889eb79fd4c6e575412ed248549adb2785ebf02d85b85ca5"} err="failed to get container status \"b520c50e84b79edb889eb79fd4c6e575412ed248549adb2785ebf02d85b85ca5\": rpc error: code = NotFound desc = could not find container \"b520c50e84b79edb889eb79fd4c6e575412ed248549adb2785ebf02d85b85ca5\": container with ID starting with b520c50e84b79edb889eb79fd4c6e575412ed248549adb2785ebf02d85b85ca5 not found: ID does not exist"
Jul 24 17:54:13.170587 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.170552    2562 scope.go:117] "RemoveContainer" containerID="322b180ba5a025c7c676a4940ef6e0feba941e0451b78181f905e6b6ad5899d5"
Jul 24 17:54:13.178240 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.178222    2562 scope.go:117] "RemoveContainer" containerID="322b180ba5a025c7c676a4940ef6e0feba941e0451b78181f905e6b6ad5899d5"
Jul 24 17:54:13.178505 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:54:13.178486    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"322b180ba5a025c7c676a4940ef6e0feba941e0451b78181f905e6b6ad5899d5\": container with ID starting with 322b180ba5a025c7c676a4940ef6e0feba941e0451b78181f905e6b6ad5899d5 not found: ID does not exist" containerID="322b180ba5a025c7c676a4940ef6e0feba941e0451b78181f905e6b6ad5899d5"
Jul 24 17:54:13.178567 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.178515    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"322b180ba5a025c7c676a4940ef6e0feba941e0451b78181f905e6b6ad5899d5"} err="failed to get container status \"322b180ba5a025c7c676a4940ef6e0feba941e0451b78181f905e6b6ad5899d5\": rpc error: code = NotFound desc = could not find container \"322b180ba5a025c7c676a4940ef6e0feba941e0451b78181f905e6b6ad5899d5\": container with ID starting with 322b180ba5a025c7c676a4940ef6e0feba941e0451b78181f905e6b6ad5899d5 not found: ID does not exist"
Jul 24 17:54:13.181590 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.181568    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64"]
Jul 24 17:54:13.189891 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.189868    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9ae4eef5-6b69ccc967-wqg64"]
Jul 24 17:54:13.201983 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.201960    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt"]
Jul 24 17:54:13.206434 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.206404    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9ae4eef5-5ddd994c94-t45tt"]
Jul 24 17:54:13.220216 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.220189    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8"]
Jul 24 17:54:13.227529 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.227506    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9ae4eef5-f8d7db6d-q9qx8"]
Jul 24 17:54:13.825248 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.825167    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5186e79-90a7-40e3-a439-5d39338d5326" path="/var/lib/kubelet/pods/c5186e79-90a7-40e3-a439-5d39338d5326/volumes"
Jul 24 17:54:13.825597 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.825586    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca298aa0-ac5b-4146-a628-9bb8259fc6bd" path="/var/lib/kubelet/pods/ca298aa0-ac5b-4146-a628-9bb8259fc6bd/volumes"
Jul 24 17:54:13.825905 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:54:13.825891    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3" path="/var/lib/kubelet/pods/cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3/volumes"
Jul 24 17:56:28.530784 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:28.530752    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2"]
Jul 24 17:56:28.531247 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:28.531188    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca298aa0-ac5b-4146-a628-9bb8259fc6bd" containerName="maas-api"
Jul 24 17:56:28.531247 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:28.531201    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca298aa0-ac5b-4146-a628-9bb8259fc6bd" containerName="maas-api"
Jul 24 17:56:28.531247 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:28.531216    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3" containerName="payload-processing"
Jul 24 17:56:28.531247 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:28.531223    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3" containerName="payload-processing"
Jul 24 17:56:28.531247 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:28.531237    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5186e79-90a7-40e3-a439-5d39338d5326" containerName="payload-pre-processing"
Jul 24 17:56:28.531247 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:28.531243    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5186e79-90a7-40e3-a439-5d39338d5326" containerName="payload-pre-processing"
Jul 24 17:56:28.531478 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:28.531327    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5186e79-90a7-40e3-a439-5d39338d5326" containerName="payload-pre-processing"
Jul 24 17:56:28.531478 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:28.531341    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb2f2b69-f80e-4a5f-9d2a-873a9a1f2ca3" containerName="payload-processing"
Jul 24 17:56:28.531478 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:28.531348    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca298aa0-ac5b-4146-a628-9bb8259fc6bd" containerName="maas-api"
Jul 24 17:56:28.534416 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:28.534392    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2"
Jul 24 17:56:28.537247 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:28.537219    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-cdf40f-serving-cert\""
Jul 24 17:56:28.540761 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:28.540735    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f37fdfc-18ec-45f5-8344-3b9681677935-openshift-service-ca\") pod \"maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2\" (UID: \"4f37fdfc-18ec-45f5-8344-3b9681677935\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2"
Jul 24 17:56:28.540865 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:28.540772    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8kz9\" (UniqueName: \"kubernetes.io/projected/4f37fdfc-18ec-45f5-8344-3b9681677935-kube-api-access-d8kz9\") pod \"maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2\" (UID: \"4f37fdfc-18ec-45f5-8344-3b9681677935\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2"
Jul 24 17:56:28.540865 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:28.540840    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f37fdfc-18ec-45f5-8344-3b9681677935-maas-api-tls\") pod \"maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2\" (UID: \"4f37fdfc-18ec-45f5-8344-3b9681677935\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2"
Jul 24 17:56:28.555054 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:28.555030    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2"]
Jul 24 17:56:28.641905 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:28.641869    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8kz9\" (UniqueName: \"kubernetes.io/projected/4f37fdfc-18ec-45f5-8344-3b9681677935-kube-api-access-d8kz9\") pod \"maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2\" (UID: \"4f37fdfc-18ec-45f5-8344-3b9681677935\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2"
Jul 24 17:56:28.642092 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:28.641929    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f37fdfc-18ec-45f5-8344-3b9681677935-maas-api-tls\") pod \"maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2\" (UID: \"4f37fdfc-18ec-45f5-8344-3b9681677935\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2"
Jul 24 17:56:28.642092 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:28.642000    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f37fdfc-18ec-45f5-8344-3b9681677935-openshift-service-ca\") pod \"maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2\" (UID: \"4f37fdfc-18ec-45f5-8344-3b9681677935\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2"
Jul 24 17:56:28.642259 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:56:28.642161    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-serving-cert: secret "maas-api-e2e-inf-a-cdf40f-serving-cert" not found
Jul 24 17:56:28.642259 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:56:28.642253    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f37fdfc-18ec-45f5-8344-3b9681677935-maas-api-tls podName:4f37fdfc-18ec-45f5-8344-3b9681677935 nodeName:}" failed. No retries permitted until 2026-07-24 17:56:29.142228302 +0000 UTC m=+2351.869384502 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4f37fdfc-18ec-45f5-8344-3b9681677935-maas-api-tls") pod "maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2" (UID: "4f37fdfc-18ec-45f5-8344-3b9681677935") : secret "maas-api-e2e-inf-a-cdf40f-serving-cert" not found
Jul 24 17:56:28.642660 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:28.642640    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f37fdfc-18ec-45f5-8344-3b9681677935-openshift-service-ca\") pod \"maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2\" (UID: \"4f37fdfc-18ec-45f5-8344-3b9681677935\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2"
Jul 24 17:56:28.656923 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:28.656897    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8kz9\" (UniqueName: \"kubernetes.io/projected/4f37fdfc-18ec-45f5-8344-3b9681677935-kube-api-access-d8kz9\") pod \"maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2\" (UID: \"4f37fdfc-18ec-45f5-8344-3b9681677935\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2"
Jul 24 17:56:29.147272 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.147233    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f37fdfc-18ec-45f5-8344-3b9681677935-maas-api-tls\") pod \"maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2\" (UID: \"4f37fdfc-18ec-45f5-8344-3b9681677935\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2"
Jul 24 17:56:29.149693 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.149664    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f37fdfc-18ec-45f5-8344-3b9681677935-maas-api-tls\") pod \"maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2\" (UID: \"4f37fdfc-18ec-45f5-8344-3b9681677935\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2"
Jul 24 17:56:29.296068 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.296035    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j"]
Jul 24 17:56:29.299583 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.299565    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j"
Jul 24 17:56:29.305592 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.305566    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-cdf40f\""
Jul 24 17:56:29.318281 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.318257    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j"]
Jul 24 17:56:29.348719 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.348688    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqz6c\" (UniqueName: \"kubernetes.io/projected/8452d620-74d7-4a94-81e6-a6fbd919eb9f-kube-api-access-wqz6c\") pod \"payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j\" (UID: \"8452d620-74d7-4a94-81e6-a6fbd919eb9f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j"
Jul 24 17:56:29.348833 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.348725    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8452d620-74d7-4a94-81e6-a6fbd919eb9f-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j\" (UID: \"8452d620-74d7-4a94-81e6-a6fbd919eb9f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j"
Jul 24 17:56:29.363371 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.363350    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65"]
Jul 24 17:56:29.366652 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.366637    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65"
Jul 24 17:56:29.369420 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.369406    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-cdf40f-dockercfg-g48kt\""
Jul 24 17:56:29.379410 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.379389    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65"]
Jul 24 17:56:29.444178 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.444086    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2"
Jul 24 17:56:29.449417 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.449393    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqz6c\" (UniqueName: \"kubernetes.io/projected/8452d620-74d7-4a94-81e6-a6fbd919eb9f-kube-api-access-wqz6c\") pod \"payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j\" (UID: \"8452d620-74d7-4a94-81e6-a6fbd919eb9f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j"
Jul 24 17:56:29.449488 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.449437    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8452d620-74d7-4a94-81e6-a6fbd919eb9f-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j\" (UID: \"8452d620-74d7-4a94-81e6-a6fbd919eb9f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j"
Jul 24 17:56:29.449488 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.449462    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29mql\" (UniqueName: \"kubernetes.io/projected/f021e2fe-b467-4dcd-a5a5-819ef65059df-kube-api-access-29mql\") pod \"payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65\" (UID: \"f021e2fe-b467-4dcd-a5a5-819ef65059df\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65"
Jul 24 17:56:29.449587 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.449491    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f021e2fe-b467-4dcd-a5a5-819ef65059df-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65\" (UID: \"f021e2fe-b467-4dcd-a5a5-819ef65059df\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65"
Jul 24 17:56:29.450182 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.450133    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8452d620-74d7-4a94-81e6-a6fbd919eb9f-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j\" (UID: \"8452d620-74d7-4a94-81e6-a6fbd919eb9f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j"
Jul 24 17:56:29.459191 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.459168    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqz6c\" (UniqueName: \"kubernetes.io/projected/8452d620-74d7-4a94-81e6-a6fbd919eb9f-kube-api-access-wqz6c\") pod \"payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j\" (UID: \"8452d620-74d7-4a94-81e6-a6fbd919eb9f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j"
Jul 24 17:56:29.551057 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.551018    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29mql\" (UniqueName: \"kubernetes.io/projected/f021e2fe-b467-4dcd-a5a5-819ef65059df-kube-api-access-29mql\") pod \"payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65\" (UID: \"f021e2fe-b467-4dcd-a5a5-819ef65059df\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65"
Jul 24 17:56:29.551473 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.551076    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f021e2fe-b467-4dcd-a5a5-819ef65059df-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65\" (UID: \"f021e2fe-b467-4dcd-a5a5-819ef65059df\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65"
Jul 24 17:56:29.552101 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.552070    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f021e2fe-b467-4dcd-a5a5-819ef65059df-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65\" (UID: \"f021e2fe-b467-4dcd-a5a5-819ef65059df\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65"
Jul 24 17:56:29.560183 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.560152    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29mql\" (UniqueName: \"kubernetes.io/projected/f021e2fe-b467-4dcd-a5a5-819ef65059df-kube-api-access-29mql\") pod \"payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65\" (UID: \"f021e2fe-b467-4dcd-a5a5-819ef65059df\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65"
Jul 24 17:56:29.588112 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.588086    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2"]
Jul 24 17:56:29.590441 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:56:29.590414    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f37fdfc_18ec_45f5_8344_3b9681677935.slice/crio-4812dbabb67af813ff8d89952699d1903e07efc257af3690f7222f70a70d1f23 WatchSource:0}: Error finding container 4812dbabb67af813ff8d89952699d1903e07efc257af3690f7222f70a70d1f23: Status 404 returned error can't find the container with id 4812dbabb67af813ff8d89952699d1903e07efc257af3690f7222f70a70d1f23
Jul 24 17:56:29.610343 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.610315    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j"
Jul 24 17:56:29.676766 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.676736    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65"
Jul 24 17:56:29.758631 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.758593    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j"]
Jul 24 17:56:29.761440 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:56:29.761415    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8452d620_74d7_4a94_81e6_a6fbd919eb9f.slice/crio-d3c3ef525850c77a29e7da176e261e9ee7998685107dc1335de2c8445d922655 WatchSource:0}: Error finding container d3c3ef525850c77a29e7da176e261e9ee7998685107dc1335de2c8445d922655: Status 404 returned error can't find the container with id d3c3ef525850c77a29e7da176e261e9ee7998685107dc1335de2c8445d922655
Jul 24 17:56:29.827492 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.827451    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65"]
Jul 24 17:56:29.827660 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:56:29.827636    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf021e2fe_b467_4dcd_a5a5_819ef65059df.slice/crio-be7117ce6c90717142277fd92281803c22ced5c3a40df6f34c16f3a14c8afc4f WatchSource:0}: Error finding container be7117ce6c90717142277fd92281803c22ced5c3a40df6f34c16f3a14c8afc4f: Status 404 returned error can't find the container with id be7117ce6c90717142277fd92281803c22ced5c3a40df6f34c16f3a14c8afc4f
Jul 24 17:56:29.835943 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.835894    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j" event={"ID":"8452d620-74d7-4a94-81e6-a6fbd919eb9f","Type":"ContainerStarted","Data":"ea88a840bc1e93b2b0ae71461a02e27bff36d0e2776092c3f0bf61020ad6c72a"}
Jul 24 17:56:29.836260 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.836121    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j" event={"ID":"8452d620-74d7-4a94-81e6-a6fbd919eb9f","Type":"ContainerStarted","Data":"d3c3ef525850c77a29e7da176e261e9ee7998685107dc1335de2c8445d922655"}
Jul 24 17:56:29.836734 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.836554    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j"
Jul 24 17:56:29.837881 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.837767    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2" event={"ID":"4f37fdfc-18ec-45f5-8344-3b9681677935","Type":"ContainerStarted","Data":"4812dbabb67af813ff8d89952699d1903e07efc257af3690f7222f70a70d1f23"}
Jul 24 17:56:29.839393 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.839371    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65" event={"ID":"f021e2fe-b467-4dcd-a5a5-819ef65059df","Type":"ContainerStarted","Data":"be7117ce6c90717142277fd92281803c22ced5c3a40df6f34c16f3a14c8afc4f"}
Jul 24 17:56:29.856707 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:29.856665    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j" podStartSLOduration=0.856652011 podStartE2EDuration="856.652011ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:56:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:56:29.854806051 +0000 UTC m=+2352.581962278" watchObservedRunningTime="2026-07-24 17:56:29.856652011 +0000 UTC m=+2352.583808258"
Jul 24 17:56:30.846435 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:30.846393    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65" event={"ID":"f021e2fe-b467-4dcd-a5a5-819ef65059df","Type":"ContainerStarted","Data":"3fbd046d8109d6d105fb240a91b47d91bb4309b28261b048d1412c9197d73def"}
Jul 24 17:56:30.846905 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:30.846486    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65"
Jul 24 17:56:30.848029 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:30.848004    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2" event={"ID":"4f37fdfc-18ec-45f5-8344-3b9681677935","Type":"ContainerStarted","Data":"924574e8ccae4cb2c4142e93cf35a2fca2946b49f365df19cdd236d8d67bff42"}
Jul 24 17:56:30.848165 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:30.848084    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2"
Jul 24 17:56:30.866542 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:30.866501    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65" podStartSLOduration=1.8664880940000002 podStartE2EDuration="1.866488094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:56:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:56:30.86568375 +0000 UTC m=+2353.592839965" watchObservedRunningTime="2026-07-24 17:56:30.866488094 +0000 UTC m=+2353.593644307"
Jul 24 17:56:30.887179 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:30.887124    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2" podStartSLOduration=2.4362608359999998 podStartE2EDuration="2.887110282s" totalImagesPullingTime="450.849446ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:56:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:56:30.886466631 +0000 UTC m=+2353.613622845" watchObservedRunningTime="2026-07-24 17:56:30.887110282 +0000 UTC m=+2353.614266497"
Jul 24 17:56:36.744275 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:36.744236    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd"]
Jul 24 17:56:36.748161 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:36.748121    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd"
Jul 24 17:56:36.752136 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:36.752115    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-3922c3-serving-cert\""
Jul 24 17:56:36.773641 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:36.773615    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd"]
Jul 24 17:56:36.822471 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:36.822438    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20852f34-832c-4899-8116-c6b6e1f8a425-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd\" (UID: \"20852f34-832c-4899-8116-c6b6e1f8a425\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd"
Jul 24 17:56:36.822635 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:36.822493    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20852f34-832c-4899-8116-c6b6e1f8a425-maas-api-tls\") pod \"maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd\" (UID: \"20852f34-832c-4899-8116-c6b6e1f8a425\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd"
Jul 24 17:56:36.822695 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:36.822622    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsmvk\" (UniqueName: \"kubernetes.io/projected/20852f34-832c-4899-8116-c6b6e1f8a425-kube-api-access-lsmvk\") pod \"maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd\" (UID: \"20852f34-832c-4899-8116-c6b6e1f8a425\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd"
Jul 24 17:56:36.858018 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:36.857991    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2"
Jul 24 17:56:36.923816 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:36.923767    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsmvk\" (UniqueName: \"kubernetes.io/projected/20852f34-832c-4899-8116-c6b6e1f8a425-kube-api-access-lsmvk\") pod \"maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd\" (UID: \"20852f34-832c-4899-8116-c6b6e1f8a425\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd"
Jul 24 17:56:36.924214 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:36.924191    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20852f34-832c-4899-8116-c6b6e1f8a425-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd\" (UID: \"20852f34-832c-4899-8116-c6b6e1f8a425\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd"
Jul 24 17:56:36.924382 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:36.924367    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20852f34-832c-4899-8116-c6b6e1f8a425-maas-api-tls\") pod \"maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd\" (UID: \"20852f34-832c-4899-8116-c6b6e1f8a425\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd"
Jul 24 17:56:36.925532 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:56:36.925509    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-serving-cert: secret "maas-api-e2e-inf-b-3922c3-serving-cert" not found
Jul 24 17:56:36.925729 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:56:36.925717    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20852f34-832c-4899-8116-c6b6e1f8a425-maas-api-tls podName:20852f34-832c-4899-8116-c6b6e1f8a425 nodeName:}" failed. No retries permitted until 2026-07-24 17:56:37.425688989 +0000 UTC m=+2360.152845195 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/20852f34-832c-4899-8116-c6b6e1f8a425-maas-api-tls") pod "maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd" (UID: "20852f34-832c-4899-8116-c6b6e1f8a425") : secret "maas-api-e2e-inf-b-3922c3-serving-cert" not found
Jul 24 17:56:36.925874 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:36.925845    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20852f34-832c-4899-8116-c6b6e1f8a425-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd\" (UID: \"20852f34-832c-4899-8116-c6b6e1f8a425\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd"
Jul 24 17:56:36.935511 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:36.935487    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsmvk\" (UniqueName: \"kubernetes.io/projected/20852f34-832c-4899-8116-c6b6e1f8a425-kube-api-access-lsmvk\") pod \"maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd\" (UID: \"20852f34-832c-4899-8116-c6b6e1f8a425\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd"
Jul 24 17:56:37.429231 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:37.429184    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20852f34-832c-4899-8116-c6b6e1f8a425-maas-api-tls\") pod \"maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd\" (UID: \"20852f34-832c-4899-8116-c6b6e1f8a425\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd"
Jul 24 17:56:37.431632 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:37.431606    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20852f34-832c-4899-8116-c6b6e1f8a425-maas-api-tls\") pod \"maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd\" (UID: \"20852f34-832c-4899-8116-c6b6e1f8a425\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd"
Jul 24 17:56:37.568616 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:37.568577    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6"]
Jul 24 17:56:37.572773 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:37.572749    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6"
Jul 24 17:56:37.575676 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:37.575652    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-3922c3\""
Jul 24 17:56:37.583049 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:37.583023    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft"]
Jul 24 17:56:37.587223 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:37.587199    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft"
Jul 24 17:56:37.588869 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:37.588842    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6"]
Jul 24 17:56:37.590039 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:37.590018    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-3922c3-dockercfg-wq2mh\""
Jul 24 17:56:37.604799 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:37.604777    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft"]
Jul 24 17:56:37.630931 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:37.630890    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2ffg\" (UniqueName: \"kubernetes.io/projected/29542207-0e8d-483b-aa3b-040e4d8b0acc-kube-api-access-k2ffg\") pod \"payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6\" (UID: \"29542207-0e8d-483b-aa3b-040e4d8b0acc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6"
Jul 24 17:56:37.631088 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:37.630953    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29542207-0e8d-483b-aa3b-040e4d8b0acc-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6\" (UID: \"29542207-0e8d-483b-aa3b-040e4d8b0acc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6"
Jul 24 17:56:37.631088 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:37.630985    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a42cbded-018b-416f-b096-a2ffa2b58d2b-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft\" (UID: \"a42cbded-018b-416f-b096-a2ffa2b58d2b\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft"
Jul 24 17:56:37.631214 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:37.631134    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgbtx\" (UniqueName: \"kubernetes.io/projected/a42cbded-018b-416f-b096-a2ffa2b58d2b-kube-api-access-dgbtx\") pod \"payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft\" (UID: \"a42cbded-018b-416f-b096-a2ffa2b58d2b\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft"
Jul 24 17:56:37.658988 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:37.658956    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd"
Jul 24 17:56:37.732207 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:37.732172    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgbtx\" (UniqueName: \"kubernetes.io/projected/a42cbded-018b-416f-b096-a2ffa2b58d2b-kube-api-access-dgbtx\") pod \"payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft\" (UID: \"a42cbded-018b-416f-b096-a2ffa2b58d2b\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft"
Jul 24 17:56:37.732394 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:37.732235    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2ffg\" (UniqueName: \"kubernetes.io/projected/29542207-0e8d-483b-aa3b-040e4d8b0acc-kube-api-access-k2ffg\") pod \"payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6\" (UID: \"29542207-0e8d-483b-aa3b-040e4d8b0acc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6"
Jul 24 17:56:37.732394 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:37.732289    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29542207-0e8d-483b-aa3b-040e4d8b0acc-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6\" (UID: \"29542207-0e8d-483b-aa3b-040e4d8b0acc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6"
Jul 24 17:56:37.732394 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:37.732331    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a42cbded-018b-416f-b096-a2ffa2b58d2b-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft\" (UID: \"a42cbded-018b-416f-b096-a2ffa2b58d2b\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft"
Jul 24 17:56:37.733355 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:37.733328    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a42cbded-018b-416f-b096-a2ffa2b58d2b-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft\" (UID: \"a42cbded-018b-416f-b096-a2ffa2b58d2b\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft"
Jul 24 17:56:37.733484 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:37.733328    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29542207-0e8d-483b-aa3b-040e4d8b0acc-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6\" (UID: \"29542207-0e8d-483b-aa3b-040e4d8b0acc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6"
Jul 24 17:56:37.749778 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:37.749661    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2ffg\" (UniqueName: \"kubernetes.io/projected/29542207-0e8d-483b-aa3b-040e4d8b0acc-kube-api-access-k2ffg\") pod \"payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6\" (UID: \"29542207-0e8d-483b-aa3b-040e4d8b0acc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6"
Jul 24 17:56:37.750955 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:37.750478    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgbtx\" (UniqueName: \"kubernetes.io/projected/a42cbded-018b-416f-b096-a2ffa2b58d2b-kube-api-access-dgbtx\") pod \"payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft\" (UID: \"a42cbded-018b-416f-b096-a2ffa2b58d2b\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft"
Jul 24 17:56:37.822919 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:56:37.822878    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20852f34_832c_4899_8116_c6b6e1f8a425.slice/crio-004eb11fc9746a6f4ff5c616351735f0cb0459b27f8d91af2e5416dd9fe6fd88 WatchSource:0}: Error finding container 004eb11fc9746a6f4ff5c616351735f0cb0459b27f8d91af2e5416dd9fe6fd88: Status 404 returned error can't find the container with id 004eb11fc9746a6f4ff5c616351735f0cb0459b27f8d91af2e5416dd9fe6fd88
Jul 24 17:56:37.826839 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:37.826816    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd"]
Jul 24 17:56:37.883541 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:37.883513    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6"
Jul 24 17:56:37.884067 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:37.884037    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd" event={"ID":"20852f34-832c-4899-8116-c6b6e1f8a425","Type":"ContainerStarted","Data":"004eb11fc9746a6f4ff5c616351735f0cb0459b27f8d91af2e5416dd9fe6fd88"}
Jul 24 17:56:37.897664 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:37.897637    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft"
Jul 24 17:56:38.035252 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:38.035208    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6"]
Jul 24 17:56:38.037671 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:56:38.037475    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29542207_0e8d_483b_aa3b_040e4d8b0acc.slice/crio-a3723b710bb4e316a77b968af739948ba55357b52a204a6f0e9f493917c8f298 WatchSource:0}: Error finding container a3723b710bb4e316a77b968af739948ba55357b52a204a6f0e9f493917c8f298: Status 404 returned error can't find the container with id a3723b710bb4e316a77b968af739948ba55357b52a204a6f0e9f493917c8f298
Jul 24 17:56:38.061589 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:38.061566    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft"]
Jul 24 17:56:38.063108 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:56:38.063071    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda42cbded_018b_416f_b096_a2ffa2b58d2b.slice/crio-e9d02fc9c0a2fed7b2afff77c55a2eb748672724478a52796cf7dd7810e7c0a3 WatchSource:0}: Error finding container e9d02fc9c0a2fed7b2afff77c55a2eb748672724478a52796cf7dd7810e7c0a3: Status 404 returned error can't find the container with id e9d02fc9c0a2fed7b2afff77c55a2eb748672724478a52796cf7dd7810e7c0a3
Jul 24 17:56:38.890988 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:38.890950    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft" event={"ID":"a42cbded-018b-416f-b096-a2ffa2b58d2b","Type":"ContainerStarted","Data":"7e5e1028c8728248acb7415cc77925d13447364bed6a3d99719fdceb03bb555d"}
Jul 24 17:56:38.891505 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:38.891010    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft" event={"ID":"a42cbded-018b-416f-b096-a2ffa2b58d2b","Type":"ContainerStarted","Data":"e9d02fc9c0a2fed7b2afff77c55a2eb748672724478a52796cf7dd7810e7c0a3"}
Jul 24 17:56:38.891505 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:38.891056    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft"
Jul 24 17:56:38.892575 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:38.892544    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd" event={"ID":"20852f34-832c-4899-8116-c6b6e1f8a425","Type":"ContainerStarted","Data":"4df9a78be95a1d29106c0880703a220f33324336a9a80ddb94d1ae870950271c"}
Jul 24 17:56:38.892676 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:38.892602    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd"
Jul 24 17:56:38.893919 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:38.893899    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6" event={"ID":"29542207-0e8d-483b-aa3b-040e4d8b0acc","Type":"ContainerStarted","Data":"69fa59209666df0b2f0cb0fc6c32224e215bd4a2ca2484e66edde908dd0ae1f3"}
Jul 24 17:56:38.894021 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:38.893933    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6" event={"ID":"29542207-0e8d-483b-aa3b-040e4d8b0acc","Type":"ContainerStarted","Data":"a3723b710bb4e316a77b968af739948ba55357b52a204a6f0e9f493917c8f298"}
Jul 24 17:56:38.894064 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:38.894033    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6"
Jul 24 17:56:38.927980 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:38.927927    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft" podStartSLOduration=1.927908221 podStartE2EDuration="1.927908221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:56:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:56:38.911017351 +0000 UTC m=+2361.638173566" watchObservedRunningTime="2026-07-24 17:56:38.927908221 +0000 UTC m=+2361.655064441"
Jul 24 17:56:38.929725 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:38.929685    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd" podStartSLOduration=2.575226117 podStartE2EDuration="2.929669545s" totalImagesPullingTime="354.443428ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:56:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:56:38.927441935 +0000 UTC m=+2361.654598151" watchObservedRunningTime="2026-07-24 17:56:38.929669545 +0000 UTC m=+2361.656825759"
Jul 24 17:56:40.850821 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:40.850784    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j"
Jul 24 17:56:40.868892 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:40.868842    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6" podStartSLOduration=3.868825485 podStartE2EDuration="3.868825485s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:56:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:56:38.952167406 +0000 UTC m=+2361.679323613" watchObservedRunningTime="2026-07-24 17:56:40.868825485 +0000 UTC m=+2363.595981712"
Jul 24 17:56:41.855945 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:41.855911    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65"
Jul 24 17:56:44.905291 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:44.905259    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd"
Jul 24 17:56:49.901314 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:49.901282    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6"
Jul 24 17:56:49.901874 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:56:49.901683    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft"
Jul 24 17:58:18.077292 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.077260    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2"]
Jul 24 17:58:18.077737 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.077490    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2" podUID="4f37fdfc-18ec-45f5-8344-3b9681677935" containerName="maas-api" containerID="cri-o://924574e8ccae4cb2c4142e93cf35a2fca2946b49f365df19cdd236d8d67bff42" gracePeriod=30
Jul 24 17:58:18.164136 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.164106    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j"]
Jul 24 17:58:18.164366 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.164325    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j" podUID="8452d620-74d7-4a94-81e6-a6fbd919eb9f" containerName="payload-processing" containerID="cri-o://ea88a840bc1e93b2b0ae71461a02e27bff36d0e2776092c3f0bf61020ad6c72a" gracePeriod=30
Jul 24 17:58:18.170921 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.170897    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65"]
Jul 24 17:58:18.171112 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.171088    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65" podUID="f021e2fe-b467-4dcd-a5a5-819ef65059df" containerName="payload-pre-processing" containerID="cri-o://3fbd046d8109d6d105fb240a91b47d91bb4309b28261b048d1412c9197d73def" gracePeriod=30
Jul 24 17:58:18.420292 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.420233    2562 generic.go:356] "Generic (PLEG): container finished" podID="4f37fdfc-18ec-45f5-8344-3b9681677935" containerID="924574e8ccae4cb2c4142e93cf35a2fca2946b49f365df19cdd236d8d67bff42" exitCode=0
Jul 24 17:58:18.420454 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.420310    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2" event={"ID":"4f37fdfc-18ec-45f5-8344-3b9681677935","Type":"ContainerDied","Data":"924574e8ccae4cb2c4142e93cf35a2fca2946b49f365df19cdd236d8d67bff42"}
Jul 24 17:58:18.420528 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.420461    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2" event={"ID":"4f37fdfc-18ec-45f5-8344-3b9681677935","Type":"ContainerDied","Data":"4812dbabb67af813ff8d89952699d1903e07efc257af3690f7222f70a70d1f23"}
Jul 24 17:58:18.420528 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.420495    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4812dbabb67af813ff8d89952699d1903e07efc257af3690f7222f70a70d1f23"
Jul 24 17:58:18.422968 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.422946    2562 generic.go:356] "Generic (PLEG): container finished" podID="f021e2fe-b467-4dcd-a5a5-819ef65059df" containerID="3fbd046d8109d6d105fb240a91b47d91bb4309b28261b048d1412c9197d73def" exitCode=0
Jul 24 17:58:18.423084 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.423020    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65" event={"ID":"f021e2fe-b467-4dcd-a5a5-819ef65059df","Type":"ContainerDied","Data":"3fbd046d8109d6d105fb240a91b47d91bb4309b28261b048d1412c9197d73def"}
Jul 24 17:58:18.424922 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.424899    2562 generic.go:356] "Generic (PLEG): container finished" podID="8452d620-74d7-4a94-81e6-a6fbd919eb9f" containerID="ea88a840bc1e93b2b0ae71461a02e27bff36d0e2776092c3f0bf61020ad6c72a" exitCode=0
Jul 24 17:58:18.425022 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.424932    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j" event={"ID":"8452d620-74d7-4a94-81e6-a6fbd919eb9f","Type":"ContainerDied","Data":"ea88a840bc1e93b2b0ae71461a02e27bff36d0e2776092c3f0bf61020ad6c72a"}
Jul 24 17:58:18.436014 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.435990    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2"
Jul 24 17:58:18.480121 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.480092    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j"
Jul 24 17:58:18.485994 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.485967    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65"
Jul 24 17:58:18.588209 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.588109    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f37fdfc-18ec-45f5-8344-3b9681677935-openshift-service-ca\") pod \"4f37fdfc-18ec-45f5-8344-3b9681677935\" (UID: \"4f37fdfc-18ec-45f5-8344-3b9681677935\") "
Jul 24 17:58:18.588372 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.588206    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f37fdfc-18ec-45f5-8344-3b9681677935-maas-api-tls\") pod \"4f37fdfc-18ec-45f5-8344-3b9681677935\" (UID: \"4f37fdfc-18ec-45f5-8344-3b9681677935\") "
Jul 24 17:58:18.588372 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.588233    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f021e2fe-b467-4dcd-a5a5-819ef65059df-plugins-config-volume\") pod \"f021e2fe-b467-4dcd-a5a5-819ef65059df\" (UID: \"f021e2fe-b467-4dcd-a5a5-819ef65059df\") "
Jul 24 17:58:18.588372 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.588281    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8kz9\" (UniqueName: \"kubernetes.io/projected/4f37fdfc-18ec-45f5-8344-3b9681677935-kube-api-access-d8kz9\") pod \"4f37fdfc-18ec-45f5-8344-3b9681677935\" (UID: \"4f37fdfc-18ec-45f5-8344-3b9681677935\") "
Jul 24 17:58:18.588372 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.588341    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqz6c\" (UniqueName: \"kubernetes.io/projected/8452d620-74d7-4a94-81e6-a6fbd919eb9f-kube-api-access-wqz6c\") pod \"8452d620-74d7-4a94-81e6-a6fbd919eb9f\" (UID: \"8452d620-74d7-4a94-81e6-a6fbd919eb9f\") "
Jul 24 17:58:18.588372 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.588366    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8452d620-74d7-4a94-81e6-a6fbd919eb9f-plugins-config-volume\") pod \"8452d620-74d7-4a94-81e6-a6fbd919eb9f\" (UID: \"8452d620-74d7-4a94-81e6-a6fbd919eb9f\") "
Jul 24 17:58:18.588636 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.588419    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29mql\" (UniqueName: \"kubernetes.io/projected/f021e2fe-b467-4dcd-a5a5-819ef65059df-kube-api-access-29mql\") pod \"f021e2fe-b467-4dcd-a5a5-819ef65059df\" (UID: \"f021e2fe-b467-4dcd-a5a5-819ef65059df\") "
Jul 24 17:58:18.588636 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.588551    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f37fdfc-18ec-45f5-8344-3b9681677935-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4f37fdfc-18ec-45f5-8344-3b9681677935" (UID: "4f37fdfc-18ec-45f5-8344-3b9681677935"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:58:18.588810 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.588784    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f021e2fe-b467-4dcd-a5a5-819ef65059df-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f021e2fe-b467-4dcd-a5a5-819ef65059df" (UID: "f021e2fe-b467-4dcd-a5a5-819ef65059df"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:58:18.588883 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.588823    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8452d620-74d7-4a94-81e6-a6fbd919eb9f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8452d620-74d7-4a94-81e6-a6fbd919eb9f" (UID: "8452d620-74d7-4a94-81e6-a6fbd919eb9f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:58:18.588967 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.588946    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f37fdfc-18ec-45f5-8344-3b9681677935-openshift-service-ca\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:58:18.589024 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.588974    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f021e2fe-b467-4dcd-a5a5-819ef65059df-plugins-config-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:58:18.589024 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.588990    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8452d620-74d7-4a94-81e6-a6fbd919eb9f-plugins-config-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:58:18.590531 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.590486    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8452d620-74d7-4a94-81e6-a6fbd919eb9f-kube-api-access-wqz6c" (OuterVolumeSpecName: "kube-api-access-wqz6c") pod "8452d620-74d7-4a94-81e6-a6fbd919eb9f" (UID: "8452d620-74d7-4a94-81e6-a6fbd919eb9f"). InnerVolumeSpecName "kube-api-access-wqz6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:58:18.590636 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.590525    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f021e2fe-b467-4dcd-a5a5-819ef65059df-kube-api-access-29mql" (OuterVolumeSpecName: "kube-api-access-29mql") pod "f021e2fe-b467-4dcd-a5a5-819ef65059df" (UID: "f021e2fe-b467-4dcd-a5a5-819ef65059df"). InnerVolumeSpecName "kube-api-access-29mql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:58:18.590636 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.590553    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f37fdfc-18ec-45f5-8344-3b9681677935-kube-api-access-d8kz9" (OuterVolumeSpecName: "kube-api-access-d8kz9") pod "4f37fdfc-18ec-45f5-8344-3b9681677935" (UID: "4f37fdfc-18ec-45f5-8344-3b9681677935"). InnerVolumeSpecName "kube-api-access-d8kz9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:58:18.590636 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.590578    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f37fdfc-18ec-45f5-8344-3b9681677935-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4f37fdfc-18ec-45f5-8344-3b9681677935" (UID: "4f37fdfc-18ec-45f5-8344-3b9681677935"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:58:18.690304 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.690270    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqz6c\" (UniqueName: \"kubernetes.io/projected/8452d620-74d7-4a94-81e6-a6fbd919eb9f-kube-api-access-wqz6c\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:58:18.690304 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.690299    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29mql\" (UniqueName: \"kubernetes.io/projected/f021e2fe-b467-4dcd-a5a5-819ef65059df-kube-api-access-29mql\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:58:18.690304 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.690310    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f37fdfc-18ec-45f5-8344-3b9681677935-maas-api-tls\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:58:18.690526 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:18.690320    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8kz9\" (UniqueName: \"kubernetes.io/projected/4f37fdfc-18ec-45f5-8344-3b9681677935-kube-api-access-d8kz9\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:58:19.431549 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:19.431465    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65"
Jul 24 17:58:19.431992 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:19.431464    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65" event={"ID":"f021e2fe-b467-4dcd-a5a5-819ef65059df","Type":"ContainerDied","Data":"be7117ce6c90717142277fd92281803c22ced5c3a40df6f34c16f3a14c8afc4f"}
Jul 24 17:58:19.431992 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:19.431636    2562 scope.go:117] "RemoveContainer" containerID="3fbd046d8109d6d105fb240a91b47d91bb4309b28261b048d1412c9197d73def"
Jul 24 17:58:19.433020 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:19.432973    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j"
Jul 24 17:58:19.433020 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:19.433010    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j" event={"ID":"8452d620-74d7-4a94-81e6-a6fbd919eb9f","Type":"ContainerDied","Data":"d3c3ef525850c77a29e7da176e261e9ee7998685107dc1335de2c8445d922655"}
Jul 24 17:58:19.433249 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:19.433215    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2"
Jul 24 17:58:19.444547 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:19.444529    2562 scope.go:117] "RemoveContainer" containerID="ea88a840bc1e93b2b0ae71461a02e27bff36d0e2776092c3f0bf61020ad6c72a"
Jul 24 17:58:19.469065 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:19.469041    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65"]
Jul 24 17:58:19.474627 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:19.474604    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-cdf40f-7785f8db5b-rbs65"]
Jul 24 17:58:19.486837 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:19.486816    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j"]
Jul 24 17:58:19.492482 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:19.492461    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-cdf40f-5c6bd88df6-zhp2j"]
Jul 24 17:58:19.503876 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:19.503856    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2"]
Jul 24 17:58:19.508954 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:19.508931    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cdf40f-7d9b46d486-6zvv2"]
Jul 24 17:58:19.824518 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:19.824432    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f37fdfc-18ec-45f5-8344-3b9681677935" path="/var/lib/kubelet/pods/4f37fdfc-18ec-45f5-8344-3b9681677935/volumes"
Jul 24 17:58:19.824790 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:19.824778    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8452d620-74d7-4a94-81e6-a6fbd919eb9f" path="/var/lib/kubelet/pods/8452d620-74d7-4a94-81e6-a6fbd919eb9f/volumes"
Jul 24 17:58:19.825097 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:19.825085    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f021e2fe-b467-4dcd-a5a5-819ef65059df" path="/var/lib/kubelet/pods/f021e2fe-b467-4dcd-a5a5-819ef65059df/volumes"
Jul 24 17:58:40.752061 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:40.752022    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd"]
Jul 24 17:58:40.752651 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:40.752299    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd" podUID="20852f34-832c-4899-8116-c6b6e1f8a425" containerName="maas-api" containerID="cri-o://4df9a78be95a1d29106c0880703a220f33324336a9a80ddb94d1ae870950271c" gracePeriod=30
Jul 24 17:58:40.799092 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:40.799059    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft"]
Jul 24 17:58:40.799315 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:40.799290    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft" podUID="a42cbded-018b-416f-b096-a2ffa2b58d2b" containerName="payload-processing" containerID="cri-o://7e5e1028c8728248acb7415cc77925d13447364bed6a3d99719fdceb03bb555d" gracePeriod=30
Jul 24 17:58:40.810240 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:40.810213    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6"]
Jul 24 17:58:40.810601 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:40.810576    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6" podUID="29542207-0e8d-483b-aa3b-040e4d8b0acc" containerName="payload-pre-processing" containerID="cri-o://69fa59209666df0b2f0cb0fc6c32224e215bd4a2ca2484e66edde908dd0ae1f3" gracePeriod=30
Jul 24 17:58:41.079477 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.079024    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd"
Jul 24 17:58:41.082372 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.082350    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft"
Jul 24 17:58:41.113135 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.113114    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6"
Jul 24 17:58:41.180631 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.180605    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lsmvk\" (UniqueName: \"kubernetes.io/projected/20852f34-832c-4899-8116-c6b6e1f8a425-kube-api-access-lsmvk\") pod \"20852f34-832c-4899-8116-c6b6e1f8a425\" (UID: \"20852f34-832c-4899-8116-c6b6e1f8a425\") "
Jul 24 17:58:41.180790 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.180645    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20852f34-832c-4899-8116-c6b6e1f8a425-openshift-service-ca\") pod \"20852f34-832c-4899-8116-c6b6e1f8a425\" (UID: \"20852f34-832c-4899-8116-c6b6e1f8a425\") "
Jul 24 17:58:41.180790 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.180674    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29542207-0e8d-483b-aa3b-040e4d8b0acc-plugins-config-volume\") pod \"29542207-0e8d-483b-aa3b-040e4d8b0acc\" (UID: \"29542207-0e8d-483b-aa3b-040e4d8b0acc\") "
Jul 24 17:58:41.180923 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.180787    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20852f34-832c-4899-8116-c6b6e1f8a425-maas-api-tls\") pod \"20852f34-832c-4899-8116-c6b6e1f8a425\" (UID: \"20852f34-832c-4899-8116-c6b6e1f8a425\") "
Jul 24 17:58:41.180923 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.180844    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2ffg\" (UniqueName: \"kubernetes.io/projected/29542207-0e8d-483b-aa3b-040e4d8b0acc-kube-api-access-k2ffg\") pod \"29542207-0e8d-483b-aa3b-040e4d8b0acc\" (UID: \"29542207-0e8d-483b-aa3b-040e4d8b0acc\") "
Jul 24 17:58:41.180923 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.180887    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a42cbded-018b-416f-b096-a2ffa2b58d2b-plugins-config-volume\") pod \"a42cbded-018b-416f-b096-a2ffa2b58d2b\" (UID: \"a42cbded-018b-416f-b096-a2ffa2b58d2b\") "
Jul 24 17:58:41.181077 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.180928    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgbtx\" (UniqueName: \"kubernetes.io/projected/a42cbded-018b-416f-b096-a2ffa2b58d2b-kube-api-access-dgbtx\") pod \"a42cbded-018b-416f-b096-a2ffa2b58d2b\" (UID: \"a42cbded-018b-416f-b096-a2ffa2b58d2b\") "
Jul 24 17:58:41.181077 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.180995    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20852f34-832c-4899-8116-c6b6e1f8a425-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "20852f34-832c-4899-8116-c6b6e1f8a425" (UID: "20852f34-832c-4899-8116-c6b6e1f8a425"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:58:41.181077 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.181007    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29542207-0e8d-483b-aa3b-040e4d8b0acc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "29542207-0e8d-483b-aa3b-040e4d8b0acc" (UID: "29542207-0e8d-483b-aa3b-040e4d8b0acc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:58:41.181258 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.181239    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a42cbded-018b-416f-b096-a2ffa2b58d2b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a42cbded-018b-416f-b096-a2ffa2b58d2b" (UID: "a42cbded-018b-416f-b096-a2ffa2b58d2b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:58:41.181360 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.181342    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20852f34-832c-4899-8116-c6b6e1f8a425-openshift-service-ca\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:58:41.181424 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.181365    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29542207-0e8d-483b-aa3b-040e4d8b0acc-plugins-config-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:58:41.181424 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.181381    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a42cbded-018b-416f-b096-a2ffa2b58d2b-plugins-config-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:58:41.182838 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.182816    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20852f34-832c-4899-8116-c6b6e1f8a425-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "20852f34-832c-4899-8116-c6b6e1f8a425" (UID: "20852f34-832c-4899-8116-c6b6e1f8a425"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:58:41.182995 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.182976    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a42cbded-018b-416f-b096-a2ffa2b58d2b-kube-api-access-dgbtx" (OuterVolumeSpecName: "kube-api-access-dgbtx") pod "a42cbded-018b-416f-b096-a2ffa2b58d2b" (UID: "a42cbded-018b-416f-b096-a2ffa2b58d2b"). InnerVolumeSpecName "kube-api-access-dgbtx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:58:41.183223 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.183200    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20852f34-832c-4899-8116-c6b6e1f8a425-kube-api-access-lsmvk" (OuterVolumeSpecName: "kube-api-access-lsmvk") pod "20852f34-832c-4899-8116-c6b6e1f8a425" (UID: "20852f34-832c-4899-8116-c6b6e1f8a425"). InnerVolumeSpecName "kube-api-access-lsmvk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:58:41.183315 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.183226    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29542207-0e8d-483b-aa3b-040e4d8b0acc-kube-api-access-k2ffg" (OuterVolumeSpecName: "kube-api-access-k2ffg") pod "29542207-0e8d-483b-aa3b-040e4d8b0acc" (UID: "29542207-0e8d-483b-aa3b-040e4d8b0acc"). InnerVolumeSpecName "kube-api-access-k2ffg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:58:41.282401 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.282333    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgbtx\" (UniqueName: \"kubernetes.io/projected/a42cbded-018b-416f-b096-a2ffa2b58d2b-kube-api-access-dgbtx\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:58:41.282401 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.282355    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsmvk\" (UniqueName: \"kubernetes.io/projected/20852f34-832c-4899-8116-c6b6e1f8a425-kube-api-access-lsmvk\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:58:41.282401 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.282364    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20852f34-832c-4899-8116-c6b6e1f8a425-maas-api-tls\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:58:41.282401 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.282373    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2ffg\" (UniqueName: \"kubernetes.io/projected/29542207-0e8d-483b-aa3b-040e4d8b0acc-kube-api-access-k2ffg\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:58:41.549361 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.549272    2562 generic.go:356] "Generic (PLEG): container finished" podID="a42cbded-018b-416f-b096-a2ffa2b58d2b" containerID="7e5e1028c8728248acb7415cc77925d13447364bed6a3d99719fdceb03bb555d" exitCode=0
Jul 24 17:58:41.549361 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.549323    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft"
Jul 24 17:58:41.549568 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.549356    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft" event={"ID":"a42cbded-018b-416f-b096-a2ffa2b58d2b","Type":"ContainerDied","Data":"7e5e1028c8728248acb7415cc77925d13447364bed6a3d99719fdceb03bb555d"}
Jul 24 17:58:41.549568 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.549418    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft" event={"ID":"a42cbded-018b-416f-b096-a2ffa2b58d2b","Type":"ContainerDied","Data":"e9d02fc9c0a2fed7b2afff77c55a2eb748672724478a52796cf7dd7810e7c0a3"}
Jul 24 17:58:41.549568 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.549450    2562 scope.go:117] "RemoveContainer" containerID="7e5e1028c8728248acb7415cc77925d13447364bed6a3d99719fdceb03bb555d"
Jul 24 17:58:41.550738 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.550719    2562 generic.go:356] "Generic (PLEG): container finished" podID="20852f34-832c-4899-8116-c6b6e1f8a425" containerID="4df9a78be95a1d29106c0880703a220f33324336a9a80ddb94d1ae870950271c" exitCode=0
Jul 24 17:58:41.550836 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.550797    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd"
Jul 24 17:58:41.550911 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.550797    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd" event={"ID":"20852f34-832c-4899-8116-c6b6e1f8a425","Type":"ContainerDied","Data":"4df9a78be95a1d29106c0880703a220f33324336a9a80ddb94d1ae870950271c"}
Jul 24 17:58:41.550973 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.550932    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd" event={"ID":"20852f34-832c-4899-8116-c6b6e1f8a425","Type":"ContainerDied","Data":"004eb11fc9746a6f4ff5c616351735f0cb0459b27f8d91af2e5416dd9fe6fd88"}
Jul 24 17:58:41.552269 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.552244    2562 generic.go:356] "Generic (PLEG): container finished" podID="29542207-0e8d-483b-aa3b-040e4d8b0acc" containerID="69fa59209666df0b2f0cb0fc6c32224e215bd4a2ca2484e66edde908dd0ae1f3" exitCode=0
Jul 24 17:58:41.552397 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.552292    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6"
Jul 24 17:58:41.552397 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.552293    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6" event={"ID":"29542207-0e8d-483b-aa3b-040e4d8b0acc","Type":"ContainerDied","Data":"69fa59209666df0b2f0cb0fc6c32224e215bd4a2ca2484e66edde908dd0ae1f3"}
Jul 24 17:58:41.552493 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.552408    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6" event={"ID":"29542207-0e8d-483b-aa3b-040e4d8b0acc","Type":"ContainerDied","Data":"a3723b710bb4e316a77b968af739948ba55357b52a204a6f0e9f493917c8f298"}
Jul 24 17:58:41.560506 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.560494    2562 scope.go:117] "RemoveContainer" containerID="7e5e1028c8728248acb7415cc77925d13447364bed6a3d99719fdceb03bb555d"
Jul 24 17:58:41.560777 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:58:41.560755    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e5e1028c8728248acb7415cc77925d13447364bed6a3d99719fdceb03bb555d\": container with ID starting with 7e5e1028c8728248acb7415cc77925d13447364bed6a3d99719fdceb03bb555d not found: ID does not exist" containerID="7e5e1028c8728248acb7415cc77925d13447364bed6a3d99719fdceb03bb555d"
Jul 24 17:58:41.560854 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.560782    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e5e1028c8728248acb7415cc77925d13447364bed6a3d99719fdceb03bb555d"} err="failed to get container status \"7e5e1028c8728248acb7415cc77925d13447364bed6a3d99719fdceb03bb555d\": rpc error: code = NotFound desc = could not find container \"7e5e1028c8728248acb7415cc77925d13447364bed6a3d99719fdceb03bb555d\": container with ID starting with 7e5e1028c8728248acb7415cc77925d13447364bed6a3d99719fdceb03bb555d not found: ID does not exist"
Jul 24 17:58:41.560854 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.560797    2562 scope.go:117] "RemoveContainer" containerID="4df9a78be95a1d29106c0880703a220f33324336a9a80ddb94d1ae870950271c"
Jul 24 17:58:41.568876 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.568859    2562 scope.go:117] "RemoveContainer" containerID="4df9a78be95a1d29106c0880703a220f33324336a9a80ddb94d1ae870950271c"
Jul 24 17:58:41.569158 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:58:41.569123    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4df9a78be95a1d29106c0880703a220f33324336a9a80ddb94d1ae870950271c\": container with ID starting with 4df9a78be95a1d29106c0880703a220f33324336a9a80ddb94d1ae870950271c not found: ID does not exist" containerID="4df9a78be95a1d29106c0880703a220f33324336a9a80ddb94d1ae870950271c"
Jul 24 17:58:41.569209 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.569160    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4df9a78be95a1d29106c0880703a220f33324336a9a80ddb94d1ae870950271c"} err="failed to get container status \"4df9a78be95a1d29106c0880703a220f33324336a9a80ddb94d1ae870950271c\": rpc error: code = NotFound desc = could not find container \"4df9a78be95a1d29106c0880703a220f33324336a9a80ddb94d1ae870950271c\": container with ID starting with 4df9a78be95a1d29106c0880703a220f33324336a9a80ddb94d1ae870950271c not found: ID does not exist"
Jul 24 17:58:41.569209 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.569175    2562 scope.go:117] "RemoveContainer" containerID="69fa59209666df0b2f0cb0fc6c32224e215bd4a2ca2484e66edde908dd0ae1f3"
Jul 24 17:58:41.575108 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.575086    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft"]
Jul 24 17:58:41.579046 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.579025    2562 scope.go:117] "RemoveContainer" containerID="69fa59209666df0b2f0cb0fc6c32224e215bd4a2ca2484e66edde908dd0ae1f3"
Jul 24 17:58:41.579669 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:58:41.579644    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69fa59209666df0b2f0cb0fc6c32224e215bd4a2ca2484e66edde908dd0ae1f3\": container with ID starting with 69fa59209666df0b2f0cb0fc6c32224e215bd4a2ca2484e66edde908dd0ae1f3 not found: ID does not exist" containerID="69fa59209666df0b2f0cb0fc6c32224e215bd4a2ca2484e66edde908dd0ae1f3"
Jul 24 17:58:41.579785 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.579675    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69fa59209666df0b2f0cb0fc6c32224e215bd4a2ca2484e66edde908dd0ae1f3"} err="failed to get container status \"69fa59209666df0b2f0cb0fc6c32224e215bd4a2ca2484e66edde908dd0ae1f3\": rpc error: code = NotFound desc = could not find container \"69fa59209666df0b2f0cb0fc6c32224e215bd4a2ca2484e66edde908dd0ae1f3\": container with ID starting with 69fa59209666df0b2f0cb0fc6c32224e215bd4a2ca2484e66edde908dd0ae1f3 not found: ID does not exist"
Jul 24 17:58:41.582322 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.582303    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3922c3-66857479fb-j4rft"]
Jul 24 17:58:41.594120 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.594099    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd"]
Jul 24 17:58:41.598782 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.598763    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3922c3-66764b6fb4-9b8qd"]
Jul 24 17:58:41.613109 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.613086    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6"]
Jul 24 17:58:41.615835 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.615810    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3922c3-54b7d8cfb-ktwn6"]
Jul 24 17:58:41.825806 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.825719    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20852f34-832c-4899-8116-c6b6e1f8a425" path="/var/lib/kubelet/pods/20852f34-832c-4899-8116-c6b6e1f8a425/volumes"
Jul 24 17:58:41.826262 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.826246    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29542207-0e8d-483b-aa3b-040e4d8b0acc" path="/var/lib/kubelet/pods/29542207-0e8d-483b-aa3b-040e4d8b0acc/volumes"
Jul 24 17:58:41.826620 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:41.826601    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a42cbded-018b-416f-b096-a2ffa2b58d2b" path="/var/lib/kubelet/pods/a42cbded-018b-416f-b096-a2ffa2b58d2b/volumes"
Jul 24 17:58:57.528438 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.528404    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv"]
Jul 24 17:58:57.529272 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.528819    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8452d620-74d7-4a94-81e6-a6fbd919eb9f" containerName="payload-processing"
Jul 24 17:58:57.529272 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.528831    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8452d620-74d7-4a94-81e6-a6fbd919eb9f" containerName="payload-processing"
Jul 24 17:58:57.529272 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.528845    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f37fdfc-18ec-45f5-8344-3b9681677935" containerName="maas-api"
Jul 24 17:58:57.529272 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.528851    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f37fdfc-18ec-45f5-8344-3b9681677935" containerName="maas-api"
Jul 24 17:58:57.529272 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.528859    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f021e2fe-b467-4dcd-a5a5-819ef65059df" containerName="payload-pre-processing"
Jul 24 17:58:57.529272 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.528868    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f021e2fe-b467-4dcd-a5a5-819ef65059df" containerName="payload-pre-processing"
Jul 24 17:58:57.529272 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.528883    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a42cbded-018b-416f-b096-a2ffa2b58d2b" containerName="payload-processing"
Jul 24 17:58:57.529272 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.528890    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a42cbded-018b-416f-b096-a2ffa2b58d2b" containerName="payload-processing"
Jul 24 17:58:57.529272 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.528897    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20852f34-832c-4899-8116-c6b6e1f8a425" containerName="maas-api"
Jul 24 17:58:57.529272 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.528902    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="20852f34-832c-4899-8116-c6b6e1f8a425" containerName="maas-api"
Jul 24 17:58:57.529272 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.528918    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29542207-0e8d-483b-aa3b-040e4d8b0acc" containerName="payload-pre-processing"
Jul 24 17:58:57.529272 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.528923    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="29542207-0e8d-483b-aa3b-040e4d8b0acc" containerName="payload-pre-processing"
Jul 24 17:58:57.529272 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.528984    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="20852f34-832c-4899-8116-c6b6e1f8a425" containerName="maas-api"
Jul 24 17:58:57.529272 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.528995    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f021e2fe-b467-4dcd-a5a5-819ef65059df" containerName="payload-pre-processing"
Jul 24 17:58:57.529272 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.529003    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a42cbded-018b-416f-b096-a2ffa2b58d2b" containerName="payload-processing"
Jul 24 17:58:57.529272 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.529009    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="29542207-0e8d-483b-aa3b-040e4d8b0acc" containerName="payload-pre-processing"
Jul 24 17:58:57.529272 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.529016    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8452d620-74d7-4a94-81e6-a6fbd919eb9f" containerName="payload-processing"
Jul 24 17:58:57.529272 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.529021    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f37fdfc-18ec-45f5-8344-3b9681677935" containerName="maas-api"
Jul 24 17:58:57.532241 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.532224    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv"
Jul 24 17:58:57.535374 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.535352    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-5e940d-serving-cert\""
Jul 24 17:58:57.548114 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.548088    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv"]
Jul 24 17:58:57.635396 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.635359    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d85b59a-2c8e-43a5-8189-f1a773b186b0-maas-api-tls\") pod \"maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv\" (UID: \"3d85b59a-2c8e-43a5-8189-f1a773b186b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv"
Jul 24 17:58:57.635693 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.635442    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsv8w\" (UniqueName: \"kubernetes.io/projected/3d85b59a-2c8e-43a5-8189-f1a773b186b0-kube-api-access-hsv8w\") pod \"maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv\" (UID: \"3d85b59a-2c8e-43a5-8189-f1a773b186b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv"
Jul 24 17:58:57.635693 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.635511    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d85b59a-2c8e-43a5-8189-f1a773b186b0-openshift-service-ca\") pod \"maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv\" (UID: \"3d85b59a-2c8e-43a5-8189-f1a773b186b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv"
Jul 24 17:58:57.736297 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.736255    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d85b59a-2c8e-43a5-8189-f1a773b186b0-maas-api-tls\") pod \"maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv\" (UID: \"3d85b59a-2c8e-43a5-8189-f1a773b186b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv"
Jul 24 17:58:57.736457 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.736314    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsv8w\" (UniqueName: \"kubernetes.io/projected/3d85b59a-2c8e-43a5-8189-f1a773b186b0-kube-api-access-hsv8w\") pod \"maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv\" (UID: \"3d85b59a-2c8e-43a5-8189-f1a773b186b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv"
Jul 24 17:58:57.736457 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.736364    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d85b59a-2c8e-43a5-8189-f1a773b186b0-openshift-service-ca\") pod \"maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv\" (UID: \"3d85b59a-2c8e-43a5-8189-f1a773b186b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv"
Jul 24 17:58:57.736457 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:58:57.736410    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-serving-cert: secret "maas-api-e2e-api-a-5e940d-serving-cert" not found
Jul 24 17:58:57.736571 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:58:57.736472    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d85b59a-2c8e-43a5-8189-f1a773b186b0-maas-api-tls podName:3d85b59a-2c8e-43a5-8189-f1a773b186b0 nodeName:}" failed. No retries permitted until 2026-07-24 17:58:58.236454592 +0000 UTC m=+2500.963610785 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3d85b59a-2c8e-43a5-8189-f1a773b186b0-maas-api-tls") pod "maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv" (UID: "3d85b59a-2c8e-43a5-8189-f1a773b186b0") : secret "maas-api-e2e-api-a-5e940d-serving-cert" not found
Jul 24 17:58:57.737100 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.737081    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d85b59a-2c8e-43a5-8189-f1a773b186b0-openshift-service-ca\") pod \"maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv\" (UID: \"3d85b59a-2c8e-43a5-8189-f1a773b186b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv"
Jul 24 17:58:57.765010 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:57.764978    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsv8w\" (UniqueName: \"kubernetes.io/projected/3d85b59a-2c8e-43a5-8189-f1a773b186b0-kube-api-access-hsv8w\") pod \"maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv\" (UID: \"3d85b59a-2c8e-43a5-8189-f1a773b186b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv"
Jul 24 17:58:58.241872 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:58.241804    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d85b59a-2c8e-43a5-8189-f1a773b186b0-maas-api-tls\") pod \"maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv\" (UID: \"3d85b59a-2c8e-43a5-8189-f1a773b186b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv"
Jul 24 17:58:58.244328 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:58.244304    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d85b59a-2c8e-43a5-8189-f1a773b186b0-maas-api-tls\") pod \"maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv\" (UID: \"3d85b59a-2c8e-43a5-8189-f1a773b186b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv"
Jul 24 17:58:58.442330 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:58.442295    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv"
Jul 24 17:58:58.612485 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:58.612459    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv"]
Jul 24 17:58:58.615346 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:58:58.615312    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d85b59a_2c8e_43a5_8189_f1a773b186b0.slice/crio-b86bb7f9ae5dd4bfd3e4e731002a65c81abdab2f12aa40927bf58409294b6e08 WatchSource:0}: Error finding container b86bb7f9ae5dd4bfd3e4e731002a65c81abdab2f12aa40927bf58409294b6e08: Status 404 returned error can't find the container with id b86bb7f9ae5dd4bfd3e4e731002a65c81abdab2f12aa40927bf58409294b6e08
Jul 24 17:58:58.616829 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:58.616799    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 17:58:58.647772 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:58.647737    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv" event={"ID":"3d85b59a-2c8e-43a5-8189-f1a773b186b0","Type":"ContainerStarted","Data":"b86bb7f9ae5dd4bfd3e4e731002a65c81abdab2f12aa40927bf58409294b6e08"}
Jul 24 17:58:58.731414 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:58.731379    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc"]
Jul 24 17:58:58.736030 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:58.736008    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc"
Jul 24 17:58:58.739756 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:58.739735    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-5e940d\""
Jul 24 17:58:58.751669 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:58.751617    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc"]
Jul 24 17:58:58.795421 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:58.795390    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7"]
Jul 24 17:58:58.798880 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:58.798854    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7"
Jul 24 17:58:58.805205 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:58.805186    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-5e940d-dockercfg-mf882\""
Jul 24 17:58:58.847050 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:58.847021    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hspx6\" (UniqueName: \"kubernetes.io/projected/8991b189-5bed-4acc-8867-f77b660108ae-kube-api-access-hspx6\") pod \"payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc\" (UID: \"8991b189-5bed-4acc-8867-f77b660108ae\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc"
Jul 24 17:58:58.847255 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:58.847064    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8991b189-5bed-4acc-8867-f77b660108ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc\" (UID: \"8991b189-5bed-4acc-8867-f77b660108ae\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc"
Jul 24 17:58:58.849423 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:58.849400    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7"]
Jul 24 17:58:58.948039 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:58.948005    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlxxv\" (UniqueName: \"kubernetes.io/projected/14f782c1-2807-4da5-8d5f-fdff99fadf38-kube-api-access-zlxxv\") pod \"payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7\" (UID: \"14f782c1-2807-4da5-8d5f-fdff99fadf38\") " pod="openshift-ingress/payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7"
Jul 24 17:58:58.948231 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:58.948045    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hspx6\" (UniqueName: \"kubernetes.io/projected/8991b189-5bed-4acc-8867-f77b660108ae-kube-api-access-hspx6\") pod \"payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc\" (UID: \"8991b189-5bed-4acc-8867-f77b660108ae\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc"
Jul 24 17:58:58.948231 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:58.948094    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8991b189-5bed-4acc-8867-f77b660108ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc\" (UID: \"8991b189-5bed-4acc-8867-f77b660108ae\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc"
Jul 24 17:58:58.948326 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:58.948280    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14f782c1-2807-4da5-8d5f-fdff99fadf38-plugins-config-volume\") pod \"payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7\" (UID: \"14f782c1-2807-4da5-8d5f-fdff99fadf38\") " pod="openshift-ingress/payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7"
Jul 24 17:58:58.948813 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:58.948795    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8991b189-5bed-4acc-8867-f77b660108ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc\" (UID: \"8991b189-5bed-4acc-8867-f77b660108ae\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc"
Jul 24 17:58:58.961923 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:58.961898    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hspx6\" (UniqueName: \"kubernetes.io/projected/8991b189-5bed-4acc-8867-f77b660108ae-kube-api-access-hspx6\") pod \"payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc\" (UID: \"8991b189-5bed-4acc-8867-f77b660108ae\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc"
Jul 24 17:58:59.046539 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:59.046516    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc"
Jul 24 17:58:59.048969 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:59.048942    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14f782c1-2807-4da5-8d5f-fdff99fadf38-plugins-config-volume\") pod \"payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7\" (UID: \"14f782c1-2807-4da5-8d5f-fdff99fadf38\") " pod="openshift-ingress/payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7"
Jul 24 17:58:59.049059 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:59.049030    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlxxv\" (UniqueName: \"kubernetes.io/projected/14f782c1-2807-4da5-8d5f-fdff99fadf38-kube-api-access-zlxxv\") pod \"payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7\" (UID: \"14f782c1-2807-4da5-8d5f-fdff99fadf38\") " pod="openshift-ingress/payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7"
Jul 24 17:58:59.049893 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:59.049770    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14f782c1-2807-4da5-8d5f-fdff99fadf38-plugins-config-volume\") pod \"payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7\" (UID: \"14f782c1-2807-4da5-8d5f-fdff99fadf38\") " pod="openshift-ingress/payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7"
Jul 24 17:58:59.068959 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:59.068937    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlxxv\" (UniqueName: \"kubernetes.io/projected/14f782c1-2807-4da5-8d5f-fdff99fadf38-kube-api-access-zlxxv\") pod \"payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7\" (UID: \"14f782c1-2807-4da5-8d5f-fdff99fadf38\") " pod="openshift-ingress/payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7"
Jul 24 17:58:59.110199 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:59.110032    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7"
Jul 24 17:58:59.218298 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:59.218257    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc"]
Jul 24 17:58:59.274188 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:59.274123    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7"]
Jul 24 17:58:59.276104 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:58:59.276077    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14f782c1_2807_4da5_8d5f_fdff99fadf38.slice/crio-92a09bec18fea43132da2bbcb66f593a750ee1fd2c51a00afa704c44ea96c2e8 WatchSource:0}: Error finding container 92a09bec18fea43132da2bbcb66f593a750ee1fd2c51a00afa704c44ea96c2e8: Status 404 returned error can't find the container with id 92a09bec18fea43132da2bbcb66f593a750ee1fd2c51a00afa704c44ea96c2e8
Jul 24 17:58:59.654383 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:59.654338    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7" event={"ID":"14f782c1-2807-4da5-8d5f-fdff99fadf38","Type":"ContainerStarted","Data":"32524de0270a5e1e04ea11be952ad688794fe24a534e2c8a3871f8b7663150fc"}
Jul 24 17:58:59.654844 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:59.654417    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7" event={"ID":"14f782c1-2807-4da5-8d5f-fdff99fadf38","Type":"ContainerStarted","Data":"92a09bec18fea43132da2bbcb66f593a750ee1fd2c51a00afa704c44ea96c2e8"}
Jul 24 17:58:59.654844 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:59.654447    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7"
Jul 24 17:58:59.655976 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:59.655951    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc" event={"ID":"8991b189-5bed-4acc-8867-f77b660108ae","Type":"ContainerStarted","Data":"13b57944936c77c808be4bcb074e98b244c72ffdef10c6c1b754872e9438bebe"}
Jul 24 17:58:59.656105 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:59.655995    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc" event={"ID":"8991b189-5bed-4acc-8867-f77b660108ae","Type":"ContainerStarted","Data":"0bc124d5d9f6ec8f07b03a7348b0cea67d06ac49b70201315361511693007187"}
Jul 24 17:58:59.656105 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:59.656057    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc"
Jul 24 17:58:59.657345 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:59.657322    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv" event={"ID":"3d85b59a-2c8e-43a5-8189-f1a773b186b0","Type":"ContainerStarted","Data":"1f89c2f503b5860241766a82f19bd1a5bfe10b8340097a93ee26a3f98807561a"}
Jul 24 17:58:59.657461 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:59.657445    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv"
Jul 24 17:58:59.688104 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:59.688057    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7" podStartSLOduration=1.68804283 podStartE2EDuration="1.68804283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:58:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:58:59.687102853 +0000 UTC m=+2502.414259068" watchObservedRunningTime="2026-07-24 17:58:59.68804283 +0000 UTC m=+2502.415199043"
Jul 24 17:58:59.713777 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:59.713730    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv" podStartSLOduration=2.330120891 podStartE2EDuration="2.713718102s" totalImagesPullingTime="383.597211ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:58:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:58:59.713109926 +0000 UTC m=+2502.440266142" watchObservedRunningTime="2026-07-24 17:58:59.713718102 +0000 UTC m=+2502.440874315"
Jul 24 17:58:59.745965 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:58:59.745910    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc" podStartSLOduration=1.745892123 podStartE2EDuration="1.745892123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:58:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:58:59.743872575 +0000 UTC m=+2502.471028790" watchObservedRunningTime="2026-07-24 17:58:59.745892123 +0000 UTC m=+2502.473048338"
Jul 24 17:59:05.667819 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:05.667789    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv"
Jul 24 17:59:06.582723 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:06.582687    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85"]
Jul 24 17:59:06.586701 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:06.586674    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85"
Jul 24 17:59:06.589338 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:06.589312    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-0ed7b2-serving-cert\""
Jul 24 17:59:06.602584 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:06.602559    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85"]
Jul 24 17:59:06.724394 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:06.724358    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9e8acc1-f0aa-4f66-a426-45178ba677c1-openshift-service-ca\") pod \"maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85\" (UID: \"b9e8acc1-f0aa-4f66-a426-45178ba677c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85"
Jul 24 17:59:06.724774 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:06.724461    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9e8acc1-f0aa-4f66-a426-45178ba677c1-maas-api-tls\") pod \"maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85\" (UID: \"b9e8acc1-f0aa-4f66-a426-45178ba677c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85"
Jul 24 17:59:06.724774 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:06.724512    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmqqk\" (UniqueName: \"kubernetes.io/projected/b9e8acc1-f0aa-4f66-a426-45178ba677c1-kube-api-access-wmqqk\") pod \"maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85\" (UID: \"b9e8acc1-f0aa-4f66-a426-45178ba677c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85"
Jul 24 17:59:06.825596 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:06.825563    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmqqk\" (UniqueName: \"kubernetes.io/projected/b9e8acc1-f0aa-4f66-a426-45178ba677c1-kube-api-access-wmqqk\") pod \"maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85\" (UID: \"b9e8acc1-f0aa-4f66-a426-45178ba677c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85"
Jul 24 17:59:06.825768 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:06.825607    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9e8acc1-f0aa-4f66-a426-45178ba677c1-openshift-service-ca\") pod \"maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85\" (UID: \"b9e8acc1-f0aa-4f66-a426-45178ba677c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85"
Jul 24 17:59:06.825768 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:06.825669    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9e8acc1-f0aa-4f66-a426-45178ba677c1-maas-api-tls\") pod \"maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85\" (UID: \"b9e8acc1-f0aa-4f66-a426-45178ba677c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85"
Jul 24 17:59:06.825768 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:59:06.825760    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-serving-cert: secret "maas-api-e2e-api-b-0ed7b2-serving-cert" not found
Jul 24 17:59:06.825913 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:59:06.825818    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9e8acc1-f0aa-4f66-a426-45178ba677c1-maas-api-tls podName:b9e8acc1-f0aa-4f66-a426-45178ba677c1 nodeName:}" failed. No retries permitted until 2026-07-24 17:59:07.325802696 +0000 UTC m=+2510.052958891 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b9e8acc1-f0aa-4f66-a426-45178ba677c1-maas-api-tls") pod "maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85" (UID: "b9e8acc1-f0aa-4f66-a426-45178ba677c1") : secret "maas-api-e2e-api-b-0ed7b2-serving-cert" not found
Jul 24 17:59:06.826337 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:06.826319    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9e8acc1-f0aa-4f66-a426-45178ba677c1-openshift-service-ca\") pod \"maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85\" (UID: \"b9e8acc1-f0aa-4f66-a426-45178ba677c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85"
Jul 24 17:59:06.870842 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:06.870816    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmqqk\" (UniqueName: \"kubernetes.io/projected/b9e8acc1-f0aa-4f66-a426-45178ba677c1-kube-api-access-wmqqk\") pod \"maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85\" (UID: \"b9e8acc1-f0aa-4f66-a426-45178ba677c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85"
Jul 24 17:59:07.330911 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:07.330820    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9e8acc1-f0aa-4f66-a426-45178ba677c1-maas-api-tls\") pod \"maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85\" (UID: \"b9e8acc1-f0aa-4f66-a426-45178ba677c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85"
Jul 24 17:59:07.333307 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:07.333274    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9e8acc1-f0aa-4f66-a426-45178ba677c1-maas-api-tls\") pod \"maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85\" (UID: \"b9e8acc1-f0aa-4f66-a426-45178ba677c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85"
Jul 24 17:59:07.499329 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:07.499300    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85"
Jul 24 17:59:07.663550 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:07.663503    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85"]
Jul 24 17:59:07.664847 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:59:07.664796    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9e8acc1_f0aa_4f66_a426_45178ba677c1.slice/crio-2a7868cf51ecd10f0488d7553db953eb6aff512045df68862b3cc597c5db8229 WatchSource:0}: Error finding container 2a7868cf51ecd10f0488d7553db953eb6aff512045df68862b3cc597c5db8229: Status 404 returned error can't find the container with id 2a7868cf51ecd10f0488d7553db953eb6aff512045df68862b3cc597c5db8229
Jul 24 17:59:07.696675 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:07.696641    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85" event={"ID":"b9e8acc1-f0aa-4f66-a426-45178ba677c1","Type":"ContainerStarted","Data":"2a7868cf51ecd10f0488d7553db953eb6aff512045df68862b3cc597c5db8229"}
Jul 24 17:59:07.870071 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:07.869986    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb"]
Jul 24 17:59:07.874763 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:07.874726    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb"
Jul 24 17:59:07.881928 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:07.881903    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-0ed7b2\""
Jul 24 17:59:07.882070 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:07.881992    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-0ed7b2-dockercfg-f54zf\""
Jul 24 17:59:07.894803 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:07.894779    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2"]
Jul 24 17:59:07.898391 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:07.898373    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2"
Jul 24 17:59:07.906901 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:07.906870    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb"]
Jul 24 17:59:07.925068 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:07.925038    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2"]
Jul 24 17:59:08.037195 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:08.037133    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jth8\" (UniqueName: \"kubernetes.io/projected/d731a8f3-0a23-486b-9b0d-ed70024885af-kube-api-access-7jth8\") pod \"payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2\" (UID: \"d731a8f3-0a23-486b-9b0d-ed70024885af\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2"
Jul 24 17:59:08.037354 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:08.037271    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwvws\" (UniqueName: \"kubernetes.io/projected/1aadb605-8774-4436-9ff1-492b7b419648-kube-api-access-lwvws\") pod \"payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb\" (UID: \"1aadb605-8774-4436-9ff1-492b7b419648\") " pod="openshift-ingress/payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb"
Jul 24 17:59:08.037354 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:08.037309    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d731a8f3-0a23-486b-9b0d-ed70024885af-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2\" (UID: \"d731a8f3-0a23-486b-9b0d-ed70024885af\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2"
Jul 24 17:59:08.037354 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:08.037338    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1aadb605-8774-4436-9ff1-492b7b419648-plugins-config-volume\") pod \"payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb\" (UID: \"1aadb605-8774-4436-9ff1-492b7b419648\") " pod="openshift-ingress/payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb"
Jul 24 17:59:08.138572 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:08.138477    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwvws\" (UniqueName: \"kubernetes.io/projected/1aadb605-8774-4436-9ff1-492b7b419648-kube-api-access-lwvws\") pod \"payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb\" (UID: \"1aadb605-8774-4436-9ff1-492b7b419648\") " pod="openshift-ingress/payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb"
Jul 24 17:59:08.138572 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:08.138521    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d731a8f3-0a23-486b-9b0d-ed70024885af-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2\" (UID: \"d731a8f3-0a23-486b-9b0d-ed70024885af\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2"
Jul 24 17:59:08.138572 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:08.138547    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1aadb605-8774-4436-9ff1-492b7b419648-plugins-config-volume\") pod \"payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb\" (UID: \"1aadb605-8774-4436-9ff1-492b7b419648\") " pod="openshift-ingress/payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb"
Jul 24 17:59:08.138847 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:08.138629    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jth8\" (UniqueName: \"kubernetes.io/projected/d731a8f3-0a23-486b-9b0d-ed70024885af-kube-api-access-7jth8\") pod \"payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2\" (UID: \"d731a8f3-0a23-486b-9b0d-ed70024885af\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2"
Jul 24 17:59:08.139401 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:08.139374    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d731a8f3-0a23-486b-9b0d-ed70024885af-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2\" (UID: \"d731a8f3-0a23-486b-9b0d-ed70024885af\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2"
Jul 24 17:59:08.139514 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:08.139457    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1aadb605-8774-4436-9ff1-492b7b419648-plugins-config-volume\") pod \"payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb\" (UID: \"1aadb605-8774-4436-9ff1-492b7b419648\") " pod="openshift-ingress/payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb"
Jul 24 17:59:08.153004 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:08.152979    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwvws\" (UniqueName: \"kubernetes.io/projected/1aadb605-8774-4436-9ff1-492b7b419648-kube-api-access-lwvws\") pod \"payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb\" (UID: \"1aadb605-8774-4436-9ff1-492b7b419648\") " pod="openshift-ingress/payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb"
Jul 24 17:59:08.154727 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:08.154702    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jth8\" (UniqueName: \"kubernetes.io/projected/d731a8f3-0a23-486b-9b0d-ed70024885af-kube-api-access-7jth8\") pod \"payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2\" (UID: \"d731a8f3-0a23-486b-9b0d-ed70024885af\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2"
Jul 24 17:59:08.186516 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:08.186482    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb"
Jul 24 17:59:08.211328 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:08.211292    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2"
Jul 24 17:59:08.417502 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:59:08.417475    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd731a8f3_0a23_486b_9b0d_ed70024885af.slice/crio-be7d7593e434ea97e931ba5eabf9974cdec423a890c5c1d00453f59e61f8f2d9 WatchSource:0}: Error finding container be7d7593e434ea97e931ba5eabf9974cdec423a890c5c1d00453f59e61f8f2d9: Status 404 returned error can't find the container with id be7d7593e434ea97e931ba5eabf9974cdec423a890c5c1d00453f59e61f8f2d9
Jul 24 17:59:08.419215 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:08.419194    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2"]
Jul 24 17:59:08.574959 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:08.574926    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb"]
Jul 24 17:59:08.578361 ip-10-0-131-159 kubenswrapper[2562]: W0724 17:59:08.578333    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aadb605_8774_4436_9ff1_492b7b419648.slice/crio-1b5ba5469208a45b51a42c5e64e9e942970681f2ad5405f52957c2c9f3a4835d WatchSource:0}: Error finding container 1b5ba5469208a45b51a42c5e64e9e942970681f2ad5405f52957c2c9f3a4835d: Status 404 returned error can't find the container with id 1b5ba5469208a45b51a42c5e64e9e942970681f2ad5405f52957c2c9f3a4835d
Jul 24 17:59:08.703438 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:08.703331    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb" event={"ID":"1aadb605-8774-4436-9ff1-492b7b419648","Type":"ContainerStarted","Data":"ef6af305a7ad54dee4397590a62a0c7f80e6f0fe5562a1f152fce256df729eeb"}
Jul 24 17:59:08.703438 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:08.703389    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb" event={"ID":"1aadb605-8774-4436-9ff1-492b7b419648","Type":"ContainerStarted","Data":"1b5ba5469208a45b51a42c5e64e9e942970681f2ad5405f52957c2c9f3a4835d"}
Jul 24 17:59:08.703438 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:08.703423    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb"
Jul 24 17:59:08.705170 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:08.705114    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85" event={"ID":"b9e8acc1-f0aa-4f66-a426-45178ba677c1","Type":"ContainerStarted","Data":"2206136d33579149f579e7220e576140113134452b39a64b6f125b98efd370db"}
Jul 24 17:59:08.705296 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:08.705198    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85"
Jul 24 17:59:08.707195 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:08.707074    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2" event={"ID":"d731a8f3-0a23-486b-9b0d-ed70024885af","Type":"ContainerStarted","Data":"9febf4bf8fb74cf79374b8b860836185139006e0a12def5e3e9e3a0178b1485b"}
Jul 24 17:59:08.707195 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:08.707127    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2" event={"ID":"d731a8f3-0a23-486b-9b0d-ed70024885af","Type":"ContainerStarted","Data":"be7d7593e434ea97e931ba5eabf9974cdec423a890c5c1d00453f59e61f8f2d9"}
Jul 24 17:59:08.707372 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:08.707193    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2"
Jul 24 17:59:08.819380 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:08.819326    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb" podStartSLOduration=1.819311211 podStartE2EDuration="1.819311211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:59:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:59:08.76412227 +0000 UTC m=+2511.491278497" watchObservedRunningTime="2026-07-24 17:59:08.819311211 +0000 UTC m=+2511.546467475"
Jul 24 17:59:08.819640 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:08.819620    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2" podStartSLOduration=1.819614946 podStartE2EDuration="1.819614946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:59:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:59:08.815368893 +0000 UTC m=+2511.542525119" watchObservedRunningTime="2026-07-24 17:59:08.819614946 +0000 UTC m=+2511.546771161"
Jul 24 17:59:08.873106 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:08.873050    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85" podStartSLOduration=2.497856694 podStartE2EDuration="2.873034105s" totalImagesPullingTime="375.177411ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:59:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:59:08.871320945 +0000 UTC m=+2511.598477161" watchObservedRunningTime="2026-07-24 17:59:08.873034105 +0000 UTC m=+2511.600190319"
Jul 24 17:59:10.664769 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:10.664690    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc"
Jul 24 17:59:10.664769 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:10.664768    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7"
Jul 24 17:59:14.721060 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:14.721030    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85"
Jul 24 17:59:19.716321 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:19.716288    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb"
Jul 24 17:59:19.716789 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:19.716364    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2"
Jul 24 17:59:34.467156 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.467103    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv"]
Jul 24 17:59:34.467662 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.467368    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv" podUID="3d85b59a-2c8e-43a5-8189-f1a773b186b0" containerName="maas-api" containerID="cri-o://1f89c2f503b5860241766a82f19bd1a5bfe10b8340097a93ee26a3f98807561a" gracePeriod=30
Jul 24 17:59:34.535703 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.535666    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7"]
Jul 24 17:59:34.536067 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.536028    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7" podUID="14f782c1-2807-4da5-8d5f-fdff99fadf38" containerName="payload-processing" containerID="cri-o://32524de0270a5e1e04ea11be952ad688794fe24a534e2c8a3871f8b7663150fc" gracePeriod=30
Jul 24 17:59:34.559070 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.557745    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc"]
Jul 24 17:59:34.559070 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.557986    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc" podUID="8991b189-5bed-4acc-8867-f77b660108ae" containerName="payload-pre-processing" containerID="cri-o://13b57944936c77c808be4bcb074e98b244c72ffdef10c6c1b754872e9438bebe" gracePeriod=30
Jul 24 17:59:34.795857 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.795826    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv"
Jul 24 17:59:34.851463 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.851434    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7"
Jul 24 17:59:34.852201 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.852177    2562 generic.go:356] "Generic (PLEG): container finished" podID="14f782c1-2807-4da5-8d5f-fdff99fadf38" containerID="32524de0270a5e1e04ea11be952ad688794fe24a534e2c8a3871f8b7663150fc" exitCode=0
Jul 24 17:59:34.852329 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.852259    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7" event={"ID":"14f782c1-2807-4da5-8d5f-fdff99fadf38","Type":"ContainerDied","Data":"32524de0270a5e1e04ea11be952ad688794fe24a534e2c8a3871f8b7663150fc"}
Jul 24 17:59:34.852386 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.852359    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7" event={"ID":"14f782c1-2807-4da5-8d5f-fdff99fadf38","Type":"ContainerDied","Data":"92a09bec18fea43132da2bbcb66f593a750ee1fd2c51a00afa704c44ea96c2e8"}
Jul 24 17:59:34.852445 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.852393    2562 scope.go:117] "RemoveContainer" containerID="32524de0270a5e1e04ea11be952ad688794fe24a534e2c8a3871f8b7663150fc"
Jul 24 17:59:34.854032 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.854007    2562 generic.go:356] "Generic (PLEG): container finished" podID="8991b189-5bed-4acc-8867-f77b660108ae" containerID="13b57944936c77c808be4bcb074e98b244c72ffdef10c6c1b754872e9438bebe" exitCode=0
Jul 24 17:59:34.854113 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.854080    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc" event={"ID":"8991b189-5bed-4acc-8867-f77b660108ae","Type":"ContainerDied","Data":"13b57944936c77c808be4bcb074e98b244c72ffdef10c6c1b754872e9438bebe"}
Jul 24 17:59:34.855538 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.855514    2562 generic.go:356] "Generic (PLEG): container finished" podID="3d85b59a-2c8e-43a5-8189-f1a773b186b0" containerID="1f89c2f503b5860241766a82f19bd1a5bfe10b8340097a93ee26a3f98807561a" exitCode=0
Jul 24 17:59:34.855646 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.855557    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv" event={"ID":"3d85b59a-2c8e-43a5-8189-f1a773b186b0","Type":"ContainerDied","Data":"1f89c2f503b5860241766a82f19bd1a5bfe10b8340097a93ee26a3f98807561a"}
Jul 24 17:59:34.855646 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.855574    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv"
Jul 24 17:59:34.855646 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.855610    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv" event={"ID":"3d85b59a-2c8e-43a5-8189-f1a773b186b0","Type":"ContainerDied","Data":"b86bb7f9ae5dd4bfd3e4e731002a65c81abdab2f12aa40927bf58409294b6e08"}
Jul 24 17:59:34.863396 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.863378    2562 scope.go:117] "RemoveContainer" containerID="32524de0270a5e1e04ea11be952ad688794fe24a534e2c8a3871f8b7663150fc"
Jul 24 17:59:34.863646 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:59:34.863621    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32524de0270a5e1e04ea11be952ad688794fe24a534e2c8a3871f8b7663150fc\": container with ID starting with 32524de0270a5e1e04ea11be952ad688794fe24a534e2c8a3871f8b7663150fc not found: ID does not exist" containerID="32524de0270a5e1e04ea11be952ad688794fe24a534e2c8a3871f8b7663150fc"
Jul 24 17:59:34.863722 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.863653    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32524de0270a5e1e04ea11be952ad688794fe24a534e2c8a3871f8b7663150fc"} err="failed to get container status \"32524de0270a5e1e04ea11be952ad688794fe24a534e2c8a3871f8b7663150fc\": rpc error: code = NotFound desc = could not find container \"32524de0270a5e1e04ea11be952ad688794fe24a534e2c8a3871f8b7663150fc\": container with ID starting with 32524de0270a5e1e04ea11be952ad688794fe24a534e2c8a3871f8b7663150fc not found: ID does not exist"
Jul 24 17:59:34.863722 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.863671    2562 scope.go:117] "RemoveContainer" containerID="1f89c2f503b5860241766a82f19bd1a5bfe10b8340097a93ee26a3f98807561a"
Jul 24 17:59:34.864731 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.864713    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc"
Jul 24 17:59:34.873355 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.873336    2562 scope.go:117] "RemoveContainer" containerID="1f89c2f503b5860241766a82f19bd1a5bfe10b8340097a93ee26a3f98807561a"
Jul 24 17:59:34.873703 ip-10-0-131-159 kubenswrapper[2562]: E0724 17:59:34.873678    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f89c2f503b5860241766a82f19bd1a5bfe10b8340097a93ee26a3f98807561a\": container with ID starting with 1f89c2f503b5860241766a82f19bd1a5bfe10b8340097a93ee26a3f98807561a not found: ID does not exist" containerID="1f89c2f503b5860241766a82f19bd1a5bfe10b8340097a93ee26a3f98807561a"
Jul 24 17:59:34.873797 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.873723    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f89c2f503b5860241766a82f19bd1a5bfe10b8340097a93ee26a3f98807561a"} err="failed to get container status \"1f89c2f503b5860241766a82f19bd1a5bfe10b8340097a93ee26a3f98807561a\": rpc error: code = NotFound desc = could not find container \"1f89c2f503b5860241766a82f19bd1a5bfe10b8340097a93ee26a3f98807561a\": container with ID starting with 1f89c2f503b5860241766a82f19bd1a5bfe10b8340097a93ee26a3f98807561a not found: ID does not exist"
Jul 24 17:59:34.891379 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.891355    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d85b59a-2c8e-43a5-8189-f1a773b186b0-maas-api-tls\") pod \"3d85b59a-2c8e-43a5-8189-f1a773b186b0\" (UID: \"3d85b59a-2c8e-43a5-8189-f1a773b186b0\") "
Jul 24 17:59:34.891507 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.891398    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hsv8w\" (UniqueName: \"kubernetes.io/projected/3d85b59a-2c8e-43a5-8189-f1a773b186b0-kube-api-access-hsv8w\") pod \"3d85b59a-2c8e-43a5-8189-f1a773b186b0\" (UID: \"3d85b59a-2c8e-43a5-8189-f1a773b186b0\") "
Jul 24 17:59:34.891507 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.891440    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d85b59a-2c8e-43a5-8189-f1a773b186b0-openshift-service-ca\") pod \"3d85b59a-2c8e-43a5-8189-f1a773b186b0\" (UID: \"3d85b59a-2c8e-43a5-8189-f1a773b186b0\") "
Jul 24 17:59:34.892117 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.892088    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d85b59a-2c8e-43a5-8189-f1a773b186b0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3d85b59a-2c8e-43a5-8189-f1a773b186b0" (UID: "3d85b59a-2c8e-43a5-8189-f1a773b186b0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:59:34.893608 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.893580    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d85b59a-2c8e-43a5-8189-f1a773b186b0-kube-api-access-hsv8w" (OuterVolumeSpecName: "kube-api-access-hsv8w") pod "3d85b59a-2c8e-43a5-8189-f1a773b186b0" (UID: "3d85b59a-2c8e-43a5-8189-f1a773b186b0"). InnerVolumeSpecName "kube-api-access-hsv8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:59:34.893718 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.893586    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d85b59a-2c8e-43a5-8189-f1a773b186b0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3d85b59a-2c8e-43a5-8189-f1a773b186b0" (UID: "3d85b59a-2c8e-43a5-8189-f1a773b186b0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:59:34.992624 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.992587    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8991b189-5bed-4acc-8867-f77b660108ae-plugins-config-volume\") pod \"8991b189-5bed-4acc-8867-f77b660108ae\" (UID: \"8991b189-5bed-4acc-8867-f77b660108ae\") "
Jul 24 17:59:34.992624 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.992626    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zlxxv\" (UniqueName: \"kubernetes.io/projected/14f782c1-2807-4da5-8d5f-fdff99fadf38-kube-api-access-zlxxv\") pod \"14f782c1-2807-4da5-8d5f-fdff99fadf38\" (UID: \"14f782c1-2807-4da5-8d5f-fdff99fadf38\") "
Jul 24 17:59:34.992914 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.992674    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hspx6\" (UniqueName: \"kubernetes.io/projected/8991b189-5bed-4acc-8867-f77b660108ae-kube-api-access-hspx6\") pod \"8991b189-5bed-4acc-8867-f77b660108ae\" (UID: \"8991b189-5bed-4acc-8867-f77b660108ae\") "
Jul 24 17:59:34.992914 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.992691    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14f782c1-2807-4da5-8d5f-fdff99fadf38-plugins-config-volume\") pod \"14f782c1-2807-4da5-8d5f-fdff99fadf38\" (UID: \"14f782c1-2807-4da5-8d5f-fdff99fadf38\") "
Jul 24 17:59:34.993032 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.992925    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d85b59a-2c8e-43a5-8189-f1a773b186b0-maas-api-tls\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:59:34.993032 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.992937    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsv8w\" (UniqueName: \"kubernetes.io/projected/3d85b59a-2c8e-43a5-8189-f1a773b186b0-kube-api-access-hsv8w\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:59:34.993032 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.992946    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d85b59a-2c8e-43a5-8189-f1a773b186b0-openshift-service-ca\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:59:34.993242 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.993066    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8991b189-5bed-4acc-8867-f77b660108ae-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8991b189-5bed-4acc-8867-f77b660108ae" (UID: "8991b189-5bed-4acc-8867-f77b660108ae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:59:34.993242 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.993204    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14f782c1-2807-4da5-8d5f-fdff99fadf38-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "14f782c1-2807-4da5-8d5f-fdff99fadf38" (UID: "14f782c1-2807-4da5-8d5f-fdff99fadf38"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:59:34.994794 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.994771    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14f782c1-2807-4da5-8d5f-fdff99fadf38-kube-api-access-zlxxv" (OuterVolumeSpecName: "kube-api-access-zlxxv") pod "14f782c1-2807-4da5-8d5f-fdff99fadf38" (UID: "14f782c1-2807-4da5-8d5f-fdff99fadf38"). InnerVolumeSpecName "kube-api-access-zlxxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:59:34.994794 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:34.994786    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8991b189-5bed-4acc-8867-f77b660108ae-kube-api-access-hspx6" (OuterVolumeSpecName: "kube-api-access-hspx6") pod "8991b189-5bed-4acc-8867-f77b660108ae" (UID: "8991b189-5bed-4acc-8867-f77b660108ae"). InnerVolumeSpecName "kube-api-access-hspx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:59:35.093801 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:35.093765    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8991b189-5bed-4acc-8867-f77b660108ae-plugins-config-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:59:35.093801 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:35.093801    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlxxv\" (UniqueName: \"kubernetes.io/projected/14f782c1-2807-4da5-8d5f-fdff99fadf38-kube-api-access-zlxxv\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:59:35.093994 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:35.093815    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hspx6\" (UniqueName: \"kubernetes.io/projected/8991b189-5bed-4acc-8867-f77b660108ae-kube-api-access-hspx6\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:59:35.093994 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:35.093824    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14f782c1-2807-4da5-8d5f-fdff99fadf38-plugins-config-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 17:59:35.178297 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:35.178262    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv"]
Jul 24 17:59:35.184051 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:35.184025    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5e940d-76884bf7f5-gwbmv"]
Jul 24 17:59:35.825132 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:35.825055    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d85b59a-2c8e-43a5-8189-f1a773b186b0" path="/var/lib/kubelet/pods/3d85b59a-2c8e-43a5-8189-f1a773b186b0/volumes"
Jul 24 17:59:35.861515 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:35.861494    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc"
Jul 24 17:59:35.861515 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:35.861505    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc" event={"ID":"8991b189-5bed-4acc-8867-f77b660108ae","Type":"ContainerDied","Data":"0bc124d5d9f6ec8f07b03a7348b0cea67d06ac49b70201315361511693007187"}
Jul 24 17:59:35.861710 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:35.861561    2562 scope.go:117] "RemoveContainer" containerID="13b57944936c77c808be4bcb074e98b244c72ffdef10c6c1b754872e9438bebe"
Jul 24 17:59:35.863853 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:35.863830    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7"
Jul 24 17:59:35.881758 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:35.881734    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc"]
Jul 24 17:59:35.885368 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:35.885348    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5e940d-85cddcfd76-7qgnc"]
Jul 24 17:59:35.897047 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:35.897028    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7"]
Jul 24 17:59:35.901899 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:35.901877    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5e940d-5b967db8db-g68h7"]
Jul 24 17:59:37.824713 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:37.824680    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14f782c1-2807-4da5-8d5f-fdff99fadf38" path="/var/lib/kubelet/pods/14f782c1-2807-4da5-8d5f-fdff99fadf38/volumes"
Jul 24 17:59:37.825094 ip-10-0-131-159 kubenswrapper[2562]: I0724 17:59:37.825007    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8991b189-5bed-4acc-8867-f77b660108ae" path="/var/lib/kubelet/pods/8991b189-5bed-4acc-8867-f77b660108ae/volumes"
Jul 24 18:00:02.921688 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:02.921651    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85"]
Jul 24 18:00:02.922253 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:02.921914    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85" podUID="b9e8acc1-f0aa-4f66-a426-45178ba677c1" containerName="maas-api" containerID="cri-o://2206136d33579149f579e7220e576140113134452b39a64b6f125b98efd370db" gracePeriod=30
Jul 24 18:00:02.997993 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:02.997955    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb"]
Jul 24 18:00:02.998327 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:02.998300    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb" podUID="1aadb605-8774-4436-9ff1-492b7b419648" containerName="payload-processing" containerID="cri-o://ef6af305a7ad54dee4397590a62a0c7f80e6f0fe5562a1f152fce256df729eeb" gracePeriod=30
Jul 24 18:00:03.008733 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:03.003941    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2"]
Jul 24 18:00:03.008733 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:03.004154    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2" podUID="d731a8f3-0a23-486b-9b0d-ed70024885af" containerName="payload-pre-processing" containerID="cri-o://9febf4bf8fb74cf79374b8b860836185139006e0a12def5e3e9e3a0178b1485b" gracePeriod=30
Jul 24 18:00:03.254417 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:03.254321    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85"
Jul 24 18:00:03.281216 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:03.281187    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb"
Jul 24 18:00:03.317492 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:03.317469    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2"
Jul 24 18:00:03.351458 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:03.351433    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1aadb605-8774-4436-9ff1-492b7b419648-plugins-config-volume\") pod \"1aadb605-8774-4436-9ff1-492b7b419648\" (UID: \"1aadb605-8774-4436-9ff1-492b7b419648\") "
Jul 24 18:00:03.351588 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:03.351531    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwvws\" (UniqueName: \"kubernetes.io/projected/1aadb605-8774-4436-9ff1-492b7b419648-kube-api-access-lwvws\") pod \"1aadb605-8774-4436-9ff1-492b7b419648\" (UID: \"1aadb605-8774-4436-9ff1-492b7b419648\") "
Jul 24 18:00:03.351588 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:03.351555    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9e8acc1-f0aa-4f66-a426-45178ba677c1-maas-api-tls\") pod \"b9e8acc1-f0aa-4f66-a426-45178ba677c1\" (UID: \"b9e8acc1-f0aa-4f66-a426-45178ba677c1\") "
Jul 24 18:00:03.351588 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:03.351571    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d731a8f3-0a23-486b-9b0d-ed70024885af-plugins-config-volume\") pod \"d731a8f3-0a23-486b-9b0d-ed70024885af\" (UID: \"d731a8f3-0a23-486b-9b0d-ed70024885af\") "
Jul 24 18:00:03.351718 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:03.351621    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jth8\" (UniqueName: \"kubernetes.io/projected/d731a8f3-0a23-486b-9b0d-ed70024885af-kube-api-access-7jth8\") pod \"d731a8f3-0a23-486b-9b0d-ed70024885af\" (UID: \"d731a8f3-0a23-486b-9b0d-ed70024885af\") "
Jul 24 18:00:03.352002 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:03.351980    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9e8acc1-f0aa-4f66-a426-45178ba677c1-openshift-service-ca\") pod \"b9e8acc1-f0aa-4f66-a426-45178ba677c1\" (UID: \"b9e8acc1-f0aa-4f66-a426-45178ba677c1\") "
Jul 24 18:00:03.352002 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:03.352005    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmqqk\" (UniqueName: \"kubernetes.io/projected/b9e8acc1-f0aa-4f66-a426-45178ba677c1-kube-api-access-wmqqk\") pod \"b9e8acc1-f0aa-4f66-a426-45178ba677c1\" (UID: \"b9e8acc1-f0aa-4f66-a426-45178ba677c1\") "
Jul 24 18:00:03.352271 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:03.351931    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1aadb605-8774-4436-9ff1-492b7b419648-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1aadb605-8774-4436-9ff1-492b7b419648" (UID: "1aadb605-8774-4436-9ff1-492b7b419648"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:00:03.352271 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:03.352003    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d731a8f3-0a23-486b-9b0d-ed70024885af-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d731a8f3-0a23-486b-9b0d-ed70024885af" (UID: "d731a8f3-0a23-486b-9b0d-ed70024885af"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:00:03.352395 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:03.352328    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d731a8f3-0a23-486b-9b0d-ed70024885af-plugins-config-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:00:03.352395 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:03.352349    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1aadb605-8774-4436-9ff1-492b7b419648-plugins-config-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:00:03.352395 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:03.352382    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9e8acc1-f0aa-4f66-a426-45178ba677c1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b9e8acc1-f0aa-4f66-a426-45178ba677c1" (UID: "b9e8acc1-f0aa-4f66-a426-45178ba677c1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:00:03.354020 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:03.353998    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9e8acc1-f0aa-4f66-a426-45178ba677c1-kube-api-access-wmqqk" (OuterVolumeSpecName: "kube-api-access-wmqqk") pod "b9e8acc1-f0aa-4f66-a426-45178ba677c1" (UID: "b9e8acc1-f0aa-4f66-a426-45178ba677c1"). InnerVolumeSpecName "kube-api-access-wmqqk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:00:03.354287 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:03.354256    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9e8acc1-f0aa-4f66-a426-45178ba677c1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b9e8acc1-f0aa-4f66-a426-45178ba677c1" (UID: "b9e8acc1-f0aa-4f66-a426-45178ba677c1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 18:00:03.354373 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:03.354283    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d731a8f3-0a23-486b-9b0d-ed70024885af-kube-api-access-7jth8" (OuterVolumeSpecName: "kube-api-access-7jth8") pod "d731a8f3-0a23-486b-9b0d-ed70024885af" (UID: "d731a8f3-0a23-486b-9b0d-ed70024885af"). InnerVolumeSpecName "kube-api-access-7jth8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:00:03.354437 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:03.354406    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aadb605-8774-4436-9ff1-492b7b419648-kube-api-access-lwvws" (OuterVolumeSpecName: "kube-api-access-lwvws") pod "1aadb605-8774-4436-9ff1-492b7b419648" (UID: "1aadb605-8774-4436-9ff1-492b7b419648"). InnerVolumeSpecName "kube-api-access-lwvws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:00:03.453283 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:03.453197    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwvws\" (UniqueName: \"kubernetes.io/projected/1aadb605-8774-4436-9ff1-492b7b419648-kube-api-access-lwvws\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:00:03.453283 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:03.453228    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9e8acc1-f0aa-4f66-a426-45178ba677c1-maas-api-tls\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:00:03.453283 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:03.453238    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jth8\" (UniqueName: \"kubernetes.io/projected/d731a8f3-0a23-486b-9b0d-ed70024885af-kube-api-access-7jth8\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:00:03.453283 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:03.453252    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9e8acc1-f0aa-4f66-a426-45178ba677c1-openshift-service-ca\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:00:03.453283 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:03.453264    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmqqk\" (UniqueName: \"kubernetes.io/projected/b9e8acc1-f0aa-4f66-a426-45178ba677c1-kube-api-access-wmqqk\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:00:04.020708 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:04.020676    2562 generic.go:356] "Generic (PLEG): container finished" podID="1aadb605-8774-4436-9ff1-492b7b419648" containerID="ef6af305a7ad54dee4397590a62a0c7f80e6f0fe5562a1f152fce256df729eeb" exitCode=0
Jul 24 18:00:04.021186 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:04.020756    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb"
Jul 24 18:00:04.021186 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:04.020766    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb" event={"ID":"1aadb605-8774-4436-9ff1-492b7b419648","Type":"ContainerDied","Data":"ef6af305a7ad54dee4397590a62a0c7f80e6f0fe5562a1f152fce256df729eeb"}
Jul 24 18:00:04.021186 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:04.020839    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb" event={"ID":"1aadb605-8774-4436-9ff1-492b7b419648","Type":"ContainerDied","Data":"1b5ba5469208a45b51a42c5e64e9e942970681f2ad5405f52957c2c9f3a4835d"}
Jul 24 18:00:04.021186 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:04.020871    2562 scope.go:117] "RemoveContainer" containerID="ef6af305a7ad54dee4397590a62a0c7f80e6f0fe5562a1f152fce256df729eeb"
Jul 24 18:00:04.022356 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:04.022264    2562 generic.go:356] "Generic (PLEG): container finished" podID="b9e8acc1-f0aa-4f66-a426-45178ba677c1" containerID="2206136d33579149f579e7220e576140113134452b39a64b6f125b98efd370db" exitCode=0
Jul 24 18:00:04.022356 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:04.022325    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85"
Jul 24 18:00:04.022356 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:04.022349    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85" event={"ID":"b9e8acc1-f0aa-4f66-a426-45178ba677c1","Type":"ContainerDied","Data":"2206136d33579149f579e7220e576140113134452b39a64b6f125b98efd370db"}
Jul 24 18:00:04.022599 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:04.022392    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85" event={"ID":"b9e8acc1-f0aa-4f66-a426-45178ba677c1","Type":"ContainerDied","Data":"2a7868cf51ecd10f0488d7553db953eb6aff512045df68862b3cc597c5db8229"}
Jul 24 18:00:04.023740 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:04.023719    2562 generic.go:356] "Generic (PLEG): container finished" podID="d731a8f3-0a23-486b-9b0d-ed70024885af" containerID="9febf4bf8fb74cf79374b8b860836185139006e0a12def5e3e9e3a0178b1485b" exitCode=0
Jul 24 18:00:04.023871 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:04.023750    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2" event={"ID":"d731a8f3-0a23-486b-9b0d-ed70024885af","Type":"ContainerDied","Data":"9febf4bf8fb74cf79374b8b860836185139006e0a12def5e3e9e3a0178b1485b"}
Jul 24 18:00:04.023871 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:04.023795    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2" event={"ID":"d731a8f3-0a23-486b-9b0d-ed70024885af","Type":"ContainerDied","Data":"be7d7593e434ea97e931ba5eabf9974cdec423a890c5c1d00453f59e61f8f2d9"}
Jul 24 18:00:04.023871 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:04.023757    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2"
Jul 24 18:00:04.030939 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:04.030922    2562 scope.go:117] "RemoveContainer" containerID="ef6af305a7ad54dee4397590a62a0c7f80e6f0fe5562a1f152fce256df729eeb"
Jul 24 18:00:04.031184 ip-10-0-131-159 kubenswrapper[2562]: E0724 18:00:04.031168    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef6af305a7ad54dee4397590a62a0c7f80e6f0fe5562a1f152fce256df729eeb\": container with ID starting with ef6af305a7ad54dee4397590a62a0c7f80e6f0fe5562a1f152fce256df729eeb not found: ID does not exist" containerID="ef6af305a7ad54dee4397590a62a0c7f80e6f0fe5562a1f152fce256df729eeb"
Jul 24 18:00:04.031242 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:04.031191    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef6af305a7ad54dee4397590a62a0c7f80e6f0fe5562a1f152fce256df729eeb"} err="failed to get container status \"ef6af305a7ad54dee4397590a62a0c7f80e6f0fe5562a1f152fce256df729eeb\": rpc error: code = NotFound desc = could not find container \"ef6af305a7ad54dee4397590a62a0c7f80e6f0fe5562a1f152fce256df729eeb\": container with ID starting with ef6af305a7ad54dee4397590a62a0c7f80e6f0fe5562a1f152fce256df729eeb not found: ID does not exist"
Jul 24 18:00:04.031242 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:04.031206    2562 scope.go:117] "RemoveContainer" containerID="2206136d33579149f579e7220e576140113134452b39a64b6f125b98efd370db"
Jul 24 18:00:04.041004 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:04.040989    2562 scope.go:117] "RemoveContainer" containerID="2206136d33579149f579e7220e576140113134452b39a64b6f125b98efd370db"
Jul 24 18:00:04.042052 ip-10-0-131-159 kubenswrapper[2562]: E0724 18:00:04.041726    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2206136d33579149f579e7220e576140113134452b39a64b6f125b98efd370db\": container with ID starting with 2206136d33579149f579e7220e576140113134452b39a64b6f125b98efd370db not found: ID does not exist" containerID="2206136d33579149f579e7220e576140113134452b39a64b6f125b98efd370db"
Jul 24 18:00:04.042052 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:04.041802    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2206136d33579149f579e7220e576140113134452b39a64b6f125b98efd370db"} err="failed to get container status \"2206136d33579149f579e7220e576140113134452b39a64b6f125b98efd370db\": rpc error: code = NotFound desc = could not find container \"2206136d33579149f579e7220e576140113134452b39a64b6f125b98efd370db\": container with ID starting with 2206136d33579149f579e7220e576140113134452b39a64b6f125b98efd370db not found: ID does not exist"
Jul 24 18:00:04.042052 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:04.041937    2562 scope.go:117] "RemoveContainer" containerID="9febf4bf8fb74cf79374b8b860836185139006e0a12def5e3e9e3a0178b1485b"
Jul 24 18:00:04.044446 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:04.044423    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85"]
Jul 24 18:00:04.051172 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:04.051153    2562 scope.go:117] "RemoveContainer" containerID="9febf4bf8fb74cf79374b8b860836185139006e0a12def5e3e9e3a0178b1485b"
Jul 24 18:00:04.051405 ip-10-0-131-159 kubenswrapper[2562]: E0724 18:00:04.051385    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9febf4bf8fb74cf79374b8b860836185139006e0a12def5e3e9e3a0178b1485b\": container with ID starting with 9febf4bf8fb74cf79374b8b860836185139006e0a12def5e3e9e3a0178b1485b not found: ID does not exist" containerID="9febf4bf8fb74cf79374b8b860836185139006e0a12def5e3e9e3a0178b1485b"
Jul 24 18:00:04.051521 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:04.051411    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9febf4bf8fb74cf79374b8b860836185139006e0a12def5e3e9e3a0178b1485b"} err="failed to get container status \"9febf4bf8fb74cf79374b8b860836185139006e0a12def5e3e9e3a0178b1485b\": rpc error: code = NotFound desc = could not find container \"9febf4bf8fb74cf79374b8b860836185139006e0a12def5e3e9e3a0178b1485b\": container with ID starting with 9febf4bf8fb74cf79374b8b860836185139006e0a12def5e3e9e3a0178b1485b not found: ID does not exist"
Jul 24 18:00:04.051684 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:04.051665    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0ed7b2-648d9449df-9vv85"]
Jul 24 18:00:04.063755 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:04.063732    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb"]
Jul 24 18:00:04.074511 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:04.074479    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0ed7b2-5f68f9f6c8-zx7cb"]
Jul 24 18:00:04.085426 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:04.085350    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2"]
Jul 24 18:00:04.091931 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:04.091906    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0ed7b2-6d8fdc95fb-ftfn2"]
Jul 24 18:00:05.824828 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:05.824795    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1aadb605-8774-4436-9ff1-492b7b419648" path="/var/lib/kubelet/pods/1aadb605-8774-4436-9ff1-492b7b419648/volumes"
Jul 24 18:00:05.825326 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:05.825311    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9e8acc1-f0aa-4f66-a426-45178ba677c1" path="/var/lib/kubelet/pods/b9e8acc1-f0aa-4f66-a426-45178ba677c1/volumes"
Jul 24 18:00:05.825688 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:05.825673    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d731a8f3-0a23-486b-9b0d-ed70024885af" path="/var/lib/kubelet/pods/d731a8f3-0a23-486b-9b0d-ed70024885af/volumes"
Jul 24 18:00:16.896504 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:16.896468    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm"]
Jul 24 18:00:16.896987 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:16.896926    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9e8acc1-f0aa-4f66-a426-45178ba677c1" containerName="maas-api"
Jul 24 18:00:16.896987 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:16.896943    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9e8acc1-f0aa-4f66-a426-45178ba677c1" containerName="maas-api"
Jul 24 18:00:16.896987 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:16.896957    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1aadb605-8774-4436-9ff1-492b7b419648" containerName="payload-processing"
Jul 24 18:00:16.896987 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:16.896966    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1aadb605-8774-4436-9ff1-492b7b419648" containerName="payload-processing"
Jul 24 18:00:16.896987 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:16.896980    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d85b59a-2c8e-43a5-8189-f1a773b186b0" containerName="maas-api"
Jul 24 18:00:16.896987 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:16.896989    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d85b59a-2c8e-43a5-8189-f1a773b186b0" containerName="maas-api"
Jul 24 18:00:16.897232 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:16.897010    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14f782c1-2807-4da5-8d5f-fdff99fadf38" containerName="payload-processing"
Jul 24 18:00:16.897232 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:16.897019    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="14f782c1-2807-4da5-8d5f-fdff99fadf38" containerName="payload-processing"
Jul 24 18:00:16.897232 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:16.897029    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d731a8f3-0a23-486b-9b0d-ed70024885af" containerName="payload-pre-processing"
Jul 24 18:00:16.897232 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:16.897035    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d731a8f3-0a23-486b-9b0d-ed70024885af" containerName="payload-pre-processing"
Jul 24 18:00:16.897232 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:16.897045    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8991b189-5bed-4acc-8867-f77b660108ae" containerName="payload-pre-processing"
Jul 24 18:00:16.897232 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:16.897051    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8991b189-5bed-4acc-8867-f77b660108ae" containerName="payload-pre-processing"
Jul 24 18:00:16.897232 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:16.897126    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d85b59a-2c8e-43a5-8189-f1a773b186b0" containerName="maas-api"
Jul 24 18:00:16.897232 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:16.897135    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d731a8f3-0a23-486b-9b0d-ed70024885af" containerName="payload-pre-processing"
Jul 24 18:00:16.897232 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:16.897164    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1aadb605-8774-4436-9ff1-492b7b419648" containerName="payload-processing"
Jul 24 18:00:16.897232 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:16.897171    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="14f782c1-2807-4da5-8d5f-fdff99fadf38" containerName="payload-processing"
Jul 24 18:00:16.897232 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:16.897176    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9e8acc1-f0aa-4f66-a426-45178ba677c1" containerName="maas-api"
Jul 24 18:00:16.897232 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:16.897182    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8991b189-5bed-4acc-8867-f77b660108ae" containerName="payload-pre-processing"
Jul 24 18:00:16.901832 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:16.901811    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm"
Jul 24 18:00:16.904449 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:16.904421    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-257940-serving-cert\""
Jul 24 18:00:16.910955 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:16.910872    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm"]
Jul 24 18:00:16.978005 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:16.977966    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb781198-a4b8-4591-8ac8-0067068e1ee6-openshift-service-ca\") pod \"maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm\" (UID: \"cb781198-a4b8-4591-8ac8-0067068e1ee6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm"
Jul 24 18:00:16.978241 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:16.978034    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb781198-a4b8-4591-8ac8-0067068e1ee6-maas-api-tls\") pod \"maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm\" (UID: \"cb781198-a4b8-4591-8ac8-0067068e1ee6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm"
Jul 24 18:00:16.978241 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:16.978061    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2hk4\" (UniqueName: \"kubernetes.io/projected/cb781198-a4b8-4591-8ac8-0067068e1ee6-kube-api-access-p2hk4\") pod \"maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm\" (UID: \"cb781198-a4b8-4591-8ac8-0067068e1ee6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm"
Jul 24 18:00:17.079442 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:17.079408    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb781198-a4b8-4591-8ac8-0067068e1ee6-openshift-service-ca\") pod \"maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm\" (UID: \"cb781198-a4b8-4591-8ac8-0067068e1ee6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm"
Jul 24 18:00:17.079654 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:17.079472    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb781198-a4b8-4591-8ac8-0067068e1ee6-maas-api-tls\") pod \"maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm\" (UID: \"cb781198-a4b8-4591-8ac8-0067068e1ee6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm"
Jul 24 18:00:17.079654 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:17.079495    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2hk4\" (UniqueName: \"kubernetes.io/projected/cb781198-a4b8-4591-8ac8-0067068e1ee6-kube-api-access-p2hk4\") pod \"maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm\" (UID: \"cb781198-a4b8-4591-8ac8-0067068e1ee6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm"
Jul 24 18:00:17.079654 ip-10-0-131-159 kubenswrapper[2562]: E0724 18:00:17.079577    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-serving-cert: secret "maas-api-e2e-shared-a-257940-serving-cert" not found
Jul 24 18:00:17.079787 ip-10-0-131-159 kubenswrapper[2562]: E0724 18:00:17.079684    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb781198-a4b8-4591-8ac8-0067068e1ee6-maas-api-tls podName:cb781198-a4b8-4591-8ac8-0067068e1ee6 nodeName:}" failed. No retries permitted until 2026-07-24 18:00:17.579646071 +0000 UTC m=+2580.306802276 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cb781198-a4b8-4591-8ac8-0067068e1ee6-maas-api-tls") pod "maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm" (UID: "cb781198-a4b8-4591-8ac8-0067068e1ee6") : secret "maas-api-e2e-shared-a-257940-serving-cert" not found
Jul 24 18:00:17.080135 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:17.080117    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb781198-a4b8-4591-8ac8-0067068e1ee6-openshift-service-ca\") pod \"maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm\" (UID: \"cb781198-a4b8-4591-8ac8-0067068e1ee6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm"
Jul 24 18:00:17.089115 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:17.089093    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2hk4\" (UniqueName: \"kubernetes.io/projected/cb781198-a4b8-4591-8ac8-0067068e1ee6-kube-api-access-p2hk4\") pod \"maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm\" (UID: \"cb781198-a4b8-4591-8ac8-0067068e1ee6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm"
Jul 24 18:00:17.584187 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:17.584121    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb781198-a4b8-4591-8ac8-0067068e1ee6-maas-api-tls\") pod \"maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm\" (UID: \"cb781198-a4b8-4591-8ac8-0067068e1ee6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm"
Jul 24 18:00:17.587368 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:17.587335    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb781198-a4b8-4591-8ac8-0067068e1ee6-maas-api-tls\") pod \"maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm\" (UID: \"cb781198-a4b8-4591-8ac8-0067068e1ee6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm"
Jul 24 18:00:17.815971 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:17.815933    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm"
Jul 24 18:00:17.954244 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:17.954211    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t"]
Jul 24 18:00:17.959018 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:17.958988    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t"
Jul 24 18:00:17.964482 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:17.964448    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-257940\""
Jul 24 18:00:17.964651 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:17.964484    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-257940-dockercfg-v29sc\""
Jul 24 18:00:17.978151 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:17.978106    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t"]
Jul 24 18:00:17.989583 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:17.989549    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm"]
Jul 24 18:00:17.991446 ip-10-0-131-159 kubenswrapper[2562]: W0724 18:00:17.991421    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb781198_a4b8_4591_8ac8_0067068e1ee6.slice/crio-10de26cbf1efc44795fe35a41ca0efe9cdeb3b50fb5b50b4b2de9ab6fff4c0f4 WatchSource:0}: Error finding container 10de26cbf1efc44795fe35a41ca0efe9cdeb3b50fb5b50b4b2de9ab6fff4c0f4: Status 404 returned error can't find the container with id 10de26cbf1efc44795fe35a41ca0efe9cdeb3b50fb5b50b4b2de9ab6fff4c0f4
Jul 24 18:00:18.067775 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:18.067743    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw"]
Jul 24 18:00:18.073422 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:18.073396    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw"
Jul 24 18:00:18.089409 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:18.089332    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw"]
Jul 24 18:00:18.089828 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:18.089775    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/354203bc-be04-4217-bc83-62afe2794986-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t\" (UID: \"354203bc-be04-4217-bc83-62afe2794986\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t"
Jul 24 18:00:18.089936 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:18.089922    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtldb\" (UniqueName: \"kubernetes.io/projected/354203bc-be04-4217-bc83-62afe2794986-kube-api-access-gtldb\") pod \"payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t\" (UID: \"354203bc-be04-4217-bc83-62afe2794986\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t"
Jul 24 18:00:18.099574 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:18.099536    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm" event={"ID":"cb781198-a4b8-4591-8ac8-0067068e1ee6","Type":"ContainerStarted","Data":"10de26cbf1efc44795fe35a41ca0efe9cdeb3b50fb5b50b4b2de9ab6fff4c0f4"}
Jul 24 18:00:18.191044 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:18.191003    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/354203bc-be04-4217-bc83-62afe2794986-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t\" (UID: \"354203bc-be04-4217-bc83-62afe2794986\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t"
Jul 24 18:00:18.191306 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:18.191103    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtldb\" (UniqueName: \"kubernetes.io/projected/354203bc-be04-4217-bc83-62afe2794986-kube-api-access-gtldb\") pod \"payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t\" (UID: \"354203bc-be04-4217-bc83-62afe2794986\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t"
Jul 24 18:00:18.191306 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:18.191134    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f42fa39-8173-4ec5-af8e-915ae82ead45-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw\" (UID: \"8f42fa39-8173-4ec5-af8e-915ae82ead45\") " pod="openshift-ingress/payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw"
Jul 24 18:00:18.191306 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:18.191186    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lncjl\" (UniqueName: \"kubernetes.io/projected/8f42fa39-8173-4ec5-af8e-915ae82ead45-kube-api-access-lncjl\") pod \"payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw\" (UID: \"8f42fa39-8173-4ec5-af8e-915ae82ead45\") " pod="openshift-ingress/payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw"
Jul 24 18:00:18.191857 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:18.191830    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/354203bc-be04-4217-bc83-62afe2794986-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t\" (UID: \"354203bc-be04-4217-bc83-62afe2794986\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t"
Jul 24 18:00:18.201153 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:18.201110    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtldb\" (UniqueName: \"kubernetes.io/projected/354203bc-be04-4217-bc83-62afe2794986-kube-api-access-gtldb\") pod \"payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t\" (UID: \"354203bc-be04-4217-bc83-62afe2794986\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t"
Jul 24 18:00:18.270162 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:18.270115    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t"
Jul 24 18:00:18.292669 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:18.292625    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f42fa39-8173-4ec5-af8e-915ae82ead45-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw\" (UID: \"8f42fa39-8173-4ec5-af8e-915ae82ead45\") " pod="openshift-ingress/payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw"
Jul 24 18:00:18.292834 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:18.292674    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lncjl\" (UniqueName: \"kubernetes.io/projected/8f42fa39-8173-4ec5-af8e-915ae82ead45-kube-api-access-lncjl\") pod \"payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw\" (UID: \"8f42fa39-8173-4ec5-af8e-915ae82ead45\") " pod="openshift-ingress/payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw"
Jul 24 18:00:18.293645 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:18.293615    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f42fa39-8173-4ec5-af8e-915ae82ead45-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw\" (UID: \"8f42fa39-8173-4ec5-af8e-915ae82ead45\") " pod="openshift-ingress/payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw"
Jul 24 18:00:18.301694 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:18.301650    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lncjl\" (UniqueName: \"kubernetes.io/projected/8f42fa39-8173-4ec5-af8e-915ae82ead45-kube-api-access-lncjl\") pod \"payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw\" (UID: \"8f42fa39-8173-4ec5-af8e-915ae82ead45\") " pod="openshift-ingress/payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw"
Jul 24 18:00:18.385451 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:18.385420    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw"
Jul 24 18:00:18.417932 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:18.417894    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t"]
Jul 24 18:00:18.418951 ip-10-0-131-159 kubenswrapper[2562]: W0724 18:00:18.418911    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod354203bc_be04_4217_bc83_62afe2794986.slice/crio-a3e0be88b735e0f5e05c1d0c29f304d26fda19656f1be804c95a0878aa1f5618 WatchSource:0}: Error finding container a3e0be88b735e0f5e05c1d0c29f304d26fda19656f1be804c95a0878aa1f5618: Status 404 returned error can't find the container with id a3e0be88b735e0f5e05c1d0c29f304d26fda19656f1be804c95a0878aa1f5618
Jul 24 18:00:18.556396 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:18.556372    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw"]
Jul 24 18:00:18.559645 ip-10-0-131-159 kubenswrapper[2562]: W0724 18:00:18.559615    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f42fa39_8173_4ec5_af8e_915ae82ead45.slice/crio-8786f8918e2ca59024fe16e82d9aca79c9599e048a7c3cc6f0d5010b13603d3c WatchSource:0}: Error finding container 8786f8918e2ca59024fe16e82d9aca79c9599e048a7c3cc6f0d5010b13603d3c: Status 404 returned error can't find the container with id 8786f8918e2ca59024fe16e82d9aca79c9599e048a7c3cc6f0d5010b13603d3c
Jul 24 18:00:19.106978 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:19.106941    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t" event={"ID":"354203bc-be04-4217-bc83-62afe2794986","Type":"ContainerStarted","Data":"33aedbf2d529a7b10ca355a1bbe1a1a35b8e355616126fc43e80b9d596af8dd8"}
Jul 24 18:00:19.107578 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:19.107556    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t"
Jul 24 18:00:19.107757 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:19.107730    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t" event={"ID":"354203bc-be04-4217-bc83-62afe2794986","Type":"ContainerStarted","Data":"a3e0be88b735e0f5e05c1d0c29f304d26fda19656f1be804c95a0878aa1f5618"}
Jul 24 18:00:19.109049 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:19.109021    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm" event={"ID":"cb781198-a4b8-4591-8ac8-0067068e1ee6","Type":"ContainerStarted","Data":"b054d597f93fb74e047367f05ac03e56f4d4daf1dfd0dca7dfd9e8b5ff18b025"}
Jul 24 18:00:19.109228 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:19.109204    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm"
Jul 24 18:00:19.110970 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:19.110944    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw" event={"ID":"8f42fa39-8173-4ec5-af8e-915ae82ead45","Type":"ContainerStarted","Data":"f60fd1cf2467e2670cbbc91715bd5870fcfdd350613a19589d19aadf8065568c"}
Jul 24 18:00:19.111086 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:19.110993    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw" event={"ID":"8f42fa39-8173-4ec5-af8e-915ae82ead45","Type":"ContainerStarted","Data":"8786f8918e2ca59024fe16e82d9aca79c9599e048a7c3cc6f0d5010b13603d3c"}
Jul 24 18:00:19.111086 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:19.111045    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw"
Jul 24 18:00:19.126589 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:19.126531    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t" podStartSLOduration=2.126511032 podStartE2EDuration="2.126511032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:00:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:00:19.123676573 +0000 UTC m=+2581.850832790" watchObservedRunningTime="2026-07-24 18:00:19.126511032 +0000 UTC m=+2581.853667250"
Jul 24 18:00:19.141598 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:19.141546    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw" podStartSLOduration=1.141531793 podStartE2EDuration="1.141531793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:00:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:00:19.140222994 +0000 UTC m=+2581.867379209" watchObservedRunningTime="2026-07-24 18:00:19.141531793 +0000 UTC m=+2581.868688007"
Jul 24 18:00:19.159324 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:19.159266    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm" podStartSLOduration=2.669200794 podStartE2EDuration="3.159251451s" totalImagesPullingTime="490.050657ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:00:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:00:19.157714059 +0000 UTC m=+2581.884870274" watchObservedRunningTime="2026-07-24 18:00:19.159251451 +0000 UTC m=+2581.886407666"
Jul 24 18:00:24.932537 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:24.932505    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh"]
Jul 24 18:00:24.936298 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:24.936276    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh"
Jul 24 18:00:24.938933 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:24.938909    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-06ffdb-serving-cert\""
Jul 24 18:00:24.947815 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:24.947789    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh"]
Jul 24 18:00:25.055746 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:25.055706    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlkmz\" (UniqueName: \"kubernetes.io/projected/de410ff3-0656-41cf-b7f1-59f0a804d53c-kube-api-access-tlkmz\") pod \"maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh\" (UID: \"de410ff3-0656-41cf-b7f1-59f0a804d53c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh"
Jul 24 18:00:25.055746 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:25.055747    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de410ff3-0656-41cf-b7f1-59f0a804d53c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh\" (UID: \"de410ff3-0656-41cf-b7f1-59f0a804d53c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh"
Jul 24 18:00:25.055964 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:25.055872    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de410ff3-0656-41cf-b7f1-59f0a804d53c-maas-api-tls\") pod \"maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh\" (UID: \"de410ff3-0656-41cf-b7f1-59f0a804d53c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh"
Jul 24 18:00:25.122036 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:25.121999    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm"
Jul 24 18:00:25.156570 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:25.156534    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de410ff3-0656-41cf-b7f1-59f0a804d53c-maas-api-tls\") pod \"maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh\" (UID: \"de410ff3-0656-41cf-b7f1-59f0a804d53c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh"
Jul 24 18:00:25.156858 ip-10-0-131-159 kubenswrapper[2562]: E0724 18:00:25.156735    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-serving-cert: secret "maas-api-e2e-shared-b-06ffdb-serving-cert" not found
Jul 24 18:00:25.156997 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:25.156919    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlkmz\" (UniqueName: \"kubernetes.io/projected/de410ff3-0656-41cf-b7f1-59f0a804d53c-kube-api-access-tlkmz\") pod \"maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh\" (UID: \"de410ff3-0656-41cf-b7f1-59f0a804d53c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh"
Jul 24 18:00:25.156997 ip-10-0-131-159 kubenswrapper[2562]: E0724 18:00:25.156934    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de410ff3-0656-41cf-b7f1-59f0a804d53c-maas-api-tls podName:de410ff3-0656-41cf-b7f1-59f0a804d53c nodeName:}" failed. No retries permitted until 2026-07-24 18:00:25.656908139 +0000 UTC m=+2588.384064334 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/de410ff3-0656-41cf-b7f1-59f0a804d53c-maas-api-tls") pod "maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh" (UID: "de410ff3-0656-41cf-b7f1-59f0a804d53c") : secret "maas-api-e2e-shared-b-06ffdb-serving-cert" not found
Jul 24 18:00:25.157165 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:25.156999    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de410ff3-0656-41cf-b7f1-59f0a804d53c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh\" (UID: \"de410ff3-0656-41cf-b7f1-59f0a804d53c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh"
Jul 24 18:00:25.158730 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:25.158703    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de410ff3-0656-41cf-b7f1-59f0a804d53c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh\" (UID: \"de410ff3-0656-41cf-b7f1-59f0a804d53c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh"
Jul 24 18:00:25.169636 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:25.169606    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlkmz\" (UniqueName: \"kubernetes.io/projected/de410ff3-0656-41cf-b7f1-59f0a804d53c-kube-api-access-tlkmz\") pod \"maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh\" (UID: \"de410ff3-0656-41cf-b7f1-59f0a804d53c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh"
Jul 24 18:00:25.662215 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:25.662175    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de410ff3-0656-41cf-b7f1-59f0a804d53c-maas-api-tls\") pod \"maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh\" (UID: \"de410ff3-0656-41cf-b7f1-59f0a804d53c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh"
Jul 24 18:00:25.664581 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:25.664557    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de410ff3-0656-41cf-b7f1-59f0a804d53c-maas-api-tls\") pod \"maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh\" (UID: \"de410ff3-0656-41cf-b7f1-59f0a804d53c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh"
Jul 24 18:00:25.848542 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:25.848512    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh"
Jul 24 18:00:25.940671 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:25.940641    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt"]
Jul 24 18:00:25.944244 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:25.944221    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt"
Jul 24 18:00:25.947285 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:25.946812    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-06ffdb\""
Jul 24 18:00:25.947285 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:25.947068    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-06ffdb-dockercfg-tfwnz\""
Jul 24 18:00:25.956310 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:25.955571    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt"]
Jul 24 18:00:25.978901 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:25.978866    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v"]
Jul 24 18:00:25.982281 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:25.982261    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v"
Jul 24 18:00:25.994412 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:25.994387    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v"]
Jul 24 18:00:25.996477 ip-10-0-131-159 kubenswrapper[2562]: W0724 18:00:25.996454    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde410ff3_0656_41cf_b7f1_59f0a804d53c.slice/crio-8a8a2b067149da34d4d777ca0f425df408a894a1c9d6ea43e54ad5d914efb613 WatchSource:0}: Error finding container 8a8a2b067149da34d4d777ca0f425df408a894a1c9d6ea43e54ad5d914efb613: Status 404 returned error can't find the container with id 8a8a2b067149da34d4d777ca0f425df408a894a1c9d6ea43e54ad5d914efb613
Jul 24 18:00:25.996673 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:25.996656    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh"]
Jul 24 18:00:26.065624 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:26.065578    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tz8r\" (UniqueName: \"kubernetes.io/projected/5138e275-cd12-43ca-b492-68527f02052e-kube-api-access-5tz8r\") pod \"payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v\" (UID: \"5138e275-cd12-43ca-b492-68527f02052e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v"
Jul 24 18:00:26.065624 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:26.065626    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4zt6\" (UniqueName: \"kubernetes.io/projected/f0acefe2-8fbe-4532-994b-f3ee3ec39082-kube-api-access-r4zt6\") pod \"payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt\" (UID: \"f0acefe2-8fbe-4532-994b-f3ee3ec39082\") " pod="openshift-ingress/payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt"
Jul 24 18:00:26.065871 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:26.065740    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5138e275-cd12-43ca-b492-68527f02052e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v\" (UID: \"5138e275-cd12-43ca-b492-68527f02052e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v"
Jul 24 18:00:26.065871 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:26.065781    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0acefe2-8fbe-4532-994b-f3ee3ec39082-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt\" (UID: \"f0acefe2-8fbe-4532-994b-f3ee3ec39082\") " pod="openshift-ingress/payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt"
Jul 24 18:00:26.150389 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:26.150355    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh" event={"ID":"de410ff3-0656-41cf-b7f1-59f0a804d53c","Type":"ContainerStarted","Data":"8a8a2b067149da34d4d777ca0f425df408a894a1c9d6ea43e54ad5d914efb613"}
Jul 24 18:00:26.166886 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:26.166851    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5138e275-cd12-43ca-b492-68527f02052e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v\" (UID: \"5138e275-cd12-43ca-b492-68527f02052e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v"
Jul 24 18:00:26.167026 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:26.166896    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0acefe2-8fbe-4532-994b-f3ee3ec39082-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt\" (UID: \"f0acefe2-8fbe-4532-994b-f3ee3ec39082\") " pod="openshift-ingress/payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt"
Jul 24 18:00:26.167026 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:26.166950    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tz8r\" (UniqueName: \"kubernetes.io/projected/5138e275-cd12-43ca-b492-68527f02052e-kube-api-access-5tz8r\") pod \"payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v\" (UID: \"5138e275-cd12-43ca-b492-68527f02052e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v"
Jul 24 18:00:26.167026 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:26.166979    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4zt6\" (UniqueName: \"kubernetes.io/projected/f0acefe2-8fbe-4532-994b-f3ee3ec39082-kube-api-access-r4zt6\") pod \"payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt\" (UID: \"f0acefe2-8fbe-4532-994b-f3ee3ec39082\") " pod="openshift-ingress/payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt"
Jul 24 18:00:26.167838 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:26.167814    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5138e275-cd12-43ca-b492-68527f02052e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v\" (UID: \"5138e275-cd12-43ca-b492-68527f02052e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v"
Jul 24 18:00:26.167890 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:26.167814    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0acefe2-8fbe-4532-994b-f3ee3ec39082-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt\" (UID: \"f0acefe2-8fbe-4532-994b-f3ee3ec39082\") " pod="openshift-ingress/payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt"
Jul 24 18:00:26.175049 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:26.175020    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4zt6\" (UniqueName: \"kubernetes.io/projected/f0acefe2-8fbe-4532-994b-f3ee3ec39082-kube-api-access-r4zt6\") pod \"payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt\" (UID: \"f0acefe2-8fbe-4532-994b-f3ee3ec39082\") " pod="openshift-ingress/payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt"
Jul 24 18:00:26.175211 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:26.175188    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tz8r\" (UniqueName: \"kubernetes.io/projected/5138e275-cd12-43ca-b492-68527f02052e-kube-api-access-5tz8r\") pod \"payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v\" (UID: \"5138e275-cd12-43ca-b492-68527f02052e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v"
Jul 24 18:00:26.260369 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:26.260281    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt"
Jul 24 18:00:26.294357 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:26.294326    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v"
Jul 24 18:00:26.411571 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:26.411519    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt"]
Jul 24 18:00:26.413639 ip-10-0-131-159 kubenswrapper[2562]: W0724 18:00:26.413603    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0acefe2_8fbe_4532_994b_f3ee3ec39082.slice/crio-4115ea717d3201ec91bd68318629039feed9b4cd5138a84ef1dfac4931fd2dd5 WatchSource:0}: Error finding container 4115ea717d3201ec91bd68318629039feed9b4cd5138a84ef1dfac4931fd2dd5: Status 404 returned error can't find the container with id 4115ea717d3201ec91bd68318629039feed9b4cd5138a84ef1dfac4931fd2dd5
Jul 24 18:00:26.456576 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:26.453605    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v"]
Jul 24 18:00:27.157295 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:27.157251    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt" event={"ID":"f0acefe2-8fbe-4532-994b-f3ee3ec39082","Type":"ContainerStarted","Data":"d9e08f0af990ec023fa759cc3a3566c8292f631c98e2594bebc66c4fe661923f"}
Jul 24 18:00:27.157736 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:27.157318    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt" event={"ID":"f0acefe2-8fbe-4532-994b-f3ee3ec39082","Type":"ContainerStarted","Data":"4115ea717d3201ec91bd68318629039feed9b4cd5138a84ef1dfac4931fd2dd5"}
Jul 24 18:00:27.157736 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:27.157465    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt"
Jul 24 18:00:27.158823 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:27.158797    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh" event={"ID":"de410ff3-0656-41cf-b7f1-59f0a804d53c","Type":"ContainerStarted","Data":"ab52f4b672b98b25fa321642599a08bb649761d519501ab6f2772b0435e28702"}
Jul 24 18:00:27.158951 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:27.158872    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh"
Jul 24 18:00:27.160228 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:27.160209    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v" event={"ID":"5138e275-cd12-43ca-b492-68527f02052e","Type":"ContainerStarted","Data":"3ce236da0267658dfa1d2df13ca86a501234908f4162f7b571d1c7525949b07c"}
Jul 24 18:00:27.160311 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:27.160244    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v" event={"ID":"5138e275-cd12-43ca-b492-68527f02052e","Type":"ContainerStarted","Data":"c5fcdc8670ea5203fbeafc7ff59f1892a37de098236de0455af2bdc95b58aec1"}
Jul 24 18:00:27.160311 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:27.160265    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v"
Jul 24 18:00:27.177996 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:27.177940    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt" podStartSLOduration=2.177921823 podStartE2EDuration="2.177921823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:00:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:00:27.176104825 +0000 UTC m=+2589.903261067" watchObservedRunningTime="2026-07-24 18:00:27.177921823 +0000 UTC m=+2589.905078038"
Jul 24 18:00:27.224571 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:27.224505    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh" podStartSLOduration=2.538286349 podStartE2EDuration="3.224485047s" totalImagesPullingTime="686.198698ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:00:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:00:27.200855637 +0000 UTC m=+2589.928011851" watchObservedRunningTime="2026-07-24 18:00:27.224485047 +0000 UTC m=+2589.951641260"
Jul 24 18:00:27.225092 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:27.225055    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v" podStartSLOduration=2.225042906 podStartE2EDuration="2.225042906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:00:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:00:27.223646663 +0000 UTC m=+2589.950802874" watchObservedRunningTime="2026-07-24 18:00:27.225042906 +0000 UTC m=+2589.952199120"
Jul 24 18:00:30.119204 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:30.119170    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw"
Jul 24 18:00:31.125479 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:31.125449    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t"
Jul 24 18:00:33.171996 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:33.171965    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh"
Jul 24 18:00:38.168000 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:38.167970    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v"
Jul 24 18:00:38.168491 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:00:38.168040    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt"
Jul 24 18:03:18.098734 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:03:18.098681    2562 scope.go:117] "RemoveContainer" containerID="924574e8ccae4cb2c4142e93cf35a2fca2946b49f365df19cdd236d8d67bff42"
Jul 24 18:06:50.467613 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:50.467541    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg"]
Jul 24 18:06:50.471276 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:50.471260    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg"
Jul 24 18:06:50.473847 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:50.473820    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-737ef6-serving-cert\""
Jul 24 18:06:50.488675 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:50.488653    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg"]
Jul 24 18:06:50.497883 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:50.497856    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c161bae-9149-4cc8-8509-306f0cf43386-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg\" (UID: \"0c161bae-9149-4cc8-8509-306f0cf43386\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg"
Jul 24 18:06:50.497976 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:50.497946    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7wlj\" (UniqueName: \"kubernetes.io/projected/0c161bae-9149-4cc8-8509-306f0cf43386-kube-api-access-h7wlj\") pod \"maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg\" (UID: \"0c161bae-9149-4cc8-8509-306f0cf43386\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg"
Jul 24 18:06:50.498055 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:50.498041    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c161bae-9149-4cc8-8509-306f0cf43386-maas-api-tls\") pod \"maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg\" (UID: \"0c161bae-9149-4cc8-8509-306f0cf43386\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg"
Jul 24 18:06:50.599205 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:50.599123    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7wlj\" (UniqueName: \"kubernetes.io/projected/0c161bae-9149-4cc8-8509-306f0cf43386-kube-api-access-h7wlj\") pod \"maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg\" (UID: \"0c161bae-9149-4cc8-8509-306f0cf43386\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg"
Jul 24 18:06:50.599398 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:50.599290    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c161bae-9149-4cc8-8509-306f0cf43386-maas-api-tls\") pod \"maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg\" (UID: \"0c161bae-9149-4cc8-8509-306f0cf43386\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg"
Jul 24 18:06:50.599398 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:50.599350    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c161bae-9149-4cc8-8509-306f0cf43386-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg\" (UID: \"0c161bae-9149-4cc8-8509-306f0cf43386\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg"
Jul 24 18:06:50.599517 ip-10-0-131-159 kubenswrapper[2562]: E0724 18:06:50.599446    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-serving-cert: secret "maas-api-e2e-ipp-a-737ef6-serving-cert" not found
Jul 24 18:06:50.599581 ip-10-0-131-159 kubenswrapper[2562]: E0724 18:06:50.599567    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c161bae-9149-4cc8-8509-306f0cf43386-maas-api-tls podName:0c161bae-9149-4cc8-8509-306f0cf43386 nodeName:}" failed. No retries permitted until 2026-07-24 18:06:51.099540467 +0000 UTC m=+2973.826696664 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0c161bae-9149-4cc8-8509-306f0cf43386-maas-api-tls") pod "maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg" (UID: "0c161bae-9149-4cc8-8509-306f0cf43386") : secret "maas-api-e2e-ipp-a-737ef6-serving-cert" not found
Jul 24 18:06:50.600078 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:50.600056    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c161bae-9149-4cc8-8509-306f0cf43386-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg\" (UID: \"0c161bae-9149-4cc8-8509-306f0cf43386\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg"
Jul 24 18:06:50.609243 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:50.609214    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7wlj\" (UniqueName: \"kubernetes.io/projected/0c161bae-9149-4cc8-8509-306f0cf43386-kube-api-access-h7wlj\") pod \"maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg\" (UID: \"0c161bae-9149-4cc8-8509-306f0cf43386\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg"
Jul 24 18:06:51.106003 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:51.105967    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c161bae-9149-4cc8-8509-306f0cf43386-maas-api-tls\") pod \"maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg\" (UID: \"0c161bae-9149-4cc8-8509-306f0cf43386\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg"
Jul 24 18:06:51.108577 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:51.108543    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c161bae-9149-4cc8-8509-306f0cf43386-maas-api-tls\") pod \"maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg\" (UID: \"0c161bae-9149-4cc8-8509-306f0cf43386\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg"
Jul 24 18:06:51.381706 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:51.381627    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg"
Jul 24 18:06:51.428397 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:51.428365    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr"]
Jul 24 18:06:51.432328 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:51.432307    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr"
Jul 24 18:06:51.436366 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:51.435893    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-737ef6\""
Jul 24 18:06:51.447979 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:51.447889    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr"]
Jul 24 18:06:51.510057 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:51.510020    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/923f1d57-bfe8-46bf-9b1d-bb9c34ebb177-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr\" (UID: \"923f1d57-bfe8-46bf-9b1d-bb9c34ebb177\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr"
Jul 24 18:06:51.510535 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:51.510098    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvpfb\" (UniqueName: \"kubernetes.io/projected/923f1d57-bfe8-46bf-9b1d-bb9c34ebb177-kube-api-access-bvpfb\") pod \"payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr\" (UID: \"923f1d57-bfe8-46bf-9b1d-bb9c34ebb177\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr"
Jul 24 18:06:51.510535 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:51.510406    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn"]
Jul 24 18:06:51.515321 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:51.515298    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn"
Jul 24 18:06:51.518060 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:51.518034    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-737ef6-dockercfg-62ml5\""
Jul 24 18:06:51.534071 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:51.533299    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn"]
Jul 24 18:06:51.544780 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:51.544323    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg"]
Jul 24 18:06:51.546530 ip-10-0-131-159 kubenswrapper[2562]: W0724 18:06:51.544988    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c161bae_9149_4cc8_8509_306f0cf43386.slice/crio-57f6554ca92618fd12f006ee738e810828a4c3f1582ce34d9c7c40b24ea5711e WatchSource:0}: Error finding container 57f6554ca92618fd12f006ee738e810828a4c3f1582ce34d9c7c40b24ea5711e: Status 404 returned error can't find the container with id 57f6554ca92618fd12f006ee738e810828a4c3f1582ce34d9c7c40b24ea5711e
Jul 24 18:06:51.548030 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:51.548007    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 18:06:51.611071 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:51.611026    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66jjz\" (UniqueName: \"kubernetes.io/projected/7c1019c8-f3e9-4c39-8836-14ca62dde1e4-kube-api-access-66jjz\") pod \"payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn\" (UID: \"7c1019c8-f3e9-4c39-8836-14ca62dde1e4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn"
Jul 24 18:06:51.611071 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:51.611076    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c1019c8-f3e9-4c39-8836-14ca62dde1e4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn\" (UID: \"7c1019c8-f3e9-4c39-8836-14ca62dde1e4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn"
Jul 24 18:06:51.611344 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:51.611158    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/923f1d57-bfe8-46bf-9b1d-bb9c34ebb177-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr\" (UID: \"923f1d57-bfe8-46bf-9b1d-bb9c34ebb177\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr"
Jul 24 18:06:51.611344 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:51.611220    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvpfb\" (UniqueName: \"kubernetes.io/projected/923f1d57-bfe8-46bf-9b1d-bb9c34ebb177-kube-api-access-bvpfb\") pod \"payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr\" (UID: \"923f1d57-bfe8-46bf-9b1d-bb9c34ebb177\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr"
Jul 24 18:06:51.612073 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:51.612045    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/923f1d57-bfe8-46bf-9b1d-bb9c34ebb177-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr\" (UID: \"923f1d57-bfe8-46bf-9b1d-bb9c34ebb177\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr"
Jul 24 18:06:51.621172 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:51.621129    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvpfb\" (UniqueName: \"kubernetes.io/projected/923f1d57-bfe8-46bf-9b1d-bb9c34ebb177-kube-api-access-bvpfb\") pod \"payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr\" (UID: \"923f1d57-bfe8-46bf-9b1d-bb9c34ebb177\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr"
Jul 24 18:06:51.712501 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:51.712406    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66jjz\" (UniqueName: \"kubernetes.io/projected/7c1019c8-f3e9-4c39-8836-14ca62dde1e4-kube-api-access-66jjz\") pod \"payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn\" (UID: \"7c1019c8-f3e9-4c39-8836-14ca62dde1e4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn"
Jul 24 18:06:51.712501 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:51.712447    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c1019c8-f3e9-4c39-8836-14ca62dde1e4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn\" (UID: \"7c1019c8-f3e9-4c39-8836-14ca62dde1e4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn"
Jul 24 18:06:51.713271 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:51.713247    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c1019c8-f3e9-4c39-8836-14ca62dde1e4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn\" (UID: \"7c1019c8-f3e9-4c39-8836-14ca62dde1e4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn"
Jul 24 18:06:51.723828 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:51.723801    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66jjz\" (UniqueName: \"kubernetes.io/projected/7c1019c8-f3e9-4c39-8836-14ca62dde1e4-kube-api-access-66jjz\") pod \"payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn\" (UID: \"7c1019c8-f3e9-4c39-8836-14ca62dde1e4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn"
Jul 24 18:06:51.748537 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:51.748509    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr"
Jul 24 18:06:51.832424 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:51.832388    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn"
Jul 24 18:06:51.896351 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:51.896265    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr"]
Jul 24 18:06:51.899437 ip-10-0-131-159 kubenswrapper[2562]: W0724 18:06:51.899408    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod923f1d57_bfe8_46bf_9b1d_bb9c34ebb177.slice/crio-d85c41068d0d56d3ec93243874e8fef56a13165a22868e8f6b1310f143b5c1aa WatchSource:0}: Error finding container d85c41068d0d56d3ec93243874e8fef56a13165a22868e8f6b1310f143b5c1aa: Status 404 returned error can't find the container with id d85c41068d0d56d3ec93243874e8fef56a13165a22868e8f6b1310f143b5c1aa
Jul 24 18:06:52.169014 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:52.168976    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr" event={"ID":"923f1d57-bfe8-46bf-9b1d-bb9c34ebb177","Type":"ContainerStarted","Data":"0bd22f579c55b8f006f87a3b690173e7e5e6f7615741f645285b2eacded39ed4"}
Jul 24 18:06:52.169213 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:52.169028    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr" event={"ID":"923f1d57-bfe8-46bf-9b1d-bb9c34ebb177","Type":"ContainerStarted","Data":"d85c41068d0d56d3ec93243874e8fef56a13165a22868e8f6b1310f143b5c1aa"}
Jul 24 18:06:52.169213 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:52.169087    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr"
Jul 24 18:06:52.170426 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:52.170402    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg" event={"ID":"0c161bae-9149-4cc8-8509-306f0cf43386","Type":"ContainerStarted","Data":"57f6554ca92618fd12f006ee738e810828a4c3f1582ce34d9c7c40b24ea5711e"}
Jul 24 18:06:52.193218 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:52.193177    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr" podStartSLOduration=1.193162764 podStartE2EDuration="1.193162764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:06:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:06:52.191057566 +0000 UTC m=+2974.918213805" watchObservedRunningTime="2026-07-24 18:06:52.193162764 +0000 UTC m=+2974.920318977"
Jul 24 18:06:52.197127 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:52.197103    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn"]
Jul 24 18:06:52.199210 ip-10-0-131-159 kubenswrapper[2562]: W0724 18:06:52.199186    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c1019c8_f3e9_4c39_8836_14ca62dde1e4.slice/crio-f8e6cc1e5d35a977ffbb0d85ba420bacdd93124f32d229d1d18d19ad7ba854f3 WatchSource:0}: Error finding container f8e6cc1e5d35a977ffbb0d85ba420bacdd93124f32d229d1d18d19ad7ba854f3: Status 404 returned error can't find the container with id f8e6cc1e5d35a977ffbb0d85ba420bacdd93124f32d229d1d18d19ad7ba854f3
Jul 24 18:06:53.177960 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:53.177903    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn" event={"ID":"7c1019c8-f3e9-4c39-8836-14ca62dde1e4","Type":"ContainerStarted","Data":"a2f83e01bd5b093e3f8c8f0ea1987fa5cb74df3161f29c0a27da5a0b989682d6"}
Jul 24 18:06:53.178454 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:53.177981    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn" event={"ID":"7c1019c8-f3e9-4c39-8836-14ca62dde1e4","Type":"ContainerStarted","Data":"f8e6cc1e5d35a977ffbb0d85ba420bacdd93124f32d229d1d18d19ad7ba854f3"}
Jul 24 18:06:53.178454 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:53.178014    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn"
Jul 24 18:06:53.179598 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:53.179570    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg" event={"ID":"0c161bae-9149-4cc8-8509-306f0cf43386","Type":"ContainerStarted","Data":"caa6bb4234437108b0d457faa123e1e08251e682e3ab6acc7d5411c80e423def"}
Jul 24 18:06:53.179775 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:53.179760    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg"
Jul 24 18:06:53.196755 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:53.196705    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn" podStartSLOduration=2.196690455 podStartE2EDuration="2.196690455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:06:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:06:53.194750998 +0000 UTC m=+2975.921907213" watchObservedRunningTime="2026-07-24 18:06:53.196690455 +0000 UTC m=+2975.923846669"
Jul 24 18:06:53.214970 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:53.214919    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg" podStartSLOduration=2.155762149 podStartE2EDuration="3.214898647s" totalImagesPullingTime="1.059136498s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:06:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:06:53.214614762 +0000 UTC m=+2975.941770977" watchObservedRunningTime="2026-07-24 18:06:53.214898647 +0000 UTC m=+2975.942054864"
Jul 24 18:06:59.190655 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:06:59.190627    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg"
Jul 24 18:07:03.182535 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:03.182501    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr"
Jul 24 18:07:04.186488 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:04.186453    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn"
Jul 24 18:07:09.594020 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:09.593988    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl"]
Jul 24 18:07:09.597609 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:09.597588    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl"
Jul 24 18:07:09.603186 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:09.603135    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-53e30e-serving-cert\""
Jul 24 18:07:09.622134 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:09.622107    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl"]
Jul 24 18:07:09.691205 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:09.691162    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf94g\" (UniqueName: \"kubernetes.io/projected/6ff72e08-8a17-4dbe-996f-342c7db1c582-kube-api-access-tf94g\") pod \"maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl\" (UID: \"6ff72e08-8a17-4dbe-996f-342c7db1c582\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl"
Jul 24 18:07:09.691378 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:09.691326    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ff72e08-8a17-4dbe-996f-342c7db1c582-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl\" (UID: \"6ff72e08-8a17-4dbe-996f-342c7db1c582\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl"
Jul 24 18:07:09.691378 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:09.691357    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ff72e08-8a17-4dbe-996f-342c7db1c582-maas-api-tls\") pod \"maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl\" (UID: \"6ff72e08-8a17-4dbe-996f-342c7db1c582\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl"
Jul 24 18:07:09.792130 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:09.792083    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ff72e08-8a17-4dbe-996f-342c7db1c582-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl\" (UID: \"6ff72e08-8a17-4dbe-996f-342c7db1c582\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl"
Jul 24 18:07:09.792130 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:09.792128    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ff72e08-8a17-4dbe-996f-342c7db1c582-maas-api-tls\") pod \"maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl\" (UID: \"6ff72e08-8a17-4dbe-996f-342c7db1c582\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl"
Jul 24 18:07:09.792421 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:09.792231    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf94g\" (UniqueName: \"kubernetes.io/projected/6ff72e08-8a17-4dbe-996f-342c7db1c582-kube-api-access-tf94g\") pod \"maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl\" (UID: \"6ff72e08-8a17-4dbe-996f-342c7db1c582\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl"
Jul 24 18:07:09.792421 ip-10-0-131-159 kubenswrapper[2562]: E0724 18:07:09.792326    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-serving-cert: secret "maas-api-e2e-ipp-b-53e30e-serving-cert" not found
Jul 24 18:07:09.792421 ip-10-0-131-159 kubenswrapper[2562]: E0724 18:07:09.792396    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ff72e08-8a17-4dbe-996f-342c7db1c582-maas-api-tls podName:6ff72e08-8a17-4dbe-996f-342c7db1c582 nodeName:}" failed. No retries permitted until 2026-07-24 18:07:10.292377239 +0000 UTC m=+2993.019533436 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6ff72e08-8a17-4dbe-996f-342c7db1c582-maas-api-tls") pod "maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl" (UID: "6ff72e08-8a17-4dbe-996f-342c7db1c582") : secret "maas-api-e2e-ipp-b-53e30e-serving-cert" not found
Jul 24 18:07:09.792826 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:09.792806    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ff72e08-8a17-4dbe-996f-342c7db1c582-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl\" (UID: \"6ff72e08-8a17-4dbe-996f-342c7db1c582\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl"
Jul 24 18:07:09.807634 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:09.807602    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf94g\" (UniqueName: \"kubernetes.io/projected/6ff72e08-8a17-4dbe-996f-342c7db1c582-kube-api-access-tf94g\") pod \"maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl\" (UID: \"6ff72e08-8a17-4dbe-996f-342c7db1c582\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl"
Jul 24 18:07:10.297925 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:10.297889    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ff72e08-8a17-4dbe-996f-342c7db1c582-maas-api-tls\") pod \"maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl\" (UID: \"6ff72e08-8a17-4dbe-996f-342c7db1c582\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl"
Jul 24 18:07:10.300570 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:10.300541    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ff72e08-8a17-4dbe-996f-342c7db1c582-maas-api-tls\") pod \"maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl\" (UID: \"6ff72e08-8a17-4dbe-996f-342c7db1c582\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl"
Jul 24 18:07:10.442101 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:10.442059    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g"]
Jul 24 18:07:10.446237 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:10.446213    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g"
Jul 24 18:07:10.449088 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:10.449068    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-53e30e\""
Jul 24 18:07:10.461414 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:10.461386    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g"]
Jul 24 18:07:10.508955 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:10.508924    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl"
Jul 24 18:07:10.599936 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:10.599897    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b4fc\" (UniqueName: \"kubernetes.io/projected/2e02e8b9-848a-4bc4-8910-5b37025b0839-kube-api-access-5b4fc\") pod \"payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g\" (UID: \"2e02e8b9-848a-4bc4-8910-5b37025b0839\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g"
Jul 24 18:07:10.600392 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:10.599942    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e02e8b9-848a-4bc4-8910-5b37025b0839-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g\" (UID: \"2e02e8b9-848a-4bc4-8910-5b37025b0839\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g"
Jul 24 18:07:10.660777 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:10.660752    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl"]
Jul 24 18:07:10.663037 ip-10-0-131-159 kubenswrapper[2562]: W0724 18:07:10.663010    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ff72e08_8a17_4dbe_996f_342c7db1c582.slice/crio-0f90c5719b6c292f50511b547121a2d1c9ca28104d90a9d645aec0c9c8b85f8f WatchSource:0}: Error finding container 0f90c5719b6c292f50511b547121a2d1c9ca28104d90a9d645aec0c9c8b85f8f: Status 404 returned error can't find the container with id 0f90c5719b6c292f50511b547121a2d1c9ca28104d90a9d645aec0c9c8b85f8f
Jul 24 18:07:10.701285 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:10.701250    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b4fc\" (UniqueName: \"kubernetes.io/projected/2e02e8b9-848a-4bc4-8910-5b37025b0839-kube-api-access-5b4fc\") pod \"payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g\" (UID: \"2e02e8b9-848a-4bc4-8910-5b37025b0839\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g"
Jul 24 18:07:10.701434 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:10.701293    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e02e8b9-848a-4bc4-8910-5b37025b0839-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g\" (UID: \"2e02e8b9-848a-4bc4-8910-5b37025b0839\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g"
Jul 24 18:07:10.702000 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:10.701981    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e02e8b9-848a-4bc4-8910-5b37025b0839-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g\" (UID: \"2e02e8b9-848a-4bc4-8910-5b37025b0839\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g"
Jul 24 18:07:10.710376 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:10.710347    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b4fc\" (UniqueName: \"kubernetes.io/projected/2e02e8b9-848a-4bc4-8910-5b37025b0839-kube-api-access-5b4fc\") pod \"payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g\" (UID: \"2e02e8b9-848a-4bc4-8910-5b37025b0839\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g"
Jul 24 18:07:10.756899 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:10.756861    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g"
Jul 24 18:07:10.902413 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:10.902390    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g"]
Jul 24 18:07:10.905823 ip-10-0-131-159 kubenswrapper[2562]: W0724 18:07:10.905778    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e02e8b9_848a_4bc4_8910_5b37025b0839.slice/crio-c942a2813dbc11e22e2925316431735523a2d149af5d8ce345b32179cdaf3b36 WatchSource:0}: Error finding container c942a2813dbc11e22e2925316431735523a2d149af5d8ce345b32179cdaf3b36: Status 404 returned error can't find the container with id c942a2813dbc11e22e2925316431735523a2d149af5d8ce345b32179cdaf3b36
Jul 24 18:07:11.274950 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:11.274830    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g" event={"ID":"2e02e8b9-848a-4bc4-8910-5b37025b0839","Type":"ContainerStarted","Data":"4e6e6fbb3957c5b15f2850d5a6f9d65ef886a16f9f6f2897d4c86496bb0eb2af"}
Jul 24 18:07:11.274950 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:11.274920    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g"
Jul 24 18:07:11.274950 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:11.274944    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g" event={"ID":"2e02e8b9-848a-4bc4-8910-5b37025b0839","Type":"ContainerStarted","Data":"c942a2813dbc11e22e2925316431735523a2d149af5d8ce345b32179cdaf3b36"}
Jul 24 18:07:11.276424 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:11.276398    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl" event={"ID":"6ff72e08-8a17-4dbe-996f-342c7db1c582","Type":"ContainerStarted","Data":"8b8009a11d208aee9ad73aa4890ef28067316c312beea1426cec3c5067f80c36"}
Jul 24 18:07:11.276539 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:11.276455    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl"
Jul 24 18:07:11.276539 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:11.276481    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl" event={"ID":"6ff72e08-8a17-4dbe-996f-342c7db1c582","Type":"ContainerStarted","Data":"0f90c5719b6c292f50511b547121a2d1c9ca28104d90a9d645aec0c9c8b85f8f"}
Jul 24 18:07:11.293716 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:11.293653    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g" podStartSLOduration=1.293638375 podStartE2EDuration="1.293638375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:07:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:07:11.291501835 +0000 UTC m=+2994.018658048" watchObservedRunningTime="2026-07-24 18:07:11.293638375 +0000 UTC m=+2994.020794589"
Jul 24 18:07:11.320524 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:11.320473    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl" podStartSLOduration=1.9949866840000001 podStartE2EDuration="2.320454107s" totalImagesPullingTime="325.467423ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:07:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:07:11.319506854 +0000 UTC m=+2994.046663094" watchObservedRunningTime="2026-07-24 18:07:11.320454107 +0000 UTC m=+2994.047610321"
Jul 24 18:07:17.287882 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:17.287853    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl"
Jul 24 18:07:22.285113 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:22.285079    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g"
Jul 24 18:07:36.568624 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:07:36.568597    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-d2cqc_bf7f97e0-b594-4228-a7c6-197df6a2b4d7/payload-processing/0.log"
Jul 24 18:08:16.414862 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:16.414830    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr_923f1d57-bfe8-46bf-9b1d-bb9c34ebb177/payload-processing/0.log"
Jul 24 18:08:16.558277 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:16.558247    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-d2cqc_bf7f97e0-b594-4228-a7c6-197df6a2b4d7/payload-processing/0.log"
Jul 24 18:08:20.273009 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:20.272975    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq"]
Jul 24 18:08:20.277655 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:20.277623    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq"
Jul 24 18:08:20.280723 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:20.280696    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-fb4092-serving-cert\""
Jul 24 18:08:20.289996 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:20.289978    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq"]
Jul 24 18:08:20.359428 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:20.359400    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a94269f-5227-47b2-9a5b-acbb6d8889f8-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq\" (UID: \"7a94269f-5227-47b2-9a5b-acbb6d8889f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq"
Jul 24 18:08:20.359548 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:20.359475    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a94269f-5227-47b2-9a5b-acbb6d8889f8-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq\" (UID: \"7a94269f-5227-47b2-9a5b-acbb6d8889f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq"
Jul 24 18:08:20.359548 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:20.359510    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf46h\" (UniqueName: \"kubernetes.io/projected/7a94269f-5227-47b2-9a5b-acbb6d8889f8-kube-api-access-tf46h\") pod \"maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq\" (UID: \"7a94269f-5227-47b2-9a5b-acbb6d8889f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq"
Jul 24 18:08:20.460843 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:20.460811    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a94269f-5227-47b2-9a5b-acbb6d8889f8-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq\" (UID: \"7a94269f-5227-47b2-9a5b-acbb6d8889f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq"
Jul 24 18:08:20.460977 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:20.460855    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf46h\" (UniqueName: \"kubernetes.io/projected/7a94269f-5227-47b2-9a5b-acbb6d8889f8-kube-api-access-tf46h\") pod \"maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq\" (UID: \"7a94269f-5227-47b2-9a5b-acbb6d8889f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq"
Jul 24 18:08:20.460977 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:20.460908    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a94269f-5227-47b2-9a5b-acbb6d8889f8-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq\" (UID: \"7a94269f-5227-47b2-9a5b-acbb6d8889f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq"
Jul 24 18:08:20.460977 ip-10-0-131-159 kubenswrapper[2562]: E0724 18:08:20.460966    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-serving-cert: secret "maas-api-e2e-ipp-cleanup-fb4092-serving-cert" not found
Jul 24 18:08:20.461121 ip-10-0-131-159 kubenswrapper[2562]: E0724 18:08:20.461044    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a94269f-5227-47b2-9a5b-acbb6d8889f8-maas-api-tls podName:7a94269f-5227-47b2-9a5b-acbb6d8889f8 nodeName:}" failed. No retries permitted until 2026-07-24 18:08:20.961027394 +0000 UTC m=+3063.688183604 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7a94269f-5227-47b2-9a5b-acbb6d8889f8-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq" (UID: "7a94269f-5227-47b2-9a5b-acbb6d8889f8") : secret "maas-api-e2e-ipp-cleanup-fb4092-serving-cert" not found
Jul 24 18:08:20.461561 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:20.461543    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a94269f-5227-47b2-9a5b-acbb6d8889f8-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq\" (UID: \"7a94269f-5227-47b2-9a5b-acbb6d8889f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq"
Jul 24 18:08:20.481214 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:20.481185    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf46h\" (UniqueName: \"kubernetes.io/projected/7a94269f-5227-47b2-9a5b-acbb6d8889f8-kube-api-access-tf46h\") pod \"maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq\" (UID: \"7a94269f-5227-47b2-9a5b-acbb6d8889f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq"
Jul 24 18:08:20.965724 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:20.965683    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a94269f-5227-47b2-9a5b-acbb6d8889f8-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq\" (UID: \"7a94269f-5227-47b2-9a5b-acbb6d8889f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq"
Jul 24 18:08:20.968558 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:20.968528    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a94269f-5227-47b2-9a5b-acbb6d8889f8-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq\" (UID: \"7a94269f-5227-47b2-9a5b-acbb6d8889f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq"
Jul 24 18:08:21.189278 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:21.189233    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq"
Jul 24 18:08:21.386044 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:21.386019    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq"]
Jul 24 18:08:21.387925 ip-10-0-131-159 kubenswrapper[2562]: W0724 18:08:21.387900    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a94269f_5227_47b2_9a5b_acbb6d8889f8.slice/crio-61318a2517eb92e51d026588b14a949e8faa6de8c88d3cbe982bce5788f02b80 WatchSource:0}: Error finding container 61318a2517eb92e51d026588b14a949e8faa6de8c88d3cbe982bce5788f02b80: Status 404 returned error can't find the container with id 61318a2517eb92e51d026588b14a949e8faa6de8c88d3cbe982bce5788f02b80
Jul 24 18:08:21.658572 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:21.658533    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq" event={"ID":"7a94269f-5227-47b2-9a5b-acbb6d8889f8","Type":"ContainerStarted","Data":"61318a2517eb92e51d026588b14a949e8faa6de8c88d3cbe982bce5788f02b80"}
Jul 24 18:08:22.665314 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:22.665273    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq" event={"ID":"7a94269f-5227-47b2-9a5b-acbb6d8889f8","Type":"ContainerStarted","Data":"3722e779ff1def9ab733a58acd730f46eba157e587ab8ebd7ed87e9648682197"}
Jul 24 18:08:22.665748 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:22.665404    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq"
Jul 24 18:08:22.690757 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:22.690704    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq" podStartSLOduration=2.325207222 podStartE2EDuration="2.690688225s" totalImagesPullingTime="365.481003ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:08:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:08:22.688312104 +0000 UTC m=+3065.415468320" watchObservedRunningTime="2026-07-24 18:08:22.690688225 +0000 UTC m=+3065.417844439"
Jul 24 18:08:28.677160 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:28.677119    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq"
Jul 24 18:08:46.240714 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:46.240675    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq"]
Jul 24 18:08:46.241263 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:46.240897    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq" podUID="7a94269f-5227-47b2-9a5b-acbb6d8889f8" containerName="maas-api" containerID="cri-o://3722e779ff1def9ab733a58acd730f46eba157e587ab8ebd7ed87e9648682197" gracePeriod=30
Jul 24 18:08:46.513425 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:46.513393    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq"
Jul 24 18:08:46.598966 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:46.598933    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tf46h\" (UniqueName: \"kubernetes.io/projected/7a94269f-5227-47b2-9a5b-acbb6d8889f8-kube-api-access-tf46h\") pod \"7a94269f-5227-47b2-9a5b-acbb6d8889f8\" (UID: \"7a94269f-5227-47b2-9a5b-acbb6d8889f8\") "
Jul 24 18:08:46.599192 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:46.599010    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a94269f-5227-47b2-9a5b-acbb6d8889f8-maas-api-tls\") pod \"7a94269f-5227-47b2-9a5b-acbb6d8889f8\" (UID: \"7a94269f-5227-47b2-9a5b-acbb6d8889f8\") "
Jul 24 18:08:46.599192 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:46.599101    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a94269f-5227-47b2-9a5b-acbb6d8889f8-openshift-service-ca\") pod \"7a94269f-5227-47b2-9a5b-acbb6d8889f8\" (UID: \"7a94269f-5227-47b2-9a5b-acbb6d8889f8\") "
Jul 24 18:08:46.599522 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:46.599487    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a94269f-5227-47b2-9a5b-acbb6d8889f8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7a94269f-5227-47b2-9a5b-acbb6d8889f8" (UID: "7a94269f-5227-47b2-9a5b-acbb6d8889f8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:08:46.601442 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:46.601399    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a94269f-5227-47b2-9a5b-acbb6d8889f8-kube-api-access-tf46h" (OuterVolumeSpecName: "kube-api-access-tf46h") pod "7a94269f-5227-47b2-9a5b-acbb6d8889f8" (UID: "7a94269f-5227-47b2-9a5b-acbb6d8889f8"). InnerVolumeSpecName "kube-api-access-tf46h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:08:46.601593 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:46.601451    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a94269f-5227-47b2-9a5b-acbb6d8889f8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7a94269f-5227-47b2-9a5b-acbb6d8889f8" (UID: "7a94269f-5227-47b2-9a5b-acbb6d8889f8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 18:08:46.700096 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:46.700055    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a94269f-5227-47b2-9a5b-acbb6d8889f8-openshift-service-ca\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:08:46.700096 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:46.700088    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tf46h\" (UniqueName: \"kubernetes.io/projected/7a94269f-5227-47b2-9a5b-acbb6d8889f8-kube-api-access-tf46h\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:08:46.700096 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:46.700101    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a94269f-5227-47b2-9a5b-acbb6d8889f8-maas-api-tls\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:08:46.797250 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:46.797152    2562 generic.go:356] "Generic (PLEG): container finished" podID="7a94269f-5227-47b2-9a5b-acbb6d8889f8" containerID="3722e779ff1def9ab733a58acd730f46eba157e587ab8ebd7ed87e9648682197" exitCode=0
Jul 24 18:08:46.797250 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:46.797214    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq"
Jul 24 18:08:46.797250 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:46.797233    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq" event={"ID":"7a94269f-5227-47b2-9a5b-acbb6d8889f8","Type":"ContainerDied","Data":"3722e779ff1def9ab733a58acd730f46eba157e587ab8ebd7ed87e9648682197"}
Jul 24 18:08:46.797534 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:46.797298    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq" event={"ID":"7a94269f-5227-47b2-9a5b-acbb6d8889f8","Type":"ContainerDied","Data":"61318a2517eb92e51d026588b14a949e8faa6de8c88d3cbe982bce5788f02b80"}
Jul 24 18:08:46.797534 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:46.797323    2562 scope.go:117] "RemoveContainer" containerID="3722e779ff1def9ab733a58acd730f46eba157e587ab8ebd7ed87e9648682197"
Jul 24 18:08:46.808261 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:46.808239    2562 scope.go:117] "RemoveContainer" containerID="3722e779ff1def9ab733a58acd730f46eba157e587ab8ebd7ed87e9648682197"
Jul 24 18:08:46.808592 ip-10-0-131-159 kubenswrapper[2562]: E0724 18:08:46.808573    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3722e779ff1def9ab733a58acd730f46eba157e587ab8ebd7ed87e9648682197\": container with ID starting with 3722e779ff1def9ab733a58acd730f46eba157e587ab8ebd7ed87e9648682197 not found: ID does not exist" containerID="3722e779ff1def9ab733a58acd730f46eba157e587ab8ebd7ed87e9648682197"
Jul 24 18:08:46.808658 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:46.808605    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3722e779ff1def9ab733a58acd730f46eba157e587ab8ebd7ed87e9648682197"} err="failed to get container status \"3722e779ff1def9ab733a58acd730f46eba157e587ab8ebd7ed87e9648682197\": rpc error: code = NotFound desc = could not find container \"3722e779ff1def9ab733a58acd730f46eba157e587ab8ebd7ed87e9648682197\": container with ID starting with 3722e779ff1def9ab733a58acd730f46eba157e587ab8ebd7ed87e9648682197 not found: ID does not exist"
Jul 24 18:08:46.821402 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:46.821365    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq"]
Jul 24 18:08:46.828053 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:46.828023    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb4092-576fd5cc75-5cpmq"]
Jul 24 18:08:47.827375 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:08:47.827337    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a94269f-5227-47b2-9a5b-acbb6d8889f8" path="/var/lib/kubelet/pods/7a94269f-5227-47b2-9a5b-acbb6d8889f8/volumes"
Jul 24 18:09:17.027085 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.027046    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg"]
Jul 24 18:09:17.027533 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.027287    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg" podUID="0c161bae-9149-4cc8-8509-306f0cf43386" containerName="maas-api" containerID="cri-o://caa6bb4234437108b0d457faa123e1e08251e682e3ab6acc7d5411c80e423def" gracePeriod=30
Jul 24 18:09:17.102354 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.102310    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr"]
Jul 24 18:09:17.102595 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.102568    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr" podUID="923f1d57-bfe8-46bf-9b1d-bb9c34ebb177" containerName="payload-processing" containerID="cri-o://0bd22f579c55b8f006f87a3b690173e7e5e6f7615741f645285b2eacded39ed4" gracePeriod=30
Jul 24 18:09:17.117872 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.117834    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn"]
Jul 24 18:09:17.118216 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.118127    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn" podUID="7c1019c8-f3e9-4c39-8836-14ca62dde1e4" containerName="payload-pre-processing" containerID="cri-o://a2f83e01bd5b093e3f8c8f0ea1987fa5cb74df3161f29c0a27da5a0b989682d6" gracePeriod=30
Jul 24 18:09:17.432540 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.432516    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn"
Jul 24 18:09:17.435354 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.435335    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg"
Jul 24 18:09:17.439152 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.439121    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr"
Jul 24 18:09:17.494109 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.494075    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c1019c8-f3e9-4c39-8836-14ca62dde1e4-plugins-config-volume\") pod \"7c1019c8-f3e9-4c39-8836-14ca62dde1e4\" (UID: \"7c1019c8-f3e9-4c39-8836-14ca62dde1e4\") "
Jul 24 18:09:17.494330 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.494198    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c161bae-9149-4cc8-8509-306f0cf43386-openshift-service-ca\") pod \"0c161bae-9149-4cc8-8509-306f0cf43386\" (UID: \"0c161bae-9149-4cc8-8509-306f0cf43386\") "
Jul 24 18:09:17.494330 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.494262    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c161bae-9149-4cc8-8509-306f0cf43386-maas-api-tls\") pod \"0c161bae-9149-4cc8-8509-306f0cf43386\" (UID: \"0c161bae-9149-4cc8-8509-306f0cf43386\") "
Jul 24 18:09:17.494466 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.494329    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7wlj\" (UniqueName: \"kubernetes.io/projected/0c161bae-9149-4cc8-8509-306f0cf43386-kube-api-access-h7wlj\") pod \"0c161bae-9149-4cc8-8509-306f0cf43386\" (UID: \"0c161bae-9149-4cc8-8509-306f0cf43386\") "
Jul 24 18:09:17.494466 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.494363    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66jjz\" (UniqueName: \"kubernetes.io/projected/7c1019c8-f3e9-4c39-8836-14ca62dde1e4-kube-api-access-66jjz\") pod \"7c1019c8-f3e9-4c39-8836-14ca62dde1e4\" (UID: \"7c1019c8-f3e9-4c39-8836-14ca62dde1e4\") "
Jul 24 18:09:17.494865 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.494831    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c161bae-9149-4cc8-8509-306f0cf43386-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0c161bae-9149-4cc8-8509-306f0cf43386" (UID: "0c161bae-9149-4cc8-8509-306f0cf43386"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:09:17.494865 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.494855    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c1019c8-f3e9-4c39-8836-14ca62dde1e4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7c1019c8-f3e9-4c39-8836-14ca62dde1e4" (UID: "7c1019c8-f3e9-4c39-8836-14ca62dde1e4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:09:17.496904 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.496877    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c1019c8-f3e9-4c39-8836-14ca62dde1e4-kube-api-access-66jjz" (OuterVolumeSpecName: "kube-api-access-66jjz") pod "7c1019c8-f3e9-4c39-8836-14ca62dde1e4" (UID: "7c1019c8-f3e9-4c39-8836-14ca62dde1e4"). InnerVolumeSpecName "kube-api-access-66jjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:09:17.497221 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.497196    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c161bae-9149-4cc8-8509-306f0cf43386-kube-api-access-h7wlj" (OuterVolumeSpecName: "kube-api-access-h7wlj") pod "0c161bae-9149-4cc8-8509-306f0cf43386" (UID: "0c161bae-9149-4cc8-8509-306f0cf43386"). InnerVolumeSpecName "kube-api-access-h7wlj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:09:17.497221 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.497202    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c161bae-9149-4cc8-8509-306f0cf43386-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0c161bae-9149-4cc8-8509-306f0cf43386" (UID: "0c161bae-9149-4cc8-8509-306f0cf43386"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 18:09:17.595385 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.595350    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/923f1d57-bfe8-46bf-9b1d-bb9c34ebb177-plugins-config-volume\") pod \"923f1d57-bfe8-46bf-9b1d-bb9c34ebb177\" (UID: \"923f1d57-bfe8-46bf-9b1d-bb9c34ebb177\") "
Jul 24 18:09:17.595549 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.595493    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvpfb\" (UniqueName: \"kubernetes.io/projected/923f1d57-bfe8-46bf-9b1d-bb9c34ebb177-kube-api-access-bvpfb\") pod \"923f1d57-bfe8-46bf-9b1d-bb9c34ebb177\" (UID: \"923f1d57-bfe8-46bf-9b1d-bb9c34ebb177\") "
Jul 24 18:09:17.595680 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.595667    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7wlj\" (UniqueName: \"kubernetes.io/projected/0c161bae-9149-4cc8-8509-306f0cf43386-kube-api-access-h7wlj\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:09:17.595718 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.595683    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66jjz\" (UniqueName: \"kubernetes.io/projected/7c1019c8-f3e9-4c39-8836-14ca62dde1e4-kube-api-access-66jjz\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:09:17.595718 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.595693    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c1019c8-f3e9-4c39-8836-14ca62dde1e4-plugins-config-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:09:17.595718 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.595701    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c161bae-9149-4cc8-8509-306f0cf43386-openshift-service-ca\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:09:17.595718 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.595710    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c161bae-9149-4cc8-8509-306f0cf43386-maas-api-tls\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:09:17.595852 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.595791    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/923f1d57-bfe8-46bf-9b1d-bb9c34ebb177-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "923f1d57-bfe8-46bf-9b1d-bb9c34ebb177" (UID: "923f1d57-bfe8-46bf-9b1d-bb9c34ebb177"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:09:17.597507 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.597488    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/923f1d57-bfe8-46bf-9b1d-bb9c34ebb177-kube-api-access-bvpfb" (OuterVolumeSpecName: "kube-api-access-bvpfb") pod "923f1d57-bfe8-46bf-9b1d-bb9c34ebb177" (UID: "923f1d57-bfe8-46bf-9b1d-bb9c34ebb177"). InnerVolumeSpecName "kube-api-access-bvpfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:09:17.696515 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.696479    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvpfb\" (UniqueName: \"kubernetes.io/projected/923f1d57-bfe8-46bf-9b1d-bb9c34ebb177-kube-api-access-bvpfb\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:09:17.696515 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.696511    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/923f1d57-bfe8-46bf-9b1d-bb9c34ebb177-plugins-config-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:09:17.964986 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.964902    2562 generic.go:356] "Generic (PLEG): container finished" podID="923f1d57-bfe8-46bf-9b1d-bb9c34ebb177" containerID="0bd22f579c55b8f006f87a3b690173e7e5e6f7615741f645285b2eacded39ed4" exitCode=0
Jul 24 18:09:17.964986 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.964958    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr"
Jul 24 18:09:17.965232 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.964994    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr" event={"ID":"923f1d57-bfe8-46bf-9b1d-bb9c34ebb177","Type":"ContainerDied","Data":"0bd22f579c55b8f006f87a3b690173e7e5e6f7615741f645285b2eacded39ed4"}
Jul 24 18:09:17.965232 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.965054    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr" event={"ID":"923f1d57-bfe8-46bf-9b1d-bb9c34ebb177","Type":"ContainerDied","Data":"d85c41068d0d56d3ec93243874e8fef56a13165a22868e8f6b1310f143b5c1aa"}
Jul 24 18:09:17.965232 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.965083    2562 scope.go:117] "RemoveContainer" containerID="0bd22f579c55b8f006f87a3b690173e7e5e6f7615741f645285b2eacded39ed4"
Jul 24 18:09:17.966546 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.966518    2562 generic.go:356] "Generic (PLEG): container finished" podID="7c1019c8-f3e9-4c39-8836-14ca62dde1e4" containerID="a2f83e01bd5b093e3f8c8f0ea1987fa5cb74df3161f29c0a27da5a0b989682d6" exitCode=0
Jul 24 18:09:17.966665 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.966557    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn"
Jul 24 18:09:17.966665 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.966601    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn" event={"ID":"7c1019c8-f3e9-4c39-8836-14ca62dde1e4","Type":"ContainerDied","Data":"a2f83e01bd5b093e3f8c8f0ea1987fa5cb74df3161f29c0a27da5a0b989682d6"}
Jul 24 18:09:17.966665 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.966651    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn" event={"ID":"7c1019c8-f3e9-4c39-8836-14ca62dde1e4","Type":"ContainerDied","Data":"f8e6cc1e5d35a977ffbb0d85ba420bacdd93124f32d229d1d18d19ad7ba854f3"}
Jul 24 18:09:17.968089 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.968067    2562 generic.go:356] "Generic (PLEG): container finished" podID="0c161bae-9149-4cc8-8509-306f0cf43386" containerID="caa6bb4234437108b0d457faa123e1e08251e682e3ab6acc7d5411c80e423def" exitCode=0
Jul 24 18:09:17.968240 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.968093    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg" event={"ID":"0c161bae-9149-4cc8-8509-306f0cf43386","Type":"ContainerDied","Data":"caa6bb4234437108b0d457faa123e1e08251e682e3ab6acc7d5411c80e423def"}
Jul 24 18:09:17.968240 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.968112    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg"
Jul 24 18:09:17.968240 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.968176    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg" event={"ID":"0c161bae-9149-4cc8-8509-306f0cf43386","Type":"ContainerDied","Data":"57f6554ca92618fd12f006ee738e810828a4c3f1582ce34d9c7c40b24ea5711e"}
Jul 24 18:09:17.975679 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.975659    2562 scope.go:117] "RemoveContainer" containerID="0bd22f579c55b8f006f87a3b690173e7e5e6f7615741f645285b2eacded39ed4"
Jul 24 18:09:17.975978 ip-10-0-131-159 kubenswrapper[2562]: E0724 18:09:17.975948    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bd22f579c55b8f006f87a3b690173e7e5e6f7615741f645285b2eacded39ed4\": container with ID starting with 0bd22f579c55b8f006f87a3b690173e7e5e6f7615741f645285b2eacded39ed4 not found: ID does not exist" containerID="0bd22f579c55b8f006f87a3b690173e7e5e6f7615741f645285b2eacded39ed4"
Jul 24 18:09:17.976068 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.975988    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bd22f579c55b8f006f87a3b690173e7e5e6f7615741f645285b2eacded39ed4"} err="failed to get container status \"0bd22f579c55b8f006f87a3b690173e7e5e6f7615741f645285b2eacded39ed4\": rpc error: code = NotFound desc = could not find container \"0bd22f579c55b8f006f87a3b690173e7e5e6f7615741f645285b2eacded39ed4\": container with ID starting with 0bd22f579c55b8f006f87a3b690173e7e5e6f7615741f645285b2eacded39ed4 not found: ID does not exist"
Jul 24 18:09:17.976068 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.976012    2562 scope.go:117] "RemoveContainer" containerID="a2f83e01bd5b093e3f8c8f0ea1987fa5cb74df3161f29c0a27da5a0b989682d6"
Jul 24 18:09:17.983760 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.983745    2562 scope.go:117] "RemoveContainer" containerID="a2f83e01bd5b093e3f8c8f0ea1987fa5cb74df3161f29c0a27da5a0b989682d6"
Jul 24 18:09:17.984013 ip-10-0-131-159 kubenswrapper[2562]: E0724 18:09:17.983997    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2f83e01bd5b093e3f8c8f0ea1987fa5cb74df3161f29c0a27da5a0b989682d6\": container with ID starting with a2f83e01bd5b093e3f8c8f0ea1987fa5cb74df3161f29c0a27da5a0b989682d6 not found: ID does not exist" containerID="a2f83e01bd5b093e3f8c8f0ea1987fa5cb74df3161f29c0a27da5a0b989682d6"
Jul 24 18:09:17.984069 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.984017    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2f83e01bd5b093e3f8c8f0ea1987fa5cb74df3161f29c0a27da5a0b989682d6"} err="failed to get container status \"a2f83e01bd5b093e3f8c8f0ea1987fa5cb74df3161f29c0a27da5a0b989682d6\": rpc error: code = NotFound desc = could not find container \"a2f83e01bd5b093e3f8c8f0ea1987fa5cb74df3161f29c0a27da5a0b989682d6\": container with ID starting with a2f83e01bd5b093e3f8c8f0ea1987fa5cb74df3161f29c0a27da5a0b989682d6 not found: ID does not exist"
Jul 24 18:09:17.984069 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.984032    2562 scope.go:117] "RemoveContainer" containerID="caa6bb4234437108b0d457faa123e1e08251e682e3ab6acc7d5411c80e423def"
Jul 24 18:09:17.991095 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.991074    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg"]
Jul 24 18:09:17.992768 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.992747    2562 scope.go:117] "RemoveContainer" containerID="caa6bb4234437108b0d457faa123e1e08251e682e3ab6acc7d5411c80e423def"
Jul 24 18:09:17.993007 ip-10-0-131-159 kubenswrapper[2562]: E0724 18:09:17.992987    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caa6bb4234437108b0d457faa123e1e08251e682e3ab6acc7d5411c80e423def\": container with ID starting with caa6bb4234437108b0d457faa123e1e08251e682e3ab6acc7d5411c80e423def not found: ID does not exist" containerID="caa6bb4234437108b0d457faa123e1e08251e682e3ab6acc7d5411c80e423def"
Jul 24 18:09:17.993061 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.993016    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caa6bb4234437108b0d457faa123e1e08251e682e3ab6acc7d5411c80e423def"} err="failed to get container status \"caa6bb4234437108b0d457faa123e1e08251e682e3ab6acc7d5411c80e423def\": rpc error: code = NotFound desc = could not find container \"caa6bb4234437108b0d457faa123e1e08251e682e3ab6acc7d5411c80e423def\": container with ID starting with caa6bb4234437108b0d457faa123e1e08251e682e3ab6acc7d5411c80e423def not found: ID does not exist"
Jul 24 18:09:17.997269 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:17.997251    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-737ef6-7b457648c-hr9qg"]
Jul 24 18:09:18.009995 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:18.009977    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr"]
Jul 24 18:09:18.013588 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:18.013568    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-737ef6-77bf6f87b6-pzbmr"]
Jul 24 18:09:18.031390 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:18.031367    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn"]
Jul 24 18:09:18.035532 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:18.035504    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-737ef6-5c8f868f56-8lbcn"]
Jul 24 18:09:19.825274 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:19.825243    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c161bae-9149-4cc8-8509-306f0cf43386" path="/var/lib/kubelet/pods/0c161bae-9149-4cc8-8509-306f0cf43386/volumes"
Jul 24 18:09:19.825638 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:19.825616    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c1019c8-f3e9-4c39-8836-14ca62dde1e4" path="/var/lib/kubelet/pods/7c1019c8-f3e9-4c39-8836-14ca62dde1e4/volumes"
Jul 24 18:09:19.825931 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:09:19.825918    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="923f1d57-bfe8-46bf-9b1d-bb9c34ebb177" path="/var/lib/kubelet/pods/923f1d57-bfe8-46bf-9b1d-bb9c34ebb177/volumes"
Jul 24 18:10:27.968188 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:27.968134    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl"]
Jul 24 18:10:27.968733 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:27.968364    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl" podUID="6ff72e08-8a17-4dbe-996f-342c7db1c582" containerName="maas-api" containerID="cri-o://8b8009a11d208aee9ad73aa4890ef28067316c312beea1426cec3c5067f80c36" gracePeriod=30
Jul 24 18:10:28.045512 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.045480    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g"]
Jul 24 18:10:28.045773 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.045723    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g" podUID="2e02e8b9-848a-4bc4-8910-5b37025b0839" containerName="payload-pre-processing" containerID="cri-o://4e6e6fbb3957c5b15f2850d5a6f9d65ef886a16f9f6f2897d4c86496bb0eb2af" gracePeriod=30
Jul 24 18:10:28.310340 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.310308    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl"
Jul 24 18:10:28.343177 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.343127    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g"
Jul 24 18:10:28.347953 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.347929    2562 generic.go:356] "Generic (PLEG): container finished" podID="2e02e8b9-848a-4bc4-8910-5b37025b0839" containerID="4e6e6fbb3957c5b15f2850d5a6f9d65ef886a16f9f6f2897d4c86496bb0eb2af" exitCode=0
Jul 24 18:10:28.348079 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.347974    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g"
Jul 24 18:10:28.348079 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.348010    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g" event={"ID":"2e02e8b9-848a-4bc4-8910-5b37025b0839","Type":"ContainerDied","Data":"4e6e6fbb3957c5b15f2850d5a6f9d65ef886a16f9f6f2897d4c86496bb0eb2af"}
Jul 24 18:10:28.348079 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.348062    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g" event={"ID":"2e02e8b9-848a-4bc4-8910-5b37025b0839","Type":"ContainerDied","Data":"c942a2813dbc11e22e2925316431735523a2d149af5d8ce345b32179cdaf3b36"}
Jul 24 18:10:28.348253 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.348093    2562 scope.go:117] "RemoveContainer" containerID="4e6e6fbb3957c5b15f2850d5a6f9d65ef886a16f9f6f2897d4c86496bb0eb2af"
Jul 24 18:10:28.349458 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.349418    2562 generic.go:356] "Generic (PLEG): container finished" podID="6ff72e08-8a17-4dbe-996f-342c7db1c582" containerID="8b8009a11d208aee9ad73aa4890ef28067316c312beea1426cec3c5067f80c36" exitCode=0
Jul 24 18:10:28.349562 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.349459    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl"
Jul 24 18:10:28.349562 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.349486    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl" event={"ID":"6ff72e08-8a17-4dbe-996f-342c7db1c582","Type":"ContainerDied","Data":"8b8009a11d208aee9ad73aa4890ef28067316c312beea1426cec3c5067f80c36"}
Jul 24 18:10:28.349562 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.349532    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl" event={"ID":"6ff72e08-8a17-4dbe-996f-342c7db1c582","Type":"ContainerDied","Data":"0f90c5719b6c292f50511b547121a2d1c9ca28104d90a9d645aec0c9c8b85f8f"}
Jul 24 18:10:28.356630 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.356613    2562 scope.go:117] "RemoveContainer" containerID="4e6e6fbb3957c5b15f2850d5a6f9d65ef886a16f9f6f2897d4c86496bb0eb2af"
Jul 24 18:10:28.356930 ip-10-0-131-159 kubenswrapper[2562]: E0724 18:10:28.356910    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e6e6fbb3957c5b15f2850d5a6f9d65ef886a16f9f6f2897d4c86496bb0eb2af\": container with ID starting with 4e6e6fbb3957c5b15f2850d5a6f9d65ef886a16f9f6f2897d4c86496bb0eb2af not found: ID does not exist" containerID="4e6e6fbb3957c5b15f2850d5a6f9d65ef886a16f9f6f2897d4c86496bb0eb2af"
Jul 24 18:10:28.357000 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.356935    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e6e6fbb3957c5b15f2850d5a6f9d65ef886a16f9f6f2897d4c86496bb0eb2af"} err="failed to get container status \"4e6e6fbb3957c5b15f2850d5a6f9d65ef886a16f9f6f2897d4c86496bb0eb2af\": rpc error: code = NotFound desc = could not find container \"4e6e6fbb3957c5b15f2850d5a6f9d65ef886a16f9f6f2897d4c86496bb0eb2af\": container with ID starting with 4e6e6fbb3957c5b15f2850d5a6f9d65ef886a16f9f6f2897d4c86496bb0eb2af not found: ID does not exist"
Jul 24 18:10:28.357000 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.356951    2562 scope.go:117] "RemoveContainer" containerID="8b8009a11d208aee9ad73aa4890ef28067316c312beea1426cec3c5067f80c36"
Jul 24 18:10:28.364872 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.364769    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tf94g\" (UniqueName: \"kubernetes.io/projected/6ff72e08-8a17-4dbe-996f-342c7db1c582-kube-api-access-tf94g\") pod \"6ff72e08-8a17-4dbe-996f-342c7db1c582\" (UID: \"6ff72e08-8a17-4dbe-996f-342c7db1c582\") "
Jul 24 18:10:28.364959 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.364896    2562 scope.go:117] "RemoveContainer" containerID="8b8009a11d208aee9ad73aa4890ef28067316c312beea1426cec3c5067f80c36"
Jul 24 18:10:28.364959 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.364902    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ff72e08-8a17-4dbe-996f-342c7db1c582-openshift-service-ca\") pod \"6ff72e08-8a17-4dbe-996f-342c7db1c582\" (UID: \"6ff72e08-8a17-4dbe-996f-342c7db1c582\") "
Jul 24 18:10:28.365229 ip-10-0-131-159 kubenswrapper[2562]: E0724 18:10:28.365208    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b8009a11d208aee9ad73aa4890ef28067316c312beea1426cec3c5067f80c36\": container with ID starting with 8b8009a11d208aee9ad73aa4890ef28067316c312beea1426cec3c5067f80c36 not found: ID does not exist" containerID="8b8009a11d208aee9ad73aa4890ef28067316c312beea1426cec3c5067f80c36"
Jul 24 18:10:28.365308 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.365242    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b8009a11d208aee9ad73aa4890ef28067316c312beea1426cec3c5067f80c36"} err="failed to get container status \"8b8009a11d208aee9ad73aa4890ef28067316c312beea1426cec3c5067f80c36\": rpc error: code = NotFound desc = could not find container \"8b8009a11d208aee9ad73aa4890ef28067316c312beea1426cec3c5067f80c36\": container with ID starting with 8b8009a11d208aee9ad73aa4890ef28067316c312beea1426cec3c5067f80c36 not found: ID does not exist"
Jul 24 18:10:28.365365 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.365307    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ff72e08-8a17-4dbe-996f-342c7db1c582-maas-api-tls\") pod \"6ff72e08-8a17-4dbe-996f-342c7db1c582\" (UID: \"6ff72e08-8a17-4dbe-996f-342c7db1c582\") "
Jul 24 18:10:28.365794 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.365670    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ff72e08-8a17-4dbe-996f-342c7db1c582-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6ff72e08-8a17-4dbe-996f-342c7db1c582" (UID: "6ff72e08-8a17-4dbe-996f-342c7db1c582"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:10:28.366851 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.366771    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ff72e08-8a17-4dbe-996f-342c7db1c582-openshift-service-ca\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:10:28.367984 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.367960    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ff72e08-8a17-4dbe-996f-342c7db1c582-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6ff72e08-8a17-4dbe-996f-342c7db1c582" (UID: "6ff72e08-8a17-4dbe-996f-342c7db1c582"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 18:10:28.369050 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.369030    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ff72e08-8a17-4dbe-996f-342c7db1c582-kube-api-access-tf94g" (OuterVolumeSpecName: "kube-api-access-tf94g") pod "6ff72e08-8a17-4dbe-996f-342c7db1c582" (UID: "6ff72e08-8a17-4dbe-996f-342c7db1c582"). InnerVolumeSpecName "kube-api-access-tf94g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:10:28.467483 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.467455    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e02e8b9-848a-4bc4-8910-5b37025b0839-plugins-config-volume\") pod \"2e02e8b9-848a-4bc4-8910-5b37025b0839\" (UID: \"2e02e8b9-848a-4bc4-8910-5b37025b0839\") "
Jul 24 18:10:28.467669 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.467561    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5b4fc\" (UniqueName: \"kubernetes.io/projected/2e02e8b9-848a-4bc4-8910-5b37025b0839-kube-api-access-5b4fc\") pod \"2e02e8b9-848a-4bc4-8910-5b37025b0839\" (UID: \"2e02e8b9-848a-4bc4-8910-5b37025b0839\") "
Jul 24 18:10:28.467773 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.467758    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tf94g\" (UniqueName: \"kubernetes.io/projected/6ff72e08-8a17-4dbe-996f-342c7db1c582-kube-api-access-tf94g\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:10:28.467839 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.467773    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ff72e08-8a17-4dbe-996f-342c7db1c582-maas-api-tls\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:10:28.467897 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.467874    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e02e8b9-848a-4bc4-8910-5b37025b0839-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2e02e8b9-848a-4bc4-8910-5b37025b0839" (UID: "2e02e8b9-848a-4bc4-8910-5b37025b0839"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:10:28.469417 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.469397    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e02e8b9-848a-4bc4-8910-5b37025b0839-kube-api-access-5b4fc" (OuterVolumeSpecName: "kube-api-access-5b4fc") pod "2e02e8b9-848a-4bc4-8910-5b37025b0839" (UID: "2e02e8b9-848a-4bc4-8910-5b37025b0839"). InnerVolumeSpecName "kube-api-access-5b4fc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:10:28.568262 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.568181    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5b4fc\" (UniqueName: \"kubernetes.io/projected/2e02e8b9-848a-4bc4-8910-5b37025b0839-kube-api-access-5b4fc\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:10:28.568262 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.568207    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e02e8b9-848a-4bc4-8910-5b37025b0839-plugins-config-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:10:28.675272 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.675244    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g"]
Jul 24 18:10:28.679680 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.679650    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-53e30e-8877c8b4-7rd9g"]
Jul 24 18:10:28.692096 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.692072    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl"]
Jul 24 18:10:28.696401 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:28.696378    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-53e30e-679dd5c4f-2thxl"]
Jul 24 18:10:29.825876 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:29.825844    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e02e8b9-848a-4bc4-8910-5b37025b0839" path="/var/lib/kubelet/pods/2e02e8b9-848a-4bc4-8910-5b37025b0839/volumes"
Jul 24 18:10:29.826262 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:29.826249    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ff72e08-8a17-4dbe-996f-342c7db1c582" path="/var/lib/kubelet/pods/6ff72e08-8a17-4dbe-996f-342c7db1c582/volumes"
Jul 24 18:10:50.769069 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:50.769034    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm"]
Jul 24 18:10:50.769572 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:50.769384    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm" podUID="cb781198-a4b8-4591-8ac8-0067068e1ee6" containerName="maas-api" containerID="cri-o://b054d597f93fb74e047367f05ac03e56f4d4daf1dfd0dca7dfd9e8b5ff18b025" gracePeriod=30
Jul 24 18:10:50.830902 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:50.830860    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw"]
Jul 24 18:10:50.831258 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:50.831116    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw" podUID="8f42fa39-8173-4ec5-af8e-915ae82ead45" containerName="payload-processing" containerID="cri-o://f60fd1cf2467e2670cbbc91715bd5870fcfdd350613a19589d19aadf8065568c" gracePeriod=30
Jul 24 18:10:50.838527 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:50.838494    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t"]
Jul 24 18:10:50.838800 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:50.838719    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t" podUID="354203bc-be04-4217-bc83-62afe2794986" containerName="payload-pre-processing" containerID="cri-o://33aedbf2d529a7b10ca355a1bbe1a1a35b8e355616126fc43e80b9d596af8dd8" gracePeriod=30
Jul 24 18:10:51.098802 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.098774    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm"
Jul 24 18:10:51.140938 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.140905    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw"
Jul 24 18:10:51.167429 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.167403    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t"
Jul 24 18:10:51.292413 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.292378    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb781198-a4b8-4591-8ac8-0067068e1ee6-maas-api-tls\") pod \"cb781198-a4b8-4591-8ac8-0067068e1ee6\" (UID: \"cb781198-a4b8-4591-8ac8-0067068e1ee6\") "
Jul 24 18:10:51.292575 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.292447    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb781198-a4b8-4591-8ac8-0067068e1ee6-openshift-service-ca\") pod \"cb781198-a4b8-4591-8ac8-0067068e1ee6\" (UID: \"cb781198-a4b8-4591-8ac8-0067068e1ee6\") "
Jul 24 18:10:51.292661 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.292639    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtldb\" (UniqueName: \"kubernetes.io/projected/354203bc-be04-4217-bc83-62afe2794986-kube-api-access-gtldb\") pod \"354203bc-be04-4217-bc83-62afe2794986\" (UID: \"354203bc-be04-4217-bc83-62afe2794986\") "
Jul 24 18:10:51.292740 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.292725    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f42fa39-8173-4ec5-af8e-915ae82ead45-plugins-config-volume\") pod \"8f42fa39-8173-4ec5-af8e-915ae82ead45\" (UID: \"8f42fa39-8173-4ec5-af8e-915ae82ead45\") "
Jul 24 18:10:51.292789 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.292764    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2hk4\" (UniqueName: \"kubernetes.io/projected/cb781198-a4b8-4591-8ac8-0067068e1ee6-kube-api-access-p2hk4\") pod \"cb781198-a4b8-4591-8ac8-0067068e1ee6\" (UID: \"cb781198-a4b8-4591-8ac8-0067068e1ee6\") "
Jul 24 18:10:51.292789 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.292769    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb781198-a4b8-4591-8ac8-0067068e1ee6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cb781198-a4b8-4591-8ac8-0067068e1ee6" (UID: "cb781198-a4b8-4591-8ac8-0067068e1ee6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:10:51.292869 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.292789    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lncjl\" (UniqueName: \"kubernetes.io/projected/8f42fa39-8173-4ec5-af8e-915ae82ead45-kube-api-access-lncjl\") pod \"8f42fa39-8173-4ec5-af8e-915ae82ead45\" (UID: \"8f42fa39-8173-4ec5-af8e-915ae82ead45\") "
Jul 24 18:10:51.292915 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.292884    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/354203bc-be04-4217-bc83-62afe2794986-plugins-config-volume\") pod \"354203bc-be04-4217-bc83-62afe2794986\" (UID: \"354203bc-be04-4217-bc83-62afe2794986\") "
Jul 24 18:10:51.293272 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.293252    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb781198-a4b8-4591-8ac8-0067068e1ee6-openshift-service-ca\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:10:51.293391 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.293258    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f42fa39-8173-4ec5-af8e-915ae82ead45-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8f42fa39-8173-4ec5-af8e-915ae82ead45" (UID: "8f42fa39-8173-4ec5-af8e-915ae82ead45"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:10:51.293391 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.293292    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/354203bc-be04-4217-bc83-62afe2794986-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "354203bc-be04-4217-bc83-62afe2794986" (UID: "354203bc-be04-4217-bc83-62afe2794986"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:10:51.294670 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.294647    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/354203bc-be04-4217-bc83-62afe2794986-kube-api-access-gtldb" (OuterVolumeSpecName: "kube-api-access-gtldb") pod "354203bc-be04-4217-bc83-62afe2794986" (UID: "354203bc-be04-4217-bc83-62afe2794986"). InnerVolumeSpecName "kube-api-access-gtldb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:10:51.295050 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.295022    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb781198-a4b8-4591-8ac8-0067068e1ee6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cb781198-a4b8-4591-8ac8-0067068e1ee6" (UID: "cb781198-a4b8-4591-8ac8-0067068e1ee6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 18:10:51.295050 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.295029    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f42fa39-8173-4ec5-af8e-915ae82ead45-kube-api-access-lncjl" (OuterVolumeSpecName: "kube-api-access-lncjl") pod "8f42fa39-8173-4ec5-af8e-915ae82ead45" (UID: "8f42fa39-8173-4ec5-af8e-915ae82ead45"). InnerVolumeSpecName "kube-api-access-lncjl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:10:51.295218 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.295073    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb781198-a4b8-4591-8ac8-0067068e1ee6-kube-api-access-p2hk4" (OuterVolumeSpecName: "kube-api-access-p2hk4") pod "cb781198-a4b8-4591-8ac8-0067068e1ee6" (UID: "cb781198-a4b8-4591-8ac8-0067068e1ee6"). InnerVolumeSpecName "kube-api-access-p2hk4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:10:51.394666 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.394627    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f42fa39-8173-4ec5-af8e-915ae82ead45-plugins-config-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:10:51.394666 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.394660    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2hk4\" (UniqueName: \"kubernetes.io/projected/cb781198-a4b8-4591-8ac8-0067068e1ee6-kube-api-access-p2hk4\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:10:51.394666 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.394671    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lncjl\" (UniqueName: \"kubernetes.io/projected/8f42fa39-8173-4ec5-af8e-915ae82ead45-kube-api-access-lncjl\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:10:51.394916 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.394680    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/354203bc-be04-4217-bc83-62afe2794986-plugins-config-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:10:51.394916 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.394691    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb781198-a4b8-4591-8ac8-0067068e1ee6-maas-api-tls\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:10:51.394916 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.394699    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtldb\" (UniqueName: \"kubernetes.io/projected/354203bc-be04-4217-bc83-62afe2794986-kube-api-access-gtldb\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:10:51.475561 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.475524    2562 generic.go:356] "Generic (PLEG): container finished" podID="354203bc-be04-4217-bc83-62afe2794986" containerID="33aedbf2d529a7b10ca355a1bbe1a1a35b8e355616126fc43e80b9d596af8dd8" exitCode=0
Jul 24 18:10:51.475561 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.475565    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t"
Jul 24 18:10:51.475808 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.475616    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t" event={"ID":"354203bc-be04-4217-bc83-62afe2794986","Type":"ContainerDied","Data":"33aedbf2d529a7b10ca355a1bbe1a1a35b8e355616126fc43e80b9d596af8dd8"}
Jul 24 18:10:51.475808 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.475702    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t" event={"ID":"354203bc-be04-4217-bc83-62afe2794986","Type":"ContainerDied","Data":"a3e0be88b735e0f5e05c1d0c29f304d26fda19656f1be804c95a0878aa1f5618"}
Jul 24 18:10:51.475808 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.475730    2562 scope.go:117] "RemoveContainer" containerID="33aedbf2d529a7b10ca355a1bbe1a1a35b8e355616126fc43e80b9d596af8dd8"
Jul 24 18:10:51.477054 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.477037    2562 generic.go:356] "Generic (PLEG): container finished" podID="cb781198-a4b8-4591-8ac8-0067068e1ee6" containerID="b054d597f93fb74e047367f05ac03e56f4d4daf1dfd0dca7dfd9e8b5ff18b025" exitCode=0
Jul 24 18:10:51.477170 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.477097    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm"
Jul 24 18:10:51.477170 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.477106    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm" event={"ID":"cb781198-a4b8-4591-8ac8-0067068e1ee6","Type":"ContainerDied","Data":"b054d597f93fb74e047367f05ac03e56f4d4daf1dfd0dca7dfd9e8b5ff18b025"}
Jul 24 18:10:51.477291 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.477184    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm" event={"ID":"cb781198-a4b8-4591-8ac8-0067068e1ee6","Type":"ContainerDied","Data":"10de26cbf1efc44795fe35a41ca0efe9cdeb3b50fb5b50b4b2de9ab6fff4c0f4"}
Jul 24 18:10:51.478501 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.478464    2562 generic.go:356] "Generic (PLEG): container finished" podID="8f42fa39-8173-4ec5-af8e-915ae82ead45" containerID="f60fd1cf2467e2670cbbc91715bd5870fcfdd350613a19589d19aadf8065568c" exitCode=0
Jul 24 18:10:51.478597 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.478495    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw" event={"ID":"8f42fa39-8173-4ec5-af8e-915ae82ead45","Type":"ContainerDied","Data":"f60fd1cf2467e2670cbbc91715bd5870fcfdd350613a19589d19aadf8065568c"}
Jul 24 18:10:51.478597 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.478548    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw" event={"ID":"8f42fa39-8173-4ec5-af8e-915ae82ead45","Type":"ContainerDied","Data":"8786f8918e2ca59024fe16e82d9aca79c9599e048a7c3cc6f0d5010b13603d3c"}
Jul 24 18:10:51.478597 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.478502    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw"
Jul 24 18:10:51.487361 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.487339    2562 scope.go:117] "RemoveContainer" containerID="33aedbf2d529a7b10ca355a1bbe1a1a35b8e355616126fc43e80b9d596af8dd8"
Jul 24 18:10:51.487665 ip-10-0-131-159 kubenswrapper[2562]: E0724 18:10:51.487641    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33aedbf2d529a7b10ca355a1bbe1a1a35b8e355616126fc43e80b9d596af8dd8\": container with ID starting with 33aedbf2d529a7b10ca355a1bbe1a1a35b8e355616126fc43e80b9d596af8dd8 not found: ID does not exist" containerID="33aedbf2d529a7b10ca355a1bbe1a1a35b8e355616126fc43e80b9d596af8dd8"
Jul 24 18:10:51.487726 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.487680    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33aedbf2d529a7b10ca355a1bbe1a1a35b8e355616126fc43e80b9d596af8dd8"} err="failed to get container status \"33aedbf2d529a7b10ca355a1bbe1a1a35b8e355616126fc43e80b9d596af8dd8\": rpc error: code = NotFound desc = could not find container \"33aedbf2d529a7b10ca355a1bbe1a1a35b8e355616126fc43e80b9d596af8dd8\": container with ID starting with 33aedbf2d529a7b10ca355a1bbe1a1a35b8e355616126fc43e80b9d596af8dd8 not found: ID does not exist"
Jul 24 18:10:51.487726 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.487699    2562 scope.go:117] "RemoveContainer" containerID="b054d597f93fb74e047367f05ac03e56f4d4daf1dfd0dca7dfd9e8b5ff18b025"
Jul 24 18:10:51.496458 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.496441    2562 scope.go:117] "RemoveContainer" containerID="b054d597f93fb74e047367f05ac03e56f4d4daf1dfd0dca7dfd9e8b5ff18b025"
Jul 24 18:10:51.496716 ip-10-0-131-159 kubenswrapper[2562]: E0724 18:10:51.496698    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b054d597f93fb74e047367f05ac03e56f4d4daf1dfd0dca7dfd9e8b5ff18b025\": container with ID starting with b054d597f93fb74e047367f05ac03e56f4d4daf1dfd0dca7dfd9e8b5ff18b025 not found: ID does not exist" containerID="b054d597f93fb74e047367f05ac03e56f4d4daf1dfd0dca7dfd9e8b5ff18b025"
Jul 24 18:10:51.496760 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.496725    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b054d597f93fb74e047367f05ac03e56f4d4daf1dfd0dca7dfd9e8b5ff18b025"} err="failed to get container status \"b054d597f93fb74e047367f05ac03e56f4d4daf1dfd0dca7dfd9e8b5ff18b025\": rpc error: code = NotFound desc = could not find container \"b054d597f93fb74e047367f05ac03e56f4d4daf1dfd0dca7dfd9e8b5ff18b025\": container with ID starting with b054d597f93fb74e047367f05ac03e56f4d4daf1dfd0dca7dfd9e8b5ff18b025 not found: ID does not exist"
Jul 24 18:10:51.496760 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.496743    2562 scope.go:117] "RemoveContainer" containerID="f60fd1cf2467e2670cbbc91715bd5870fcfdd350613a19589d19aadf8065568c"
Jul 24 18:10:51.507482 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.507306    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm"]
Jul 24 18:10:51.510290 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.510268    2562 scope.go:117] "RemoveContainer" containerID="f60fd1cf2467e2670cbbc91715bd5870fcfdd350613a19589d19aadf8065568c"
Jul 24 18:10:51.510632 ip-10-0-131-159 kubenswrapper[2562]: E0724 18:10:51.510606    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f60fd1cf2467e2670cbbc91715bd5870fcfdd350613a19589d19aadf8065568c\": container with ID starting with f60fd1cf2467e2670cbbc91715bd5870fcfdd350613a19589d19aadf8065568c not found: ID does not exist" containerID="f60fd1cf2467e2670cbbc91715bd5870fcfdd350613a19589d19aadf8065568c"
Jul 24 18:10:51.510714 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.510642    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f60fd1cf2467e2670cbbc91715bd5870fcfdd350613a19589d19aadf8065568c"} err="failed to get container status \"f60fd1cf2467e2670cbbc91715bd5870fcfdd350613a19589d19aadf8065568c\": rpc error: code = NotFound desc = could not find container \"f60fd1cf2467e2670cbbc91715bd5870fcfdd350613a19589d19aadf8065568c\": container with ID starting with f60fd1cf2467e2670cbbc91715bd5870fcfdd350613a19589d19aadf8065568c not found: ID does not exist"
Jul 24 18:10:51.511659 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.511640    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-257940-64cc466b7b-zhnjm"]
Jul 24 18:10:51.525073 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.525048    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t"]
Jul 24 18:10:51.527588 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.527567    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t"]
Jul 24 18:10:51.540580 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.540557    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw"]
Jul 24 18:10:51.545040 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.545020    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-257940-b4854b7c8-hh9lw"]
Jul 24 18:10:51.825274 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.825242    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="354203bc-be04-4217-bc83-62afe2794986" path="/var/lib/kubelet/pods/354203bc-be04-4217-bc83-62afe2794986/volumes"
Jul 24 18:10:51.825613 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.825602    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f42fa39-8173-4ec5-af8e-915ae82ead45" path="/var/lib/kubelet/pods/8f42fa39-8173-4ec5-af8e-915ae82ead45/volumes"
Jul 24 18:10:51.825963 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:51.825950    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb781198-a4b8-4591-8ac8-0067068e1ee6" path="/var/lib/kubelet/pods/cb781198-a4b8-4591-8ac8-0067068e1ee6/volumes"
Jul 24 18:10:52.123378 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:10:52.123330    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-257940-997df79b8-cbp9t" podUID="354203bc-be04-4217-bc83-62afe2794986" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.67:9004: i/o timeout"
Jul 24 18:11:13.215540 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.215508    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh"]
Jul 24 18:11:13.216049 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.215721    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh" podUID="de410ff3-0656-41cf-b7f1-59f0a804d53c" containerName="maas-api" containerID="cri-o://ab52f4b672b98b25fa321642599a08bb649761d519501ab6f2772b0435e28702" gracePeriod=30
Jul 24 18:11:13.272227 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.272191    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt"]
Jul 24 18:11:13.272483 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.272455    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt" podUID="f0acefe2-8fbe-4532-994b-f3ee3ec39082" containerName="payload-processing" containerID="cri-o://d9e08f0af990ec023fa759cc3a3566c8292f631c98e2594bebc66c4fe661923f" gracePeriod=30
Jul 24 18:11:13.281283 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.281254    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v"]
Jul 24 18:11:13.281575 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.281503    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v" podUID="5138e275-cd12-43ca-b492-68527f02052e" containerName="payload-pre-processing" containerID="cri-o://3ce236da0267658dfa1d2df13ca86a501234908f4162f7b571d1c7525949b07c" gracePeriod=30
Jul 24 18:11:13.596740 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.596705    2562 generic.go:356] "Generic (PLEG): container finished" podID="f0acefe2-8fbe-4532-994b-f3ee3ec39082" containerID="d9e08f0af990ec023fa759cc3a3566c8292f631c98e2594bebc66c4fe661923f" exitCode=0
Jul 24 18:11:13.596890 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.596776    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt" event={"ID":"f0acefe2-8fbe-4532-994b-f3ee3ec39082","Type":"ContainerDied","Data":"d9e08f0af990ec023fa759cc3a3566c8292f631c98e2594bebc66c4fe661923f"}
Jul 24 18:11:13.598220 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.598194    2562 generic.go:356] "Generic (PLEG): container finished" podID="de410ff3-0656-41cf-b7f1-59f0a804d53c" containerID="ab52f4b672b98b25fa321642599a08bb649761d519501ab6f2772b0435e28702" exitCode=0
Jul 24 18:11:13.598325 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.598227    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh" event={"ID":"de410ff3-0656-41cf-b7f1-59f0a804d53c","Type":"ContainerDied","Data":"ab52f4b672b98b25fa321642599a08bb649761d519501ab6f2772b0435e28702"}
Jul 24 18:11:13.598325 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.598271    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh" event={"ID":"de410ff3-0656-41cf-b7f1-59f0a804d53c","Type":"ContainerDied","Data":"8a8a2b067149da34d4d777ca0f425df408a894a1c9d6ea43e54ad5d914efb613"}
Jul 24 18:11:13.598325 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.598295    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a8a2b067149da34d4d777ca0f425df408a894a1c9d6ea43e54ad5d914efb613"
Jul 24 18:11:13.599578 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.599550    2562 generic.go:356] "Generic (PLEG): container finished" podID="5138e275-cd12-43ca-b492-68527f02052e" containerID="3ce236da0267658dfa1d2df13ca86a501234908f4162f7b571d1c7525949b07c" exitCode=0
Jul 24 18:11:13.599578 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.599569    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v" event={"ID":"5138e275-cd12-43ca-b492-68527f02052e","Type":"ContainerDied","Data":"3ce236da0267658dfa1d2df13ca86a501234908f4162f7b571d1c7525949b07c"}
Jul 24 18:11:13.599739 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.599613    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v" event={"ID":"5138e275-cd12-43ca-b492-68527f02052e","Type":"ContainerDied","Data":"c5fcdc8670ea5203fbeafc7ff59f1892a37de098236de0455af2bdc95b58aec1"}
Jul 24 18:11:13.599739 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.599630    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c5fcdc8670ea5203fbeafc7ff59f1892a37de098236de0455af2bdc95b58aec1"
Jul 24 18:11:13.603264 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.603188    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v"
Jul 24 18:11:13.606625 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.606605    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh"
Jul 24 18:11:13.610500 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.610482    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt"
Jul 24 18:11:13.685416 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.685380    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de410ff3-0656-41cf-b7f1-59f0a804d53c-openshift-service-ca\") pod \"de410ff3-0656-41cf-b7f1-59f0a804d53c\" (UID: \"de410ff3-0656-41cf-b7f1-59f0a804d53c\") "
Jul 24 18:11:13.685618 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.685478    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4zt6\" (UniqueName: \"kubernetes.io/projected/f0acefe2-8fbe-4532-994b-f3ee3ec39082-kube-api-access-r4zt6\") pod \"f0acefe2-8fbe-4532-994b-f3ee3ec39082\" (UID: \"f0acefe2-8fbe-4532-994b-f3ee3ec39082\") "
Jul 24 18:11:13.685618 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.685520    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0acefe2-8fbe-4532-994b-f3ee3ec39082-plugins-config-volume\") pod \"f0acefe2-8fbe-4532-994b-f3ee3ec39082\" (UID: \"f0acefe2-8fbe-4532-994b-f3ee3ec39082\") "
Jul 24 18:11:13.685618 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.685559    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5tz8r\" (UniqueName: \"kubernetes.io/projected/5138e275-cd12-43ca-b492-68527f02052e-kube-api-access-5tz8r\") pod \"5138e275-cd12-43ca-b492-68527f02052e\" (UID: \"5138e275-cd12-43ca-b492-68527f02052e\") "
Jul 24 18:11:13.685618 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.685600    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5138e275-cd12-43ca-b492-68527f02052e-plugins-config-volume\") pod \"5138e275-cd12-43ca-b492-68527f02052e\" (UID: \"5138e275-cd12-43ca-b492-68527f02052e\") "
Jul 24 18:11:13.685830 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.685648    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlkmz\" (UniqueName: \"kubernetes.io/projected/de410ff3-0656-41cf-b7f1-59f0a804d53c-kube-api-access-tlkmz\") pod \"de410ff3-0656-41cf-b7f1-59f0a804d53c\" (UID: \"de410ff3-0656-41cf-b7f1-59f0a804d53c\") "
Jul 24 18:11:13.685830 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.685696    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de410ff3-0656-41cf-b7f1-59f0a804d53c-maas-api-tls\") pod \"de410ff3-0656-41cf-b7f1-59f0a804d53c\" (UID: \"de410ff3-0656-41cf-b7f1-59f0a804d53c\") "
Jul 24 18:11:13.685830 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.685800    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de410ff3-0656-41cf-b7f1-59f0a804d53c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "de410ff3-0656-41cf-b7f1-59f0a804d53c" (UID: "de410ff3-0656-41cf-b7f1-59f0a804d53c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:11:13.685984 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.685965    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0acefe2-8fbe-4532-994b-f3ee3ec39082-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f0acefe2-8fbe-4532-994b-f3ee3ec39082" (UID: "f0acefe2-8fbe-4532-994b-f3ee3ec39082"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:11:13.686107 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.686071    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5138e275-cd12-43ca-b492-68527f02052e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5138e275-cd12-43ca-b492-68527f02052e" (UID: "5138e275-cd12-43ca-b492-68527f02052e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:11:13.686271 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.686183    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0acefe2-8fbe-4532-994b-f3ee3ec39082-plugins-config-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:11:13.686271 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.686208    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5138e275-cd12-43ca-b492-68527f02052e-plugins-config-volume\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:11:13.686271 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.686223    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de410ff3-0656-41cf-b7f1-59f0a804d53c-openshift-service-ca\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:11:13.687897 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.687872    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de410ff3-0656-41cf-b7f1-59f0a804d53c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "de410ff3-0656-41cf-b7f1-59f0a804d53c" (UID: "de410ff3-0656-41cf-b7f1-59f0a804d53c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 18:11:13.688000 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.687889    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de410ff3-0656-41cf-b7f1-59f0a804d53c-kube-api-access-tlkmz" (OuterVolumeSpecName: "kube-api-access-tlkmz") pod "de410ff3-0656-41cf-b7f1-59f0a804d53c" (UID: "de410ff3-0656-41cf-b7f1-59f0a804d53c"). InnerVolumeSpecName "kube-api-access-tlkmz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:11:13.688135 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.688118    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0acefe2-8fbe-4532-994b-f3ee3ec39082-kube-api-access-r4zt6" (OuterVolumeSpecName: "kube-api-access-r4zt6") pod "f0acefe2-8fbe-4532-994b-f3ee3ec39082" (UID: "f0acefe2-8fbe-4532-994b-f3ee3ec39082"). InnerVolumeSpecName "kube-api-access-r4zt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:11:13.688212 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.688131    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5138e275-cd12-43ca-b492-68527f02052e-kube-api-access-5tz8r" (OuterVolumeSpecName: "kube-api-access-5tz8r") pod "5138e275-cd12-43ca-b492-68527f02052e" (UID: "5138e275-cd12-43ca-b492-68527f02052e"). InnerVolumeSpecName "kube-api-access-5tz8r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:11:13.787424 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.787338    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4zt6\" (UniqueName: \"kubernetes.io/projected/f0acefe2-8fbe-4532-994b-f3ee3ec39082-kube-api-access-r4zt6\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:11:13.787424 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.787370    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tz8r\" (UniqueName: \"kubernetes.io/projected/5138e275-cd12-43ca-b492-68527f02052e-kube-api-access-5tz8r\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:11:13.787424 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.787380    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlkmz\" (UniqueName: \"kubernetes.io/projected/de410ff3-0656-41cf-b7f1-59f0a804d53c-kube-api-access-tlkmz\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:11:13.787424 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:13.787391    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de410ff3-0656-41cf-b7f1-59f0a804d53c-maas-api-tls\") on node \"ip-10-0-131-159.ec2.internal\" DevicePath \"\""
Jul 24 18:11:14.612215 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:14.612176    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt"
Jul 24 18:11:14.612708 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:14.612183    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt" event={"ID":"f0acefe2-8fbe-4532-994b-f3ee3ec39082","Type":"ContainerDied","Data":"4115ea717d3201ec91bd68318629039feed9b4cd5138a84ef1dfac4931fd2dd5"}
Jul 24 18:11:14.612708 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:14.612360    2562 scope.go:117] "RemoveContainer" containerID="d9e08f0af990ec023fa759cc3a3566c8292f631c98e2594bebc66c4fe661923f"
Jul 24 18:11:14.612708 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:14.612449    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh"
Jul 24 18:11:14.612708 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:14.612590    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v"
Jul 24 18:11:14.638441 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:14.638355    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v"]
Jul 24 18:11:14.640622 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:14.640594    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-06ffdb-65977549cb-2776v"]
Jul 24 18:11:14.651387 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:14.651362    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh"]
Jul 24 18:11:14.656469 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:14.656447    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-06ffdb-6cb7cb8bd6-7lfzh"]
Jul 24 18:11:14.670523 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:14.670490    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt"]
Jul 24 18:11:14.675192 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:14.675168    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-06ffdb-b59f8759d-lkwpt"]
Jul 24 18:11:15.824763 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:15.824732    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5138e275-cd12-43ca-b492-68527f02052e" path="/var/lib/kubelet/pods/5138e275-cd12-43ca-b492-68527f02052e/volumes"
Jul 24 18:11:15.825171 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:15.825117    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de410ff3-0656-41cf-b7f1-59f0a804d53c" path="/var/lib/kubelet/pods/de410ff3-0656-41cf-b7f1-59f0a804d53c/volumes"
Jul 24 18:11:15.825463 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:15.825449    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0acefe2-8fbe-4532-994b-f3ee3ec39082" path="/var/lib/kubelet/pods/f0acefe2-8fbe-4532-994b-f3ee3ec39082/volumes"
Jul 24 18:11:18.191691 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:18.191657    2562 scope.go:117] "RemoveContainer" containerID="3ce236da0267658dfa1d2df13ca86a501234908f4162f7b571d1c7525949b07c"
Jul 24 18:11:18.199910 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:18.199889    2562 scope.go:117] "RemoveContainer" containerID="ab52f4b672b98b25fa321642599a08bb649761d519501ab6f2772b0435e28702"
Jul 24 18:11:31.746866 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:31.746837    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5ff575bb75-g5kzh_f1cf923e-bc12-4e8e-aaf5-0b0e78243975/manager/0.log"
Jul 24 18:11:32.405876 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:32.405846    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8574785b6-rzf89_43188074-d619-4727-959f-b834ccf6393e/copy-manifests/0.log"
Jul 24 18:11:32.414005 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:32.413973    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8574785b6-rzf89_43188074-d619-4727-959f-b834ccf6393e/manager/0.log"
Jul 24 18:11:32.813841 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:32.813752    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-78bb94d85f-94pdg_f26a81fb-73fe-47cf-9c7e-f1a1c84eb905/maas-api/0.log"
Jul 24 18:11:35.411004 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:35.410969    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-45xlb_b7cfe6b0-acc6-47c6-8da0-a2d0463dee29/manager/0.log"
Jul 24 18:11:36.234205 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:36.234168    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-4ngvv_3f147b19-7e7d-48c3-81a1-b12bcefee1e1/discovery/0.log"
Jul 24 18:11:36.760831 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:36.760805    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-dqxnq_ae3f4133-08e5-429c-8122-a6fb03ac5bba/payload-pre-processing/0.log"
Jul 24 18:11:36.914977 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:36.914946    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-d2cqc_bf7f97e0-b594-4228-a7c6-197df6a2b4d7/payload-processing/0.log"
Jul 24 18:11:37.711404 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:37.711374    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-lxlwd_2feb1d79-5ba4-4bcf-8887-f1b664e7085a/main/0.log"
Jul 24 18:11:37.968305 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:37.968215    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-cxztg_6900cebf-8424-419b-94ad-1c631161e1bd/main/0.log"
Jul 24 18:11:47.020783 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:47.020750    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ts5kp_602851eb-593f-4245-9365-97ef80e3eb51/global-pull-secret-syncer/0.log"
Jul 24 18:11:47.128542 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:47.128502    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-z2whv_1a0c97c1-3fee-45af-bc53-36991e313aa8/konnectivity-agent/0.log"
Jul 24 18:11:47.240215 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:47.240181    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-159.ec2.internal_d4dd242ec3a9586b4b7071e62b5c988d/haproxy/0.log"
Jul 24 18:11:52.696398 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:52.696357    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-45xlb_b7cfe6b0-acc6-47c6-8da0-a2d0463dee29/manager/0.log"
Jul 24 18:11:54.584770 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:54.584739    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e/alertmanager/0.log"
Jul 24 18:11:54.616003 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:54.615976    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e/config-reloader/0.log"
Jul 24 18:11:54.655911 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:54.655880    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e/kube-rbac-proxy-web/0.log"
Jul 24 18:11:54.684332 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:54.684308    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e/kube-rbac-proxy/0.log"
Jul 24 18:11:54.708407 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:54.708376    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e/kube-rbac-proxy-metric/0.log"
Jul 24 18:11:54.733734 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:54.733705    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e/prom-label-proxy/0.log"
Jul 24 18:11:54.757886 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:54.757858    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b81cc4cf-8821-4ca7-8d3a-dbf14e46aa9e/init-config-reloader/0.log"
Jul 24 18:11:54.849433 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:54.849402    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-54mj7_8d189003-36d8-41e7-b7be-96a055c05f86/kube-state-metrics/0.log"
Jul 24 18:11:54.873404 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:54.873369    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-54mj7_8d189003-36d8-41e7-b7be-96a055c05f86/kube-rbac-proxy-main/0.log"
Jul 24 18:11:54.896623 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:54.896596    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-54mj7_8d189003-36d8-41e7-b7be-96a055c05f86/kube-rbac-proxy-self/0.log"
Jul 24 18:11:55.168621 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:55.168521    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fxdjb_721fc7de-0768-4944-8f6b-8f76b09511f9/node-exporter/0.log"
Jul 24 18:11:55.193273 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:55.193242    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fxdjb_721fc7de-0768-4944-8f6b-8f76b09511f9/kube-rbac-proxy/0.log"
Jul 24 18:11:55.216040 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:55.216008    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fxdjb_721fc7de-0768-4944-8f6b-8f76b09511f9/init-textfile/0.log"
Jul 24 18:11:55.343319 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:55.343287    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07/prometheus/0.log"
Jul 24 18:11:55.368579 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:55.368549    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07/config-reloader/0.log"
Jul 24 18:11:55.393273 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:55.393241    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07/thanos-sidecar/0.log"
Jul 24 18:11:55.422635 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:55.422573    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07/kube-rbac-proxy-web/0.log"
Jul 24 18:11:55.449993 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:55.449959    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07/kube-rbac-proxy/0.log"
Jul 24 18:11:55.474973 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:55.474947    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07/kube-rbac-proxy-thanos/0.log"
Jul 24 18:11:55.503043 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:55.503015    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2bd02c4c-3a3d-460e-8d8f-2fc00ad2fe07/init-config-reloader/0.log"
Jul 24 18:11:55.590304 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:55.590273    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-92hzw_b158ac16-fd79-4e1d-8cc3-7bdb970f49c1/prometheus-operator-admission-webhook/0.log"
Jul 24 18:11:55.723484 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:55.723394    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84bb58cf8-l5sbx_667d4e75-26be-4bfd-b20a-213dca25e280/thanos-query/0.log"
Jul 24 18:11:55.749766 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:55.749733    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84bb58cf8-l5sbx_667d4e75-26be-4bfd-b20a-213dca25e280/kube-rbac-proxy-web/0.log"
Jul 24 18:11:55.778553 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:55.778530    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84bb58cf8-l5sbx_667d4e75-26be-4bfd-b20a-213dca25e280/kube-rbac-proxy/0.log"
Jul 24 18:11:55.800135 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:55.800111    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84bb58cf8-l5sbx_667d4e75-26be-4bfd-b20a-213dca25e280/prom-label-proxy/0.log"
Jul 24 18:11:55.823621 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:55.823595    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84bb58cf8-l5sbx_667d4e75-26be-4bfd-b20a-213dca25e280/kube-rbac-proxy-rules/0.log"
Jul 24 18:11:55.859955 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:55.859926    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84bb58cf8-l5sbx_667d4e75-26be-4bfd-b20a-213dca25e280/kube-rbac-proxy-metrics/0.log"
Jul 24 18:11:56.988137 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988104    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xkcdq/perf-node-gather-daemonset-b9gdh"]
Jul 24 18:11:56.988538 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988523    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0acefe2-8fbe-4532-994b-f3ee3ec39082" containerName="payload-processing"
Jul 24 18:11:56.988538 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988536    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0acefe2-8fbe-4532-994b-f3ee3ec39082" containerName="payload-processing"
Jul 24 18:11:56.988610 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988546    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ff72e08-8a17-4dbe-996f-342c7db1c582" containerName="maas-api"
Jul 24 18:11:56.988610 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988551    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ff72e08-8a17-4dbe-996f-342c7db1c582" containerName="maas-api"
Jul 24 18:11:56.988610 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988558    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a94269f-5227-47b2-9a5b-acbb6d8889f8" containerName="maas-api"
Jul 24 18:11:56.988610 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988565    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a94269f-5227-47b2-9a5b-acbb6d8889f8" containerName="maas-api"
Jul 24 18:11:56.988610 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988574    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c161bae-9149-4cc8-8509-306f0cf43386" containerName="maas-api"
Jul 24 18:11:56.988610 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988579    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c161bae-9149-4cc8-8509-306f0cf43386" containerName="maas-api"
Jul 24 18:11:56.988610 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988588    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de410ff3-0656-41cf-b7f1-59f0a804d53c" containerName="maas-api"
Jul 24 18:11:56.988610 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988593    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="de410ff3-0656-41cf-b7f1-59f0a804d53c" containerName="maas-api"
Jul 24 18:11:56.988610 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988599    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e02e8b9-848a-4bc4-8910-5b37025b0839" containerName="payload-pre-processing"
Jul 24 18:11:56.988610 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988604    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e02e8b9-848a-4bc4-8910-5b37025b0839" containerName="payload-pre-processing"
Jul 24 18:11:56.988610 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988612    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb781198-a4b8-4591-8ac8-0067068e1ee6" containerName="maas-api"
Jul 24 18:11:56.988986 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988617    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb781198-a4b8-4591-8ac8-0067068e1ee6" containerName="maas-api"
Jul 24 18:11:56.988986 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988624    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5138e275-cd12-43ca-b492-68527f02052e" containerName="payload-pre-processing"
Jul 24 18:11:56.988986 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988630    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5138e275-cd12-43ca-b492-68527f02052e" containerName="payload-pre-processing"
Jul 24 18:11:56.988986 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988637    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="354203bc-be04-4217-bc83-62afe2794986" containerName="payload-pre-processing"
Jul 24 18:11:56.988986 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988643    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="354203bc-be04-4217-bc83-62afe2794986" containerName="payload-pre-processing"
Jul 24 18:11:56.988986 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988651    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c1019c8-f3e9-4c39-8836-14ca62dde1e4" containerName="payload-pre-processing"
Jul 24 18:11:56.988986 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988656    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c1019c8-f3e9-4c39-8836-14ca62dde1e4" containerName="payload-pre-processing"
Jul 24 18:11:56.988986 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988665    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="923f1d57-bfe8-46bf-9b1d-bb9c34ebb177" containerName="payload-processing"
Jul 24 18:11:56.988986 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988671    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="923f1d57-bfe8-46bf-9b1d-bb9c34ebb177" containerName="payload-processing"
Jul 24 18:11:56.988986 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988679    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f42fa39-8173-4ec5-af8e-915ae82ead45" containerName="payload-processing"
Jul 24 18:11:56.988986 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988685    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f42fa39-8173-4ec5-af8e-915ae82ead45" containerName="payload-processing"
Jul 24 18:11:56.988986 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988742    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb781198-a4b8-4591-8ac8-0067068e1ee6" containerName="maas-api"
Jul 24 18:11:56.988986 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988749    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f42fa39-8173-4ec5-af8e-915ae82ead45" containerName="payload-processing"
Jul 24 18:11:56.988986 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988755    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="923f1d57-bfe8-46bf-9b1d-bb9c34ebb177" containerName="payload-processing"
Jul 24 18:11:56.988986 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988761    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a94269f-5227-47b2-9a5b-acbb6d8889f8" containerName="maas-api"
Jul 24 18:11:56.988986 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988768    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="354203bc-be04-4217-bc83-62afe2794986" containerName="payload-pre-processing"
Jul 24 18:11:56.988986 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988773    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e02e8b9-848a-4bc4-8910-5b37025b0839" containerName="payload-pre-processing"
Jul 24 18:11:56.988986 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988780    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0acefe2-8fbe-4532-994b-f3ee3ec39082" containerName="payload-processing"
Jul 24 18:11:56.988986 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988784    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="de410ff3-0656-41cf-b7f1-59f0a804d53c" containerName="maas-api"
Jul 24 18:11:56.988986 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988790    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c1019c8-f3e9-4c39-8836-14ca62dde1e4" containerName="payload-pre-processing"
Jul 24 18:11:56.988986 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988794    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c161bae-9149-4cc8-8509-306f0cf43386" containerName="maas-api"
Jul 24 18:11:56.988986 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988799    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5138e275-cd12-43ca-b492-68527f02052e" containerName="payload-pre-processing"
Jul 24 18:11:56.988986 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.988804    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ff72e08-8a17-4dbe-996f-342c7db1c582" containerName="maas-api"
Jul 24 18:11:56.992365 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.992349    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xkcdq/perf-node-gather-daemonset-b9gdh"
Jul 24 18:11:56.995437 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.995411    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xkcdq\"/\"openshift-service-ca.crt\""
Jul 24 18:11:56.996301 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.996285    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xkcdq\"/\"default-dockercfg-5k9qs\""
Jul 24 18:11:56.996401 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:56.996341    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xkcdq\"/\"kube-root-ca.crt\""
Jul 24 18:11:57.003490 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:57.003469    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xkcdq/perf-node-gather-daemonset-b9gdh"]
Jul 24 18:11:57.080157 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:57.080119    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-4cmjc_f3d1e52c-502d-496f-bafd-2fcee624d002/networking-console-plugin/0.log"
Jul 24 18:11:57.087518 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:57.087492    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fd105a8-b3b7-4069-8f53-c14badc3871c-lib-modules\") pod \"perf-node-gather-daemonset-b9gdh\" (UID: \"4fd105a8-b3b7-4069-8f53-c14badc3871c\") " pod="openshift-must-gather-xkcdq/perf-node-gather-daemonset-b9gdh"
Jul 24 18:11:57.087599 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:57.087524    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5xsm\" (UniqueName: \"kubernetes.io/projected/4fd105a8-b3b7-4069-8f53-c14badc3871c-kube-api-access-x5xsm\") pod \"perf-node-gather-daemonset-b9gdh\" (UID: \"4fd105a8-b3b7-4069-8f53-c14badc3871c\") " pod="openshift-must-gather-xkcdq/perf-node-gather-daemonset-b9gdh"
Jul 24 18:11:57.087599 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:57.087548    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4fd105a8-b3b7-4069-8f53-c14badc3871c-proc\") pod \"perf-node-gather-daemonset-b9gdh\" (UID: \"4fd105a8-b3b7-4069-8f53-c14badc3871c\") " pod="openshift-must-gather-xkcdq/perf-node-gather-daemonset-b9gdh"
Jul 24 18:11:57.087599 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:57.087595    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fd105a8-b3b7-4069-8f53-c14badc3871c-sys\") pod \"perf-node-gather-daemonset-b9gdh\" (UID: \"4fd105a8-b3b7-4069-8f53-c14badc3871c\") " pod="openshift-must-gather-xkcdq/perf-node-gather-daemonset-b9gdh"
Jul 24 18:11:57.087700 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:57.087611    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4fd105a8-b3b7-4069-8f53-c14badc3871c-podres\") pod \"perf-node-gather-daemonset-b9gdh\" (UID: \"4fd105a8-b3b7-4069-8f53-c14badc3871c\") " pod="openshift-must-gather-xkcdq/perf-node-gather-daemonset-b9gdh"
Jul 24 18:11:57.188680 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:57.188649    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fd105a8-b3b7-4069-8f53-c14badc3871c-lib-modules\") pod \"perf-node-gather-daemonset-b9gdh\" (UID: \"4fd105a8-b3b7-4069-8f53-c14badc3871c\") " pod="openshift-must-gather-xkcdq/perf-node-gather-daemonset-b9gdh"
Jul 24 18:11:57.188680 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:57.188680    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5xsm\" (UniqueName: \"kubernetes.io/projected/4fd105a8-b3b7-4069-8f53-c14badc3871c-kube-api-access-x5xsm\") pod \"perf-node-gather-daemonset-b9gdh\" (UID: \"4fd105a8-b3b7-4069-8f53-c14badc3871c\") " pod="openshift-must-gather-xkcdq/perf-node-gather-daemonset-b9gdh"
Jul 24 18:11:57.188864 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:57.188701    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4fd105a8-b3b7-4069-8f53-c14badc3871c-proc\") pod \"perf-node-gather-daemonset-b9gdh\" (UID: \"4fd105a8-b3b7-4069-8f53-c14badc3871c\") " pod="openshift-must-gather-xkcdq/perf-node-gather-daemonset-b9gdh"
Jul 24 18:11:57.188864 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:57.188742    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fd105a8-b3b7-4069-8f53-c14badc3871c-sys\") pod \"perf-node-gather-daemonset-b9gdh\" (UID: \"4fd105a8-b3b7-4069-8f53-c14badc3871c\") " pod="openshift-must-gather-xkcdq/perf-node-gather-daemonset-b9gdh"
Jul 24 18:11:57.188864 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:57.188768    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4fd105a8-b3b7-4069-8f53-c14badc3871c-podres\") pod \"perf-node-gather-daemonset-b9gdh\" (UID: \"4fd105a8-b3b7-4069-8f53-c14badc3871c\") " pod="openshift-must-gather-xkcdq/perf-node-gather-daemonset-b9gdh"
Jul 24 18:11:57.188864 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:57.188842    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4fd105a8-b3b7-4069-8f53-c14badc3871c-proc\") pod \"perf-node-gather-daemonset-b9gdh\" (UID: \"4fd105a8-b3b7-4069-8f53-c14badc3871c\") " pod="openshift-must-gather-xkcdq/perf-node-gather-daemonset-b9gdh"
Jul 24 18:11:57.188864 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:57.188859    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fd105a8-b3b7-4069-8f53-c14badc3871c-sys\") pod \"perf-node-gather-daemonset-b9gdh\" (UID: \"4fd105a8-b3b7-4069-8f53-c14badc3871c\") " pod="openshift-must-gather-xkcdq/perf-node-gather-daemonset-b9gdh"
Jul 24 18:11:57.189057 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:57.188912    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4fd105a8-b3b7-4069-8f53-c14badc3871c-podres\") pod \"perf-node-gather-daemonset-b9gdh\" (UID: \"4fd105a8-b3b7-4069-8f53-c14badc3871c\") " pod="openshift-must-gather-xkcdq/perf-node-gather-daemonset-b9gdh"
Jul 24 18:11:57.189057 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:57.188950    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fd105a8-b3b7-4069-8f53-c14badc3871c-lib-modules\") pod \"perf-node-gather-daemonset-b9gdh\" (UID: \"4fd105a8-b3b7-4069-8f53-c14badc3871c\") " pod="openshift-must-gather-xkcdq/perf-node-gather-daemonset-b9gdh"
Jul 24 18:11:57.197968 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:57.197945    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5xsm\" (UniqueName: \"kubernetes.io/projected/4fd105a8-b3b7-4069-8f53-c14badc3871c-kube-api-access-x5xsm\") pod \"perf-node-gather-daemonset-b9gdh\" (UID: \"4fd105a8-b3b7-4069-8f53-c14badc3871c\") " pod="openshift-must-gather-xkcdq/perf-node-gather-daemonset-b9gdh"
Jul 24 18:11:57.303266 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:57.303183    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xkcdq/perf-node-gather-daemonset-b9gdh"
Jul 24 18:11:57.436026 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:57.435873    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xkcdq/perf-node-gather-daemonset-b9gdh"]
Jul 24 18:11:57.439068 ip-10-0-131-159 kubenswrapper[2562]: W0724 18:11:57.439024    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4fd105a8_b3b7_4069_8f53_c14badc3871c.slice/crio-aee6789a46a0a08c859b50103e4861ac7500af5755a11ce615d0def63eac020f WatchSource:0}: Error finding container aee6789a46a0a08c859b50103e4861ac7500af5755a11ce615d0def63eac020f: Status 404 returned error can't find the container with id aee6789a46a0a08c859b50103e4861ac7500af5755a11ce615d0def63eac020f
Jul 24 18:11:57.828209 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:57.828172    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xkcdq/perf-node-gather-daemonset-b9gdh" event={"ID":"4fd105a8-b3b7-4069-8f53-c14badc3871c","Type":"ContainerStarted","Data":"3d14574f909581b5f2e0ed3698de17ea32ef537a408858d0b7e05cb023413f4c"}
Jul 24 18:11:57.828395 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:57.828229    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xkcdq/perf-node-gather-daemonset-b9gdh" event={"ID":"4fd105a8-b3b7-4069-8f53-c14badc3871c","Type":"ContainerStarted","Data":"aee6789a46a0a08c859b50103e4861ac7500af5755a11ce615d0def63eac020f"}
Jul 24 18:11:57.828395 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:57.828298    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xkcdq/perf-node-gather-daemonset-b9gdh"
Jul 24 18:11:57.857229 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:57.857171    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xkcdq/perf-node-gather-daemonset-b9gdh" podStartSLOduration=1.857133146 podStartE2EDuration="1.857133146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 18:11:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:11:57.853570231 +0000 UTC m=+3280.580726445" watchObservedRunningTime="2026-07-24 18:11:57.857133146 +0000 UTC m=+3280.584289359"
Jul 24 18:11:58.421646 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:11:58.421607    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-9qb88_85259239-c35e-4c64-8cf7-e47f80c22ec7/download-server/0.log"
Jul 24 18:12:00.192626 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:00.192592    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bf5b8_4438603e-bddf-4e8f-979b-9f7a58c7e698/dns/0.log"
Jul 24 18:12:00.215364 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:00.215333    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bf5b8_4438603e-bddf-4e8f-979b-9f7a58c7e698/kube-rbac-proxy/0.log"
Jul 24 18:12:00.326760 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:00.326730    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-h2fn4_ab25ce71-a92b-4f56-b9df-2c2812e744ef/dns-node-resolver/0.log"
Jul 24 18:12:00.983428 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:00.983398    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tdnbn_44c623e2-f2fa-418d-986a-af9370891a35/node-ca/0.log"
Jul 24 18:12:02.182890 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:02.182851    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-4ngvv_3f147b19-7e7d-48c3-81a1-b12bcefee1e1/discovery/0.log"
Jul 24 18:12:02.396199 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:02.396166    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-dqxnq_ae3f4133-08e5-429c-8122-a6fb03ac5bba/payload-pre-processing/0.log"
Jul 24 18:12:02.438518 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:02.438430    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-d2cqc_bf7f97e0-b594-4228-a7c6-197df6a2b4d7/payload-processing/0.log"
Jul 24 18:12:03.285682 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:03.285657    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gwrrp_63522b79-132b-45c7-92c5-3bfbfa4c7b3a/serve-healthcheck-canary/0.log"
Jul 24 18:12:03.844090 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:03.844061    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xkcdq/perf-node-gather-daemonset-b9gdh"
Jul 24 18:12:03.931419 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:03.931386    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kp5nk_237e290b-e1f0-4685-b85f-a0cf55d124e5/kube-rbac-proxy/0.log"
Jul 24 18:12:03.953976 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:03.953955    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kp5nk_237e290b-e1f0-4685-b85f-a0cf55d124e5/exporter/0.log"
Jul 24 18:12:03.976703 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:03.976674    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kp5nk_237e290b-e1f0-4685-b85f-a0cf55d124e5/extractor/0.log"
Jul 24 18:12:06.855378 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:06.855340    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5ff575bb75-g5kzh_f1cf923e-bc12-4e8e-aaf5-0b0e78243975/manager/0.log"
Jul 24 18:12:07.200290 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:07.200265    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8574785b6-rzf89_43188074-d619-4727-959f-b834ccf6393e/manager/0.log"
Jul 24 18:12:07.229768 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:07.229745    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8574785b6-rzf89_43188074-d619-4727-959f-b834ccf6393e/copy-manifests/0.log"
Jul 24 18:12:08.327998 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:08.327964    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-4ngvv_3f147b19-7e7d-48c3-81a1-b12bcefee1e1/discovery/0.log"
Jul 24 18:12:08.515337 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:08.515251    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-dqxnq_ae3f4133-08e5-429c-8122-a6fb03ac5bba/payload-pre-processing/0.log"
Jul 24 18:12:08.549505 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:08.549465    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-d2cqc_bf7f97e0-b594-4228-a7c6-197df6a2b4d7/payload-processing/0.log"
Jul 24 18:12:08.698251 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:08.698222    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-cc7897f66-6ffhv_d68e6bb3-0b59-4ae7-ad1a-c8178e6fb582/manager/0.log"
Jul 24 18:12:08.758536 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:08.758506    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-g4zd7_05e12860-30d7-4839-b51e-4dff2dc438fc/openshift-lws-operator/0.log"
Jul 24 18:12:10.213827 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:10.213744    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-4ngvv_3f147b19-7e7d-48c3-81a1-b12bcefee1e1/discovery/0.log"
Jul 24 18:12:10.377496 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:10.377452    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-dqxnq_ae3f4133-08e5-429c-8122-a6fb03ac5bba/payload-pre-processing/0.log"
Jul 24 18:12:10.454204 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:10.454170    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-d2cqc_bf7f97e0-b594-4228-a7c6-197df6a2b4d7/payload-processing/0.log"
Jul 24 18:12:13.792723 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:13.792688    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-hbbxj_37a1af80-b8c7-4056-ab27-34a4aef0e70e/migrator/0.log"
Jul 24 18:12:13.814515 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:13.814485    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-hbbxj_37a1af80-b8c7-4056-ab27-34a4aef0e70e/graceful-termination/0.log"
Jul 24 18:12:15.215684 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:15.215653    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-45mn5_15f25938-a5da-432f-91f3-b40dae0912d3/kube-multus-additional-cni-plugins/0.log"
Jul 24 18:12:15.238799 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:15.238775    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-45mn5_15f25938-a5da-432f-91f3-b40dae0912d3/egress-router-binary-copy/0.log"
Jul 24 18:12:15.268259 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:15.268229    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-45mn5_15f25938-a5da-432f-91f3-b40dae0912d3/cni-plugins/0.log"
Jul 24 18:12:15.312355 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:15.312327    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-45mn5_15f25938-a5da-432f-91f3-b40dae0912d3/bond-cni-plugin/0.log"
Jul 24 18:12:15.348591 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:15.348567    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-45mn5_15f25938-a5da-432f-91f3-b40dae0912d3/routeoverride-cni/0.log"
Jul 24 18:12:15.391332 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:15.391304    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-45mn5_15f25938-a5da-432f-91f3-b40dae0912d3/whereabouts-cni-bincopy/0.log"
Jul 24 18:12:15.437609 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:15.437575    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-45mn5_15f25938-a5da-432f-91f3-b40dae0912d3/whereabouts-cni/0.log"
Jul 24 18:12:15.945551 ip-10-0-131-159 kubenswrapper[2562]: I0724 18:12:15.945522    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-m77tg_11f4dbdb-7967-407a-8572-8ba5dd91e233/kube-multus/0.log"