Jul 28 00:51:41.729868 ip-10-0-132-238 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 28 00:51:41.729880 ip-10-0-132-238 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 28 00:51:41.729887 ip-10-0-132-238 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 28 00:51:41.730104 ip-10-0-132-238 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 28 00:51:51.910679 ip-10-0-132-238 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 28 00:51:51.910703 ip-10-0-132-238 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot a232aa42d989402794293acb6776803a --
Jul 28 00:53:49.717519 ip-10-0-132-238 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 00:53:50.139013 ip-10-0-132-238 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 00:53:50.139013 ip-10-0-132-238 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 00:53:50.139013 ip-10-0-132-238 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 00:53:50.139013 ip-10-0-132-238 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 00:53:50.139013 ip-10-0-132-238 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 00:53:50.139756 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.139075    2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 00:53:50.145457 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145426    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 00:53:50.145457 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145448    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 00:53:50.145457 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145452    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 00:53:50.145457 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145456    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 00:53:50.145457 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145458    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 00:53:50.145457 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145463    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 00:53:50.145457 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145466    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 00:53:50.145457 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145469    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 00:53:50.145794 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145472    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 00:53:50.145794 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145474    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 00:53:50.145794 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145477    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 00:53:50.145794 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145480    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 00:53:50.145794 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145484    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 00:53:50.145794 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145489    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 00:53:50.145794 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145492    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 00:53:50.145794 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145496    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 00:53:50.145794 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145499    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 00:53:50.145794 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145502    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 00:53:50.145794 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145504    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 00:53:50.145794 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145507    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 00:53:50.145794 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145513    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 00:53:50.145794 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145516    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 00:53:50.145794 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145519    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 00:53:50.145794 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145521    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 00:53:50.145794 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145524    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 00:53:50.145794 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145527    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 00:53:50.145794 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145529    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 00:53:50.145794 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145532    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 00:53:50.146390 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145534    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 00:53:50.146390 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145537    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 00:53:50.146390 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145540    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 00:53:50.146390 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145542    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 00:53:50.146390 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145545    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 00:53:50.146390 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145547    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 00:53:50.146390 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145550    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 00:53:50.146390 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145553    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 00:53:50.146390 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145555    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 00:53:50.146390 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145558    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 00:53:50.146390 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145561    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 00:53:50.146390 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145564    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 00:53:50.146390 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145567    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 00:53:50.146390 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145569    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 00:53:50.146390 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145572    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 00:53:50.146390 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145576    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 00:53:50.146390 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145580    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 00:53:50.146390 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145585    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 00:53:50.146390 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145588    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 00:53:50.146873 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145591    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 00:53:50.146873 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145594    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 00:53:50.146873 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145597    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 00:53:50.146873 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145599    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 00:53:50.146873 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145602    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 00:53:50.146873 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145605    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 00:53:50.146873 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145608    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 00:53:50.146873 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145610    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 00:53:50.146873 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145613    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 00:53:50.146873 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145615    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 00:53:50.146873 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145618    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 00:53:50.146873 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145620    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 00:53:50.146873 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145623    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 00:53:50.146873 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145626    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 00:53:50.146873 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145628    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 00:53:50.146873 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145631    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 00:53:50.146873 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145633    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 00:53:50.146873 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145636    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 00:53:50.146873 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145638    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 00:53:50.146873 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145641    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 00:53:50.147372 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145643    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 00:53:50.147372 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145646    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 00:53:50.147372 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145649    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 00:53:50.147372 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145652    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 00:53:50.147372 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145655    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 00:53:50.147372 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145658    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 00:53:50.147372 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145660    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 00:53:50.147372 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145663    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 00:53:50.147372 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145668    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 00:53:50.147372 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145670    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 00:53:50.147372 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145672    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 00:53:50.147372 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145675    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 00:53:50.147372 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145678    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 00:53:50.147372 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145680    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 00:53:50.147372 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145683    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 00:53:50.147372 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145685    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 00:53:50.147372 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145689    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 00:53:50.147372 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145691    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 00:53:50.147372 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145694    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 00:53:50.147866 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145696    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 00:53:50.147866 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145699    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 00:53:50.147866 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145702    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 00:53:50.147866 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145704    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 00:53:50.147866 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145707    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 00:53:50.147866 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145709    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 00:53:50.147866 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145712    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 00:53:50.147866 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145716    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 00:53:50.147866 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145719    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 00:53:50.147866 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145722    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 00:53:50.147866 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145726    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 00:53:50.147866 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145729    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 00:53:50.147866 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.145732    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 00:53:50.147866 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146493    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 00:53:50.147866 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146501    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 00:53:50.147866 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146504    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 00:53:50.147866 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146507    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 00:53:50.147866 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146510    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 00:53:50.147866 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146513    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 00:53:50.148348 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146516    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 00:53:50.148348 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146519    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 00:53:50.148348 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146521    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 00:53:50.148348 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146524    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 00:53:50.148348 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146526    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 00:53:50.148348 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146532    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 00:53:50.148348 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146535    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 00:53:50.148348 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146538    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 00:53:50.148348 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146541    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 00:53:50.148348 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146544    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 00:53:50.148348 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146547    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 00:53:50.148348 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146550    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 00:53:50.148348 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146553    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 00:53:50.148348 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146555    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 00:53:50.148348 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146558    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 00:53:50.148348 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146561    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 00:53:50.148348 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146564    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 00:53:50.148348 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146566    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 00:53:50.148348 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146569    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 00:53:50.148348 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146571    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 00:53:50.148838 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146574    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 00:53:50.148838 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146577    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 00:53:50.148838 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146580    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 00:53:50.148838 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146584    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 00:53:50.148838 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146589    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 00:53:50.148838 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146593    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 00:53:50.148838 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146595    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 00:53:50.148838 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146598    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 00:53:50.148838 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146601    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 00:53:50.148838 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146603    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 00:53:50.148838 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146606    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 00:53:50.148838 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146608    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 00:53:50.148838 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146611    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 00:53:50.148838 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146614    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 00:53:50.148838 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146616    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 00:53:50.148838 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146619    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 00:53:50.148838 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146621    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 00:53:50.148838 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146624    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 00:53:50.148838 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146627    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 00:53:50.148838 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146629    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 00:53:50.149358 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146632    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 00:53:50.149358 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146635    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 00:53:50.149358 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146637    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 00:53:50.149358 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146639    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 00:53:50.149358 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146643    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 00:53:50.149358 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146646    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 00:53:50.149358 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146649    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 00:53:50.149358 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146651    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 00:53:50.149358 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146654    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 00:53:50.149358 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146656    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 00:53:50.149358 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146659    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 00:53:50.149358 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146661    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 00:53:50.149358 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146664    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 00:53:50.149358 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146667    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 00:53:50.149358 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146670    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 00:53:50.149358 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146673    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 00:53:50.149358 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146676    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 00:53:50.149358 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146679    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 00:53:50.149358 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146681    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 00:53:50.149846 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146684    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 00:53:50.149846 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146687    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 00:53:50.149846 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146689    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 00:53:50.149846 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146692    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 00:53:50.149846 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146695    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 00:53:50.149846 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146697    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 00:53:50.149846 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146700    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 00:53:50.149846 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146702    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 00:53:50.149846 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146705    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 00:53:50.149846 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146707    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 00:53:50.149846 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146710    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 00:53:50.149846 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146712    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 00:53:50.149846 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146715    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 00:53:50.149846 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146717    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 00:53:50.149846 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146720    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 00:53:50.149846 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146722    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 00:53:50.149846 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146725    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 00:53:50.149846 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146727    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 00:53:50.149846 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146730    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 00:53:50.149846 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146733    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 00:53:50.150354 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146735    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 00:53:50.150354 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146737    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 00:53:50.150354 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146740    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 00:53:50.150354 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146743    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 00:53:50.150354 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146745    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 00:53:50.150354 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146747    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 00:53:50.150354 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146750    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 00:53:50.150354 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146752    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 00:53:50.150354 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146756    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 00:53:50.150354 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146760    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 00:53:50.150354 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146763    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 00:53:50.150354 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146766    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 00:53:50.150354 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146769    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 00:53:50.150354 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.146772    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 00:53:50.150354 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147181    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 00:53:50.150354 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147187    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 00:53:50.150354 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147190    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 00:53:50.150354 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147193    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 00:53:50.150354 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147196    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 00:53:50.150820 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147199    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 00:53:50.150820 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147202    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 00:53:50.150820 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147205    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 00:53:50.150820 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147225    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 00:53:50.150820 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147229    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 00:53:50.150820 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147233    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 00:53:50.150820 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147237    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 00:53:50.150820 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147241    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 00:53:50.150820 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147246    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 00:53:50.150820 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147249    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 00:53:50.150820 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147252    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 00:53:50.150820 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147255    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 00:53:50.150820 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147259    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 00:53:50.150820 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147261    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 00:53:50.150820 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147265    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 00:53:50.150820 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147267    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 00:53:50.150820 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147270    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 00:53:50.150820 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147272    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 00:53:50.150820 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147275    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 00:53:50.150820 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147278    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 00:53:50.151383 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147280    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 00:53:50.151383 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147283    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 00:53:50.151383 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147286    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 00:53:50.151383 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147289    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 00:53:50.151383 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147292    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 00:53:50.151383 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147295    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 00:53:50.151383 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147297    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 00:53:50.151383 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147300    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 00:53:50.151383 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147302    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 00:53:50.151383 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147306    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 00:53:50.151383 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147309    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 00:53:50.151383 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147312    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 00:53:50.151383 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147315    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 00:53:50.151383 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147317    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 00:53:50.151383 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147320    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 00:53:50.151383 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147323    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 00:53:50.151383 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147325    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 00:53:50.151383 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147327    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 00:53:50.151383 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147330    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 00:53:50.151383 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147332    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 00:53:50.151895 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147335    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 00:53:50.151895 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147337    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 00:53:50.151895 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147340    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 00:53:50.151895 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147343    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 00:53:50.151895 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147346    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 00:53:50.151895 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147349    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 00:53:50.151895 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147352    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 00:53:50.151895 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147354    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 00:53:50.151895 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147357    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 00:53:50.151895 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147359    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 00:53:50.151895 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147362    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 00:53:50.151895 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147364    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 00:53:50.151895 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147366    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 00:53:50.151895 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147370    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 00:53:50.151895 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147375    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 00:53:50.151895 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147378    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 00:53:50.151895 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147381    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 00:53:50.151895 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147384    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 00:53:50.151895 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147386    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 00:53:50.151895 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147389    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 00:53:50.152385 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147393    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 00:53:50.152385 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147397    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 00:53:50.152385 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147400    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 00:53:50.152385 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147403    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 00:53:50.152385 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147406    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 00:53:50.152385 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147409    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 00:53:50.152385 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147412    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 00:53:50.152385 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147417    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 00:53:50.152385 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147420    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 00:53:50.152385 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147423    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 00:53:50.152385 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147426    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 00:53:50.152385 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147428    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 00:53:50.152385 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147431    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 00:53:50.152385 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147433    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 00:53:50.152385 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147436    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 00:53:50.152385 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147439    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 00:53:50.152385 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147441    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 00:53:50.152385 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147444    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 00:53:50.152385 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147447    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 00:53:50.152865 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147450    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 00:53:50.152865 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147452    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 00:53:50.152865 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147455    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 00:53:50.152865 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147459    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 00:53:50.152865 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147462    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 00:53:50.152865 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147465    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 00:53:50.152865 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147469    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 00:53:50.152865 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147473    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 00:53:50.152865 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147476    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 00:53:50.152865 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147479    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 00:53:50.152865 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147481    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 00:53:50.152865 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147484    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 00:53:50.152865 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147486    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 00:53:50.152865 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147489    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 00:53:50.152865 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.147492    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 00:53:50.152865 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147568    2578 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 00:53:50.152865 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147575    2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 00:53:50.152865 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147581    2578 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 00:53:50.152865 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147587    2578 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 00:53:50.152865 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147594    2578 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 00:53:50.152865 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147599    2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 00:53:50.153393 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147605    2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 00:53:50.153393 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147612    2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 00:53:50.153393 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147615    2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 00:53:50.153393 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147618    2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 00:53:50.153393 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147622    2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 00:53:50.153393 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147625    2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 00:53:50.153393 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147629    2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 00:53:50.153393 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147632    2578 flags.go:64] FLAG: --cgroup-root=""
Jul 28 00:53:50.153393 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147634    2578 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 00:53:50.153393 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147637    2578 flags.go:64] FLAG: --client-ca-file=""
Jul 28 00:53:50.153393 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147641    2578 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 00:53:50.153393 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147645    2578 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 00:53:50.153393 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147650    2578 flags.go:64] FLAG: --cluster-domain=""
Jul 28 00:53:50.153393 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147653    2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 00:53:50.153393 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147656    2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 00:53:50.153393 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147660    2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 00:53:50.153393 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147663    2578 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 00:53:50.153393 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147668    2578 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 00:53:50.153393 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147672    2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 00:53:50.153393 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147676    2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 00:53:50.153393 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147679    2578 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 00:53:50.153393 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147682    2578 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 00:53:50.153393 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147685    2578 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 00:53:50.153393 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147688    2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 00:53:50.153997 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147691    2578 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 00:53:50.153997 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147694    2578 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 00:53:50.153997 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147698    2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 00:53:50.153997 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147702    2578 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 00:53:50.153997 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147704    2578 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 00:53:50.153997 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147708    2578 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 00:53:50.153997 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147711    2578 flags.go:64] FLAG: --enable-server="true"
Jul 28 00:53:50.153997 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147713    2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 00:53:50.153997 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147717    2578 flags.go:64] FLAG: --event-burst="100"
Jul 28 00:53:50.153997 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147721    2578 flags.go:64] FLAG: --event-qps="50"
Jul 28 00:53:50.153997 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147724    2578 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 00:53:50.153997 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147726    2578 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 00:53:50.153997 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147730    2578 flags.go:64] FLAG: --eviction-hard=""
Jul 28 00:53:50.153997 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147734    2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 00:53:50.153997 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147736    2578 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 00:53:50.153997 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147739    2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 00:53:50.153997 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147743    2578 flags.go:64] FLAG: --eviction-soft=""
Jul 28 00:53:50.153997 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147746    2578 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 00:53:50.153997 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147748    2578 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 00:53:50.153997 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147759    2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 00:53:50.153997 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147762    2578 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 00:53:50.153997 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147765    2578 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 00:53:50.153997 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147768    2578 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 00:53:50.153997 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147771    2578 flags.go:64] FLAG: --feature-gates=""
Jul 28 00:53:50.153997 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147775    2578 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 00:53:50.154659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147779    2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 00:53:50.154659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147783    2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 00:53:50.154659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147786    2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 00:53:50.154659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147789    2578 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 00:53:50.154659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147793    2578 flags.go:64] FLAG: --help="false"
Jul 28 00:53:50.154659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147796    2578 flags.go:64] FLAG: --hostname-override="ip-10-0-132-238.ec2.internal"
Jul 28 00:53:50.154659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147799    2578 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 00:53:50.154659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147802    2578 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 00:53:50.154659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147804    2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 00:53:50.154659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147808    2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 00:53:50.154659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147811    2578 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 00:53:50.154659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147814    2578 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 00:53:50.154659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147817    2578 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 00:53:50.154659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147820    2578 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 00:53:50.154659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147823    2578 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 00:53:50.154659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147826    2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 00:53:50.154659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147829    2578 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 00:53:50.154659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147832    2578 flags.go:64] FLAG: --kube-reserved=""
Jul 28 00:53:50.154659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147837    2578 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 00:53:50.154659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147839    2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 00:53:50.154659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147842    2578 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 00:53:50.154659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147845    2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 00:53:50.154659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147848    2578 flags.go:64] FLAG: --lock-file=""
Jul 28 00:53:50.154659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147851    2578 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 00:53:50.155283 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147853    2578 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 00:53:50.155283 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147856    2578 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 00:53:50.155283 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147861    2578 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 00:53:50.155283 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147865    2578 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 00:53:50.155283 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147868    2578 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 00:53:50.155283 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147871    2578 flags.go:64] FLAG: --logging-format="text"
Jul 28 00:53:50.155283 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147874    2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 00:53:50.155283 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147877    2578 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 00:53:50.155283 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147880    2578 flags.go:64] FLAG: --manifest-url=""
Jul 28 00:53:50.155283 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147884    2578 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 00:53:50.155283 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147889    2578 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 00:53:50.155283 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147892    2578 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 00:53:50.155283 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147896    2578 flags.go:64] FLAG: --max-pods="110"
Jul 28 00:53:50.155283 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147899    2578 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 00:53:50.155283 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147902    2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 00:53:50.155283 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147905    2578 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 00:53:50.155283 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147907    2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 00:53:50.155283 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147910    2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 00:53:50.155283 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147913    2578 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 00:53:50.155283 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147916    2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 00:53:50.155283 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147923    2578 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 00:53:50.155283 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147927    2578 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 00:53:50.155283 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147930    2578 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 00:53:50.155283 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147933    2578 flags.go:64] FLAG: --pod-cidr=""
Jul 28 00:53:50.155889 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147935    2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 00:53:50.155889 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147941    2578 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 00:53:50.155889 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147945    2578 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 00:53:50.155889 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147949    2578 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 00:53:50.155889 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147952    2578 flags.go:64] FLAG: --port="10250"
Jul 28 00:53:50.155889 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147955    2578 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 00:53:50.155889 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147958    2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fe86a320c91c2c7b"
Jul 28 00:53:50.155889 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147961    2578 flags.go:64] FLAG: --qos-reserved=""
Jul 28 00:53:50.155889 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147964    2578 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 00:53:50.155889 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147967    2578 flags.go:64] FLAG: --register-node="true"
Jul 28 00:53:50.155889 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147970    2578 flags.go:64] FLAG: --register-with-taints=""
Jul 28 00:53:50.155889 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147974    2578 flags.go:64] FLAG: --registry-burst="10"
Jul 28 00:53:50.155889 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147977    2578 flags.go:64] FLAG: --registry-qps="5"
Jul 28 00:53:50.155889 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147980    2578 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 00:53:50.155889 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147983    2578 flags.go:64] FLAG: --reserved-memory=""
Jul 28 00:53:50.155889 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147987    2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 00:53:50.155889 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147989    2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 00:53:50.155889 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147992    2578 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 00:53:50.155889 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.147997    2578 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 00:53:50.155889 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148000    2578 flags.go:64] FLAG: --runonce="false"
Jul 28 00:53:50.155889 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148003    2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 00:53:50.155889 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148006    2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 00:53:50.155889 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148009    2578 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 00:53:50.155889 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148012    2578 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 00:53:50.155889 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148015    2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 00:53:50.156554 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148018    2578 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 00:53:50.156554 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148021    2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 00:53:50.156554 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148024    2578 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 00:53:50.156554 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148026    2578 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 00:53:50.156554 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148029    2578 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 00:53:50.156554 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148032    2578 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 00:53:50.156554 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148035    2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 00:53:50.156554 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148038    2578 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 00:53:50.156554 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148041    2578 flags.go:64] FLAG: --system-cgroups=""
Jul 28 00:53:50.156554 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148045    2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 00:53:50.156554 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148050    2578 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 00:53:50.156554 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148053    2578 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 00:53:50.156554 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148056    2578 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 00:53:50.156554 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148060    2578 flags.go:64] FLAG: --tls-min-version=""
Jul 28 00:53:50.156554 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148063    2578 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 00:53:50.156554 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148066    2578 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 00:53:50.156554 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148069    2578 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 00:53:50.156554 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148072    2578 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 00:53:50.156554 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148075    2578 flags.go:64] FLAG: --v="2"
Jul 28 00:53:50.156554 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148079    2578 flags.go:64] FLAG: --version="false"
Jul 28 00:53:50.156554 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148084    2578 flags.go:64] FLAG: --vmodule=""
Jul 28 00:53:50.156554 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148088    2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 00:53:50.156554 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.148091    2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 00:53:50.156554 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148240    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 00:53:50.156554 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148244    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 00:53:50.157148 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148249    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 00:53:50.157148 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148251    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 00:53:50.157148 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148254    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 00:53:50.157148 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148256    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 00:53:50.157148 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148259    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 00:53:50.157148 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148262    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 00:53:50.157148 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148264    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 00:53:50.157148 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148267    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 00:53:50.157148 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148270    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 00:53:50.157148 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148272    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 00:53:50.157148 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148275    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 00:53:50.157148 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148279    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 00:53:50.157148 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148283    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 00:53:50.157148 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148286    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 00:53:50.157148 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148289    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 00:53:50.157148 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148292    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 00:53:50.157148 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148296    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 00:53:50.157148 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148298    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 00:53:50.157148 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148301    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 00:53:50.157664 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148304    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 00:53:50.157664 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148306    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 00:53:50.157664 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148308    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 00:53:50.157664 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148311    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 00:53:50.157664 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148314    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 00:53:50.157664 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148316    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 00:53:50.157664 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148319    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 00:53:50.157664 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148322    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 00:53:50.157664 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148324    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 00:53:50.157664 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148327    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 00:53:50.157664 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148331    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 00:53:50.157664 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148333    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 00:53:50.157664 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148336    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 00:53:50.157664 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148340    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 00:53:50.157664 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148343    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 00:53:50.157664 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148346    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 00:53:50.157664 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148348    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 00:53:50.157664 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148351    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 00:53:50.157664 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148354    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 00:53:50.157664 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148356    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 00:53:50.158155 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148360    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 00:53:50.158155 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148362    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 00:53:50.158155 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148365    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 00:53:50.158155 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148367    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 00:53:50.158155 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148370    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 00:53:50.158155 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148372    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 00:53:50.158155 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148375    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 00:53:50.158155 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148378    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 00:53:50.158155 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148380    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 00:53:50.158155 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148384    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 00:53:50.158155 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148387    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 00:53:50.158155 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148389    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 00:53:50.158155 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148392    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 00:53:50.158155 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148395    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 00:53:50.158155 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148397    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 00:53:50.158155 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148399    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 00:53:50.158155 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148402    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 00:53:50.158155 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148404    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 00:53:50.158155 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148407    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 00:53:50.158155 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148410    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 00:53:50.158668 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148412    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 00:53:50.158668 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148415    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 00:53:50.158668 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148418    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 00:53:50.158668 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148421    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 00:53:50.158668 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148423    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 00:53:50.158668 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148428    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 00:53:50.158668 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148430    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 00:53:50.158668 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148433    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 00:53:50.158668 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148435    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 00:53:50.158668 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148438    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 00:53:50.158668 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148441    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 00:53:50.158668 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148444    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 00:53:50.158668 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148448    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 00:53:50.158668 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148453    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 00:53:50.158668 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148455    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 00:53:50.158668 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148458    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 00:53:50.158668 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148461    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 00:53:50.158668 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148464    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 00:53:50.158668 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148466    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 00:53:50.159165 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148469    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 00:53:50.159165 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148472    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 00:53:50.159165 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148477    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 00:53:50.159165 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148480    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 00:53:50.159165 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148483    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 00:53:50.159165 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148485    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 00:53:50.159165 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148488    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 00:53:50.159165 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148490    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 00:53:50.159165 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148493    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 00:53:50.159165 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148496    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 00:53:50.159165 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148498    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 00:53:50.159165 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148501    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 00:53:50.159165 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148504    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 00:53:50.159165 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148506    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 00:53:50.159165 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148509    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 00:53:50.159165 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148512    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 00:53:50.159165 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148515    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 00:53:50.159165 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148517    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 00:53:50.159165 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.148521    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 00:53:50.159662 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.149116    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 00:53:50.159662 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.155914    2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 00:53:50.159662 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.155934    2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 00:53:50.159662 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.155951    2578 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 00:53:50.159662 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.155958    2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 00:53:50.159662 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156014    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 00:53:50.159662 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156019    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 00:53:50.159662 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156022    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 00:53:50.159662 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156025    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 00:53:50.159662 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156028    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 00:53:50.159662 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156032    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 00:53:50.159662 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156036    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 00:53:50.159662 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156040    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 00:53:50.159662 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156043    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 00:53:50.160008 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156046    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 00:53:50.160008 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156049    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 00:53:50.160008 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156051    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 00:53:50.160008 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156054    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 00:53:50.160008 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156057    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 00:53:50.160008 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156060    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 00:53:50.160008 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156062    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 00:53:50.160008 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156065    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 00:53:50.160008 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156067    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 00:53:50.160008 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156070    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 00:53:50.160008 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156073    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 00:53:50.160008 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156075    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 00:53:50.160008 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156078    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 00:53:50.160008 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156081    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 00:53:50.160008 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156084    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 00:53:50.160008 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156088    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 00:53:50.160008 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156091    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 00:53:50.160008 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156094    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 00:53:50.160008 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156097    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 00:53:50.160560 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156099    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 00:53:50.160560 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156103    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 00:53:50.160560 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156106    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 00:53:50.160560 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156109    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 00:53:50.160560 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156111    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 00:53:50.160560 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156114    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 00:53:50.160560 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156117    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 00:53:50.160560 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156119    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 00:53:50.160560 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156122    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 00:53:50.160560 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156124    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 00:53:50.160560 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156127    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 00:53:50.160560 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156130    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 00:53:50.160560 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156132    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 00:53:50.160560 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156134    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 00:53:50.160560 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156139    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 00:53:50.160560 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156142    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 00:53:50.160560 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156145    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 00:53:50.160560 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156149    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 00:53:50.160560 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156152    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 00:53:50.161034 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156155    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 00:53:50.161034 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156157    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 00:53:50.161034 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156160    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 00:53:50.161034 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156163    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 00:53:50.161034 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156165    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 00:53:50.161034 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156168    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 00:53:50.161034 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156171    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 00:53:50.161034 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156173    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 00:53:50.161034 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156176    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 00:53:50.161034 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156178    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 00:53:50.161034 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156181    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 00:53:50.161034 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156183    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 00:53:50.161034 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156186    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 00:53:50.161034 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156189    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 00:53:50.161034 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156192    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 00:53:50.161034 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156196    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 00:53:50.161034 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156198    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 00:53:50.161034 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156201    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 00:53:50.161034 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156203    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 00:53:50.161034 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156206    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 00:53:50.161538 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156222    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 00:53:50.161538 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156227    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 00:53:50.161538 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156230    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 00:53:50.161538 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156234    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 00:53:50.161538 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156237    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 00:53:50.161538 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156241    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 00:53:50.161538 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156244    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 00:53:50.161538 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156247    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 00:53:50.161538 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156250    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 00:53:50.161538 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156253    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 00:53:50.161538 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156255    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 00:53:50.161538 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156258    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 00:53:50.161538 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156260    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 00:53:50.161538 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156263    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 00:53:50.161538 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156266    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 00:53:50.161538 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156268    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 00:53:50.161538 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156271    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 00:53:50.161538 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156273    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 00:53:50.161538 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156276    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 00:53:50.161538 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156279    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 00:53:50.162026 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156282    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 00:53:50.162026 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156284    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 00:53:50.162026 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156287    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 00:53:50.162026 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156289    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 00:53:50.162026 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156292    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 00:53:50.162026 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156294    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 00:53:50.162026 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156298    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 00:53:50.162026 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156301    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 00:53:50.162026 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156304    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 00:53:50.162026 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156307    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 00:53:50.162026 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156309    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 00:53:50.162026 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156312    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 00:53:50.162026 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.156318    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 00:53:50.162026 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156471    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 00:53:50.162026 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156477    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 00:53:50.162418 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156480    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 00:53:50.162418 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156483    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 00:53:50.162418 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156486    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 00:53:50.162418 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156488    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 00:53:50.162418 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156491    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 00:53:50.162418 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156493    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 00:53:50.162418 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156496    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 00:53:50.162418 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156499    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 00:53:50.162418 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156501    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 00:53:50.162418 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156504    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 00:53:50.162418 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156506    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 00:53:50.162418 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156509    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 00:53:50.162418 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156512    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 00:53:50.162418 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156514    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 00:53:50.162418 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156517    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 00:53:50.162418 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156519    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 00:53:50.162418 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156522    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 00:53:50.162418 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156524    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 00:53:50.162418 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156527    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 00:53:50.162418 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156529    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 00:53:50.162934 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156532    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 00:53:50.162934 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156535    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 00:53:50.162934 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156538    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 00:53:50.162934 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156541    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 00:53:50.162934 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156544    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 00:53:50.162934 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156546    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 00:53:50.162934 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156550    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 00:53:50.162934 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156552    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 00:53:50.162934 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156555    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 00:53:50.162934 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156557    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 00:53:50.162934 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156560    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 00:53:50.162934 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156563    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 00:53:50.162934 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156566    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 00:53:50.162934 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156569    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 00:53:50.162934 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156572    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 00:53:50.162934 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156574    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 00:53:50.162934 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156577    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 00:53:50.162934 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156579    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 00:53:50.162934 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156582    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 00:53:50.163429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156584    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 00:53:50.163429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156587    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 00:53:50.163429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156589    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 00:53:50.163429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156592    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 00:53:50.163429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156595    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 00:53:50.163429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156597    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 00:53:50.163429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156600    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 00:53:50.163429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156603    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 00:53:50.163429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156606    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 00:53:50.163429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156609    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 00:53:50.163429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156611    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 00:53:50.163429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156614    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 00:53:50.163429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156616    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 00:53:50.163429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156619    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 00:53:50.163429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156621    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 00:53:50.163429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156624    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 00:53:50.163429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156626    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 00:53:50.163429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156629    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 00:53:50.163429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156632    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 00:53:50.163429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156634    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 00:53:50.163920 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156637    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 00:53:50.163920 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156640    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 00:53:50.163920 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156642    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 00:53:50.163920 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156645    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 00:53:50.163920 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156648    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 00:53:50.163920 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156652    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 00:53:50.163920 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156655    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 00:53:50.163920 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156658    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 00:53:50.163920 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156661    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 00:53:50.163920 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156664    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 00:53:50.163920 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156666    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 00:53:50.163920 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156669    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 00:53:50.163920 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156672    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 00:53:50.163920 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156674    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 00:53:50.163920 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156677    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 00:53:50.163920 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156679    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 00:53:50.163920 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156682    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 00:53:50.163920 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156684    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 00:53:50.163920 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156687    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 00:53:50.163920 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156690    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 00:53:50.164429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156693    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 00:53:50.164429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156696    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 00:53:50.164429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156699    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 00:53:50.164429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156702    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 00:53:50.164429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156705    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 00:53:50.164429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156707    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 00:53:50.164429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156710    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 00:53:50.164429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156712    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 00:53:50.164429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156715    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 00:53:50.164429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156717    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 00:53:50.164429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156720    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 00:53:50.164429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156722    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 00:53:50.164429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156726    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 00:53:50.164429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156735    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 00:53:50.164429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156739    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 00:53:50.164429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156742    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 00:53:50.164429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156745    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 00:53:50.164429 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:50.156748    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 00:53:50.164862 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.156753    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 00:53:50.164862 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.156873    2578 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 00:53:50.164862 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.158783    2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 00:53:50.164862 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.160038    2578 server.go:1019] "Starting client certificate rotation"
Jul 28 00:53:50.164862 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.160141    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 00:53:50.164862 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.160746    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 00:53:50.164862 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.162971    2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 00:53:50.169571 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.169541    2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 00:53:50.180799 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.180778    2578 log.go:25] "Validated CRI v1 runtime API"
Jul 28 00:53:50.182852 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.182838    2578 log.go:25] "Validated CRI v1 image API"
Jul 28 00:53:50.183752 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.183734    2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 00:53:50.188606 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.188584    2578 fs.go:135] Filesystem UUIDs: map[7662b527-b688-447d-bbce-d9b3459f95a0:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 efb35f25-7789-457a-ae75-1272364cb1a4:/dev/nvme0n1p3]
Jul 28 00:53:50.188659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.188605    2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 00:53:50.189416 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.189399    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 00:53:50.193912 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.193794    2578 manager.go:216] Machine: {Timestamp:2026-07-28 00:53:50.192458082 +0000 UTC m=+0.357266181 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099383 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26ab2b68b38365fae32977f2622d6e SystemUUID:ec26ab2b-68b3-8365-fae3-2977f2622d6e BootID:a232aa42-d989-4027-9429-3acb6776803a Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d6:5e:5d:be:f9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d6:5e:5d:be:f9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:99:cf:f0:07:34 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 00:53:50.193912 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.193910    2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 00:53:50.194032 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.193995    2578 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 00:53:50.196847 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.196816    2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 00:53:50.196998 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.196851    2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-238.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 00:53:50.197541 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.197531    2578 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 00:53:50.197574 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.197544    2578 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 00:53:50.197574 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.197552    2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 00:53:50.198200 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.198190    2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 00:53:50.199488 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.199478    2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 00:53:50.202397 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.202385    2578 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 00:53:50.202535 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.202526    2578 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 00:53:50.202565 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.202559    2578 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 00:53:50.202598 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.202569    2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 00:53:50.202598 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.202587    2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 00:53:50.202654 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.202598    2578 kubelet.go:398] "Adding apiserver pod source"
Jul 28 00:53:50.202654 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.202608    2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 00:53:50.202975 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.202963    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 00:53:50.203020 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.202982    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 00:53:50.205512 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.205497    2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 00:53:50.207044 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.207026    2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 00:53:50.207123 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.207066    2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 00:53:50.207832 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.207818    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 00:53:50.207908 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.207840    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 00:53:50.207908 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.207850    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 00:53:50.207908 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.207859    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 00:53:50.207908 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.207872    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 00:53:50.207908 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.207881    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 00:53:50.207908 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.207894    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 00:53:50.207908 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.207902    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 00:53:50.208134 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.207912    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 00:53:50.208134 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.207920    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 00:53:50.208134 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.207933    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 00:53:50.208134 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.207950    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 00:53:50.208627 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.208616    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 00:53:50.208678 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.208630    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 00:53:50.211857 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.211833    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qkhnf"
Jul 28 00:53:50.212038 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.212025    2578 server.go:1268] "Started kubelet"
Jul 28 00:53:50.212176 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.212094    2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 00:53:50.212251 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.212159    2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 00:53:50.212301 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.212250    2578 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 00:53:50.216768 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.216745    2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 00:53:50.216851 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.216781    2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-238.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 00:53:50.216965 ip-10-0-132-238 systemd[1]: Started Kubernetes Kubelet.
Jul 28 00:53:50.217802 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.217780    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qkhnf"
Jul 28 00:53:50.218526 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:50.217978    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 00:53:50.218860 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:50.218831    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-238.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 00:53:50.219461 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:50.219440    2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 00:53:50.219535 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.219522    2578 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 00:53:50.223260 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:50.222263    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-238.ec2.internal.18c64df815cd7c22  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-238.ec2.internal,UID:ip-10-0-132-238.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-238.ec2.internal,},FirstTimestamp:2026-07-28 00:53:50.211996706 +0000 UTC m=+0.376804804,LastTimestamp:2026-07-28 00:53:50.211996706 +0000 UTC m=+0.376804804,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-238.ec2.internal,}"
Jul 28 00:53:50.225742 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.225725    2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 00:53:50.225803 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.225735    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 00:53:50.226000 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:50.225978    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-238.ec2.internal\" not found"
Jul 28 00:53:50.226764 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.226748    2578 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 00:53:50.226842 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.226762    2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 00:53:50.226842 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.226767    2578 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 00:53:50.226982 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.226909    2578 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 00:53:50.226982 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.226918    2578 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 00:53:50.226982 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.226975    2578 factory.go:55] Registering systemd factory
Jul 28 00:53:50.227144 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.226990    2578 factory.go:223] Registration of the systemd container factory successfully
Jul 28 00:53:50.227404 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.227385    2578 factory.go:153] Registering CRI-O factory
Jul 28 00:53:50.227404 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.227407    2578 factory.go:223] Registration of the crio container factory successfully
Jul 28 00:53:50.227547 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.227463    2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 00:53:50.227547 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.227487    2578 factory.go:103] Registering Raw factory
Jul 28 00:53:50.227547 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.227502    2578 manager.go:1196] Started watching for new ooms in manager
Jul 28 00:53:50.227936 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.227920    2578 manager.go:349] Starting recovery of all containers
Jul 28 00:53:50.230194 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.230177    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 00:53:50.232784 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:50.232758    2578 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-132-238.ec2.internal\" not found" node="ip-10-0-132-238.ec2.internal"
Jul 28 00:53:50.238122 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.237981    2578 manager.go:354] Recovery completed
Jul 28 00:53:50.242268 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.242254    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 00:53:50.244121 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.244104    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 00:53:50.244195 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.244134    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 00:53:50.244195 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.244147    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-238.ec2.internal" event="NodeHasSufficientPID"
Jul 28 00:53:50.244729 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.244714    2578 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 00:53:50.244729 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.244727    2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 00:53:50.244825 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.244743    2578 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 00:53:50.244825 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.244752    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 00:53:50.245857 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.245846    2578 policy_none.go:49] "None policy: Start"
Jul 28 00:53:50.245909 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.245863    2578 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 00:53:50.245909 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.245875    2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 00:53:50.245968 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.245877    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 00:53:50.245997 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.245983    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 00:53:50.246028 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.245997    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-238.ec2.internal" event="NodeHasSufficientPID"
Jul 28 00:53:50.246825 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.246813    2578 policy_none.go:47] "Start"
Jul 28 00:53:50.284802 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.284775    2578 manager.go:337] "Starting Device Plugin manager"
Jul 28 00:53:50.284922 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:50.284816    2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 00:53:50.284922 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.284829    2578 server.go:87] "Starting device plugin registration server"
Jul 28 00:53:50.285115 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.285101    2578 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 00:53:50.285177 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.285117    2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 00:53:50.285250 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.285222    2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 00:53:50.285329 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.285316    2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 00:53:50.285376 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.285330    2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 00:53:50.286415 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:50.286396    2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 00:53:50.286521 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:50.286441    2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-238.ec2.internal\" not found"
Jul 28 00:53:50.386026 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.385985    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 00:53:50.387556 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.387532    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 00:53:50.387673 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.387574    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 00:53:50.387673 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.387605    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-238.ec2.internal" event="NodeHasSufficientPID"
Jul 28 00:53:50.387673 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.387640    2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-238.ec2.internal"
Jul 28 00:53:50.394002 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.393940    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 00:53:50.395366 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.395345    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 00:53:50.395436 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.395424    2578 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 00:53:50.395488 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.395462    2578 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 00:53:50.395581 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:50.395513    2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 00:53:50.396582 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.396559    2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-238.ec2.internal"
Jul 28 00:53:50.396679 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:50.396592    2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-238.ec2.internal\": node \"ip-10-0-132-238.ec2.internal\" not found"
Jul 28 00:53:50.397370 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.397354    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 00:53:50.429397 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:50.429356    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-238.ec2.internal\" not found"
Jul 28 00:53:50.496684 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.496647    2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-238.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-238.ec2.internal"]
Jul 28 00:53:50.496824 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.496739    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 00:53:50.498555 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.498534    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 00:53:50.498705 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.498567    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 00:53:50.498705 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.498579    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-238.ec2.internal" event="NodeHasSufficientPID"
Jul 28 00:53:50.499850 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.499835    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 00:53:50.500002 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.499987    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-238.ec2.internal"
Jul 28 00:53:50.500063 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.500026    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 00:53:50.500615 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.500597    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 00:53:50.500730 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.500628    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 00:53:50.500730 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.500639    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-238.ec2.internal" event="NodeHasSufficientPID"
Jul 28 00:53:50.500730 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.500601    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 00:53:50.500730 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.500706    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 00:53:50.500730 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.500722    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-238.ec2.internal" event="NodeHasSufficientPID"
Jul 28 00:53:50.501774 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.501757    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-238.ec2.internal"
Jul 28 00:53:50.501866 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.501785    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 00:53:50.502512 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.502496    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 00:53:50.502610 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.502523    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 00:53:50.502610 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.502539    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-238.ec2.internal" event="NodeHasSufficientPID"
Jul 28 00:53:50.521912 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:50.521883    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-238.ec2.internal\" not found" node="ip-10-0-132-238.ec2.internal"
Jul 28 00:53:50.526194 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:50.526178    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-238.ec2.internal\" not found" node="ip-10-0-132-238.ec2.internal"
Jul 28 00:53:50.529097 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.529082    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4fb08da72a473db0ae4f0a5d5930c1dc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-238.ec2.internal\" (UID: \"4fb08da72a473db0ae4f0a5d5930c1dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-238.ec2.internal"
Jul 28 00:53:50.529150 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.529105    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fb08da72a473db0ae4f0a5d5930c1dc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-238.ec2.internal\" (UID: \"4fb08da72a473db0ae4f0a5d5930c1dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-238.ec2.internal"
Jul 28 00:53:50.529150 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.529124    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7f112d9abde8daeb5837b42f75048eeb-config\") pod \"kube-apiserver-proxy-ip-10-0-132-238.ec2.internal\" (UID: \"7f112d9abde8daeb5837b42f75048eeb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-238.ec2.internal"
Jul 28 00:53:50.530131 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:50.530114    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-238.ec2.internal\" not found"
Jul 28 00:53:50.629522 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.629487    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4fb08da72a473db0ae4f0a5d5930c1dc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-238.ec2.internal\" (UID: \"4fb08da72a473db0ae4f0a5d5930c1dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-238.ec2.internal"
Jul 28 00:53:50.629650 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.629529    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fb08da72a473db0ae4f0a5d5930c1dc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-238.ec2.internal\" (UID: \"4fb08da72a473db0ae4f0a5d5930c1dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-238.ec2.internal"
Jul 28 00:53:50.629650 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.629545    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7f112d9abde8daeb5837b42f75048eeb-config\") pod \"kube-apiserver-proxy-ip-10-0-132-238.ec2.internal\" (UID: \"7f112d9abde8daeb5837b42f75048eeb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-238.ec2.internal"
Jul 28 00:53:50.629650 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.629579    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4fb08da72a473db0ae4f0a5d5930c1dc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-238.ec2.internal\" (UID: \"4fb08da72a473db0ae4f0a5d5930c1dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-238.ec2.internal"
Jul 28 00:53:50.629650 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.629603    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fb08da72a473db0ae4f0a5d5930c1dc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-238.ec2.internal\" (UID: \"4fb08da72a473db0ae4f0a5d5930c1dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-238.ec2.internal"
Jul 28 00:53:50.629789 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.629657    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7f112d9abde8daeb5837b42f75048eeb-config\") pod \"kube-apiserver-proxy-ip-10-0-132-238.ec2.internal\" (UID: \"7f112d9abde8daeb5837b42f75048eeb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-238.ec2.internal"
Jul 28 00:53:50.630515 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:50.630501    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-238.ec2.internal\" not found"
Jul 28 00:53:50.730719 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:50.730652    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-238.ec2.internal\" not found"
Jul 28 00:53:50.824562 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.824531    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-238.ec2.internal"
Jul 28 00:53:50.830040 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:50.830018    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-238.ec2.internal"
Jul 28 00:53:50.831108 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:50.831089    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-238.ec2.internal\" not found"
Jul 28 00:53:50.931695 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:50.931657    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-238.ec2.internal\" not found"
Jul 28 00:53:51.032129 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:51.032051    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-238.ec2.internal\" not found"
Jul 28 00:53:51.132566 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:51.132532    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-238.ec2.internal\" not found"
Jul 28 00:53:51.159745 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:51.159700    2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 00:53:51.160361 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:51.159869    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 00:53:51.160361 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:51.159900    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 00:53:51.222823 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:51.222787    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 00:48:50 +0000 UTC" deadline="2028-03-24 00:29:07.97777515 +0000 UTC"
Jul 28 00:53:51.222823 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:51.222817    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14519h35m16.754960926s"
Jul 28 00:53:51.233541 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:51.233516    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-238.ec2.internal\" not found"
Jul 28 00:53:51.284539 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:51.284455    2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 00:53:51.295730 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:51.295694    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f112d9abde8daeb5837b42f75048eeb.slice/crio-d664e8f7cd8cd4007b1cc9f7ad76f3606346c467f12f8fa85a1472c990e2cd44 WatchSource:0}: Error finding container d664e8f7cd8cd4007b1cc9f7ad76f3606346c467f12f8fa85a1472c990e2cd44: Status 404 returned error can't find the container with id d664e8f7cd8cd4007b1cc9f7ad76f3606346c467f12f8fa85a1472c990e2cd44
Jul 28 00:53:51.296303 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:51.296282    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fb08da72a473db0ae4f0a5d5930c1dc.slice/crio-1ee6b29a47767be8c6d79e40cefa7972624715b76a9b3a5820567c939d1df950 WatchSource:0}: Error finding container 1ee6b29a47767be8c6d79e40cefa7972624715b76a9b3a5820567c939d1df950: Status 404 returned error can't find the container with id 1ee6b29a47767be8c6d79e40cefa7972624715b76a9b3a5820567c939d1df950
Jul 28 00:53:51.301922 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:51.301905    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 00:53:51.334329 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:51.334297    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-238.ec2.internal\" not found"
Jul 28 00:53:51.398658 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:51.398610    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-238.ec2.internal" event={"ID":"7f112d9abde8daeb5837b42f75048eeb","Type":"ContainerStarted","Data":"d664e8f7cd8cd4007b1cc9f7ad76f3606346c467f12f8fa85a1472c990e2cd44"}
Jul 28 00:53:51.399565 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:51.399544    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-238.ec2.internal" event={"ID":"4fb08da72a473db0ae4f0a5d5930c1dc","Type":"ContainerStarted","Data":"1ee6b29a47767be8c6d79e40cefa7972624715b76a9b3a5820567c939d1df950"}
Jul 28 00:53:51.434956 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:51.434930    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-238.ec2.internal\" not found"
Jul 28 00:53:51.460442 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:51.460421    2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 00:53:51.527187 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:51.527160    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-238.ec2.internal"
Jul 28 00:53:51.537900 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:51.537828    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 00:53:51.538795 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:51.538780    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-238.ec2.internal"
Jul 28 00:53:51.545845 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:51.545825    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 00:53:52.203665 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.203633    2578 apiserver.go:52] "Watching apiserver"
Jul 28 00:53:52.212115 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.212088    2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 00:53:52.214102 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.214073    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-ff2f2","openshift-network-operator/iptables-alerter-6hxnm","kube-system/global-pull-secret-syncer-lvd59","kube-system/konnectivity-agent-xqfwc","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm","openshift-dns/node-resolver-pwmfh","openshift-image-registry/node-ca-cgdpl","openshift-multus/multus-28rt8","openshift-multus/multus-additional-cni-plugins-7cgzs","openshift-ovn-kubernetes/ovnkube-node-t87lr","kube-system/kube-apiserver-proxy-ip-10-0-132-238.ec2.internal","openshift-cluster-node-tuning-operator/tuned-62rn6","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-238.ec2.internal","openshift-multus/network-metrics-daemon-gpv8q"]
Jul 28 00:53:52.215426 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.215409    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cgdpl"
Jul 28 00:53:52.216505 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.216483    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.217664 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.217484    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-trkfd\""
Jul 28 00:53:52.217798 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.217677    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 00:53:52.217798 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.217727    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:53:52.217798 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.217764    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 00:53:52.217937 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:52.217837    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvd59" podUID="27305211-7ed9-4558-bd84-f703dcff265a"
Jul 28 00:53:52.217991 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.217972    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 00:53:52.218454 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.218436    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 00:53:52.218652 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.218631    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-n77xz\""
Jul 28 00:53:52.219377 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.218940    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xqfwc"
Jul 28 00:53:52.219377 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.218953    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 00:53:52.219377 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.218981    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 00:53:52.219377 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.219193    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 00:53:52.219973 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.219852    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm"
Jul 28 00:53:52.220920 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.220900    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-q5z4q\""
Jul 28 00:53:52.221415 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.221171    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pwmfh"
Jul 28 00:53:52.221415 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.221260    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 00:53:52.221415 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.221280    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 00:53:52.222089 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.222070    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 00:53:52.222160 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.222151    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 00:53:52.222207 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.222156    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 00:53:52.222448 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.222430    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-txw5m\""
Jul 28 00:53:52.222819 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.222797    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7cgzs"
Jul 28 00:53:52.223394 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.223375    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 00:53:52.223493 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.223404    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 00:53:52.223493 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.223376    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-49nz8\""
Jul 28 00:53:52.224117 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.224099    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:53:52.224197 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:52.224175    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ff2f2" podUID="e5c337e6-650b-4ee1-a24e-3533292e41d1"
Jul 28 00:53:52.224689 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.224667    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 00:53:52.224934 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.224917    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gjx9x\""
Jul 28 00:53:52.225061 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.225045    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 00:53:52.225401 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.225385    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6hxnm"
Jul 28 00:53:52.225981 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.225958    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 00:53:52.230647 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.227312    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 00:53:52.230647 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.227360    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-z9jfm\""
Jul 28 00:53:52.230647 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.227866    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.230647 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.228086    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 00:53:52.230647 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.229382    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 00:53:52.230647 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.230230    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:53:52.230647 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:52.230315    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpv8q" podUID="532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d"
Jul 28 00:53:52.231761 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.231743    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 00:53:52.231761 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.231755    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-t9w4q\""
Jul 28 00:53:52.231904 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.231839    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 00:53:52.231993 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.231980    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 00:53:52.232044 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.232020    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 00:53:52.232254 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.232237    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 00:53:52.232332 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.232240    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 00:53:52.232332 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.232283    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.234320 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.234298    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-h7rjf\""
Jul 28 00:53:52.234648 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.234630    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 00:53:52.235228 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.235200    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 00:53:52.239157 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.239138    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 00:53:52.239359 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.239338    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/074938e2-4436-4f39-bf00-6333580a6916-cni-binary-copy\") pod \"multus-additional-cni-plugins-7cgzs\" (UID: \"074938e2-4436-4f39-bf00-6333580a6916\") " pod="openshift-multus/multus-additional-cni-plugins-7cgzs"
Jul 28 00:53:52.239447 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.239374    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-host-var-lib-kubelet\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.239447 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.239398    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-run-ovn\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.239538 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.239438    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/894bd991-2d81-4275-96c3-a6464fc1d74a-host\") pod \"node-ca-cgdpl\" (UID: \"894bd991-2d81-4275-96c3-a6464fc1d74a\") " pod="openshift-image-registry/node-ca-cgdpl"
Jul 28 00:53:52.239538 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.239495    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds7w2\" (UniqueName: \"kubernetes.io/projected/894bd991-2d81-4275-96c3-a6464fc1d74a-kube-api-access-ds7w2\") pod \"node-ca-cgdpl\" (UID: \"894bd991-2d81-4275-96c3-a6464fc1d74a\") " pod="openshift-image-registry/node-ca-cgdpl"
Jul 28 00:53:52.239538 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.239530    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-host-var-lib-cni-multus\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.239659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.239553    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-hostroot\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.239659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.239581    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-etc-kubernetes\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.239659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.239617    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h56lz\" (UniqueName: \"kubernetes.io/projected/b333573a-411c-4999-964f-daa8b82b48e4-kube-api-access-h56lz\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.239794 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.239660    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/27305211-7ed9-4558-bd84-f703dcff265a-dbus\") pod \"global-pull-secret-syncer-lvd59\" (UID: \"27305211-7ed9-4558-bd84-f703dcff265a\") " pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:53:52.239794 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.239687    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2e249ae8-06c0-4082-b90e-a7e54824bea1-socket-dir\") pod \"aws-ebs-csi-driver-node-64gsm\" (UID: \"2e249ae8-06c0-4082-b90e-a7e54824bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm"
Jul 28 00:53:52.239794 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.239716    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/074938e2-4436-4f39-bf00-6333580a6916-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7cgzs\" (UID: \"074938e2-4436-4f39-bf00-6333580a6916\") " pod="openshift-multus/multus-additional-cni-plugins-7cgzs"
Jul 28 00:53:52.239794 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.239740    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-run-openvswitch\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.239794 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.239763    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2e249ae8-06c0-4082-b90e-a7e54824bea1-registration-dir\") pod \"aws-ebs-csi-driver-node-64gsm\" (UID: \"2e249ae8-06c0-4082-b90e-a7e54824bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm"
Jul 28 00:53:52.239794 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.239790    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxpwg\" (UniqueName: \"kubernetes.io/projected/0d08553a-6760-43c3-bc66-4799fd934954-kube-api-access-fxpwg\") pod \"node-resolver-pwmfh\" (UID: \"0d08553a-6760-43c3-bc66-4799fd934954\") " pod="openshift-dns/node-resolver-pwmfh"
Jul 28 00:53:52.240040 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.239813    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b333573a-411c-4999-964f-daa8b82b48e4-cni-binary-copy\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.240040 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.239839    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-host-cni-netd\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.240040 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.239861    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a15efe45-733c-4a1c-b3d1-d22f75b18566-ovnkube-config\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.240040 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.239897    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a15efe45-733c-4a1c-b3d1-d22f75b18566-ovn-node-metrics-cert\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.240040 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.239927    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c5pc\" (UniqueName: \"kubernetes.io/projected/a15efe45-733c-4a1c-b3d1-d22f75b18566-kube-api-access-2c5pc\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.240040 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.239952    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/074938e2-4436-4f39-bf00-6333580a6916-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7cgzs\" (UID: \"074938e2-4436-4f39-bf00-6333580a6916\") " pod="openshift-multus/multus-additional-cni-plugins-7cgzs"
Jul 28 00:53:52.240040 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.239973    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-host-run-netns\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.240040 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240004    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-multus-socket-dir-parent\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.240040 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240035    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-node-log\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.240379 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240059    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a2f32b69-b720-4c3c-b345-4cbd68398e2d-konnectivity-ca\") pod \"konnectivity-agent-xqfwc\" (UID: \"a2f32b69-b720-4c3c-b345-4cbd68398e2d\") " pod="kube-system/konnectivity-agent-xqfwc"
Jul 28 00:53:52.240379 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240085    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2e249ae8-06c0-4082-b90e-a7e54824bea1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-64gsm\" (UID: \"2e249ae8-06c0-4082-b90e-a7e54824bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm"
Jul 28 00:53:52.240379 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240113    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2e249ae8-06c0-4082-b90e-a7e54824bea1-etc-selinux\") pod \"aws-ebs-csi-driver-node-64gsm\" (UID: \"2e249ae8-06c0-4082-b90e-a7e54824bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm"
Jul 28 00:53:52.240379 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240146    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-multus-cni-dir\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.240379 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240161    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d08553a-6760-43c3-bc66-4799fd934954-tmp-dir\") pod \"node-resolver-pwmfh\" (UID: \"0d08553a-6760-43c3-bc66-4799fd934954\") " pod="openshift-dns/node-resolver-pwmfh"
Jul 28 00:53:52.240379 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240182    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-var-lib-openvswitch\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.240379 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240206    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27305211-7ed9-4558-bd84-f703dcff265a-original-pull-secret\") pod \"global-pull-secret-syncer-lvd59\" (UID: \"27305211-7ed9-4558-bd84-f703dcff265a\") " pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:53:52.240379 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240244    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/074938e2-4436-4f39-bf00-6333580a6916-system-cni-dir\") pod \"multus-additional-cni-plugins-7cgzs\" (UID: \"074938e2-4436-4f39-bf00-6333580a6916\") " pod="openshift-multus/multus-additional-cni-plugins-7cgzs"
Jul 28 00:53:52.240379 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240276    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-host-var-lib-cni-bin\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.240379 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240300    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b333573a-411c-4999-964f-daa8b82b48e4-multus-daemon-config\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.240379 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240318    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0d08553a-6760-43c3-bc66-4799fd934954-hosts-file\") pod \"node-resolver-pwmfh\" (UID: \"0d08553a-6760-43c3-bc66-4799fd934954\") " pod="openshift-dns/node-resolver-pwmfh"
Jul 28 00:53:52.240379 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240335    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-host-run-netns\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.240379 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240351    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-system-cni-dir\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.240379 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240375    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f7b8125f-f60c-41f4-aca3-bc03804e41b6-host-slash\") pod \"iptables-alerter-6hxnm\" (UID: \"f7b8125f-f60c-41f4-aca3-bc03804e41b6\") " pod="openshift-network-operator/iptables-alerter-6hxnm"
Jul 28 00:53:52.240912 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240397    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-host-kubelet\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.240912 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240447    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/27305211-7ed9-4558-bd84-f703dcff265a-kubelet-config\") pod \"global-pull-secret-syncer-lvd59\" (UID: \"27305211-7ed9-4558-bd84-f703dcff265a\") " pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:53:52.240912 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240472    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2e249ae8-06c0-4082-b90e-a7e54824bea1-sys-fs\") pod \"aws-ebs-csi-driver-node-64gsm\" (UID: \"2e249ae8-06c0-4082-b90e-a7e54824bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm"
Jul 28 00:53:52.240912 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240497    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/074938e2-4436-4f39-bf00-6333580a6916-os-release\") pod \"multus-additional-cni-plugins-7cgzs\" (UID: \"074938e2-4436-4f39-bf00-6333580a6916\") " pod="openshift-multus/multus-additional-cni-plugins-7cgzs"
Jul 28 00:53:52.240912 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240520    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-host-run-multus-certs\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.240912 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240543    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-host-run-ovn-kubernetes\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.240912 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240566    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-host-cni-bin\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.240912 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240592    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm25j\" (UniqueName: \"kubernetes.io/projected/2e249ae8-06c0-4082-b90e-a7e54824bea1-kube-api-access-vm25j\") pod \"aws-ebs-csi-driver-node-64gsm\" (UID: \"2e249ae8-06c0-4082-b90e-a7e54824bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm"
Jul 28 00:53:52.240912 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240626    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knsp9\" (UniqueName: \"kubernetes.io/projected/e5c337e6-650b-4ee1-a24e-3533292e41d1-kube-api-access-knsp9\") pod \"network-check-target-ff2f2\" (UID: \"e5c337e6-650b-4ee1-a24e-3533292e41d1\") " pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:53:52.240912 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240647    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-os-release\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.240912 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240667    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-cnibin\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.240912 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240691    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lrw6\" (UniqueName: \"kubernetes.io/projected/f7b8125f-f60c-41f4-aca3-bc03804e41b6-kube-api-access-6lrw6\") pod \"iptables-alerter-6hxnm\" (UID: \"f7b8125f-f60c-41f4-aca3-bc03804e41b6\") " pod="openshift-network-operator/iptables-alerter-6hxnm"
Jul 28 00:53:52.240912 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240866    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-systemd-units\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.240912 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240895    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-etc-openvswitch\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.241546 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240933    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.241546 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240949    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a15efe45-733c-4a1c-b3d1-d22f75b18566-env-overrides\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.241546 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.240988    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2e249ae8-06c0-4082-b90e-a7e54824bea1-device-dir\") pod \"aws-ebs-csi-driver-node-64gsm\" (UID: \"2e249ae8-06c0-4082-b90e-a7e54824bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm"
Jul 28 00:53:52.241546 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.241009    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-host-run-k8s-cni-cncf-io\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.241546 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.241041    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-host-slash\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.241546 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.241057    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/074938e2-4436-4f39-bf00-6333580a6916-cnibin\") pod \"multus-additional-cni-plugins-7cgzs\" (UID: \"074938e2-4436-4f39-bf00-6333580a6916\") " pod="openshift-multus/multus-additional-cni-plugins-7cgzs"
Jul 28 00:53:52.241546 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.241135    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/894bd991-2d81-4275-96c3-a6464fc1d74a-serviceca\") pod \"node-ca-cgdpl\" (UID: \"894bd991-2d81-4275-96c3-a6464fc1d74a\") " pod="openshift-image-registry/node-ca-cgdpl"
Jul 28 00:53:52.241546 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.241172    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-multus-conf-dir\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.241546 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.241197    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-run-systemd\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.241546 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.241236    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f7b8125f-f60c-41f4-aca3-bc03804e41b6-iptables-alerter-script\") pod \"iptables-alerter-6hxnm\" (UID: \"f7b8125f-f60c-41f4-aca3-bc03804e41b6\") " pod="openshift-network-operator/iptables-alerter-6hxnm"
Jul 28 00:53:52.241546 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.241263    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-log-socket\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.241546 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.241286    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a15efe45-733c-4a1c-b3d1-d22f75b18566-ovnkube-script-lib\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.241546 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.241332    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5n2m\" (UniqueName: \"kubernetes.io/projected/074938e2-4436-4f39-bf00-6333580a6916-kube-api-access-w5n2m\") pod \"multus-additional-cni-plugins-7cgzs\" (UID: \"074938e2-4436-4f39-bf00-6333580a6916\") " pod="openshift-multus/multus-additional-cni-plugins-7cgzs"
Jul 28 00:53:52.241546 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.241376    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a2f32b69-b720-4c3c-b345-4cbd68398e2d-agent-certs\") pod \"konnectivity-agent-xqfwc\" (UID: \"a2f32b69-b720-4c3c-b345-4cbd68398e2d\") " pod="kube-system/konnectivity-agent-xqfwc"
Jul 28 00:53:52.241546 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.241403    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/074938e2-4436-4f39-bf00-6333580a6916-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7cgzs\" (UID: \"074938e2-4436-4f39-bf00-6333580a6916\") " pod="openshift-multus/multus-additional-cni-plugins-7cgzs"
Jul 28 00:53:52.256247 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.256202    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-b72fr"
Jul 28 00:53:52.262426 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.262408    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-b72fr"
Jul 28 00:53:52.328053 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.328014    2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 00:53:52.342531 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.342502    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-host-run-netns\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.342531 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.342536    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-multus-socket-dir-parent\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.342752 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.342557    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-node-log\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.342752 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.342632    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-host-run-netns\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.342752 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.342647    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a2f32b69-b720-4c3c-b345-4cbd68398e2d-konnectivity-ca\") pod \"konnectivity-agent-xqfwc\" (UID: \"a2f32b69-b720-4c3c-b345-4cbd68398e2d\") " pod="kube-system/konnectivity-agent-xqfwc"
Jul 28 00:53:52.342752 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.342664    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-multus-socket-dir-parent\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.342752 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.342687    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2e249ae8-06c0-4082-b90e-a7e54824bea1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-64gsm\" (UID: \"2e249ae8-06c0-4082-b90e-a7e54824bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm"
Jul 28 00:53:52.342752 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.342736    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2e249ae8-06c0-4082-b90e-a7e54824bea1-etc-selinux\") pod \"aws-ebs-csi-driver-node-64gsm\" (UID: \"2e249ae8-06c0-4082-b90e-a7e54824bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm"
Jul 28 00:53:52.342752 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.342632    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-node-log\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.343072 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.342759    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-multus-cni-dir\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.343072 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.342736    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2e249ae8-06c0-4082-b90e-a7e54824bea1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-64gsm\" (UID: \"2e249ae8-06c0-4082-b90e-a7e54824bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm"
Jul 28 00:53:52.343072 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.342806    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d08553a-6760-43c3-bc66-4799fd934954-tmp-dir\") pod \"node-resolver-pwmfh\" (UID: \"0d08553a-6760-43c3-bc66-4799fd934954\") " pod="openshift-dns/node-resolver-pwmfh"
Jul 28 00:53:52.343072 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.342850    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2e249ae8-06c0-4082-b90e-a7e54824bea1-etc-selinux\") pod \"aws-ebs-csi-driver-node-64gsm\" (UID: \"2e249ae8-06c0-4082-b90e-a7e54824bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm"
Jul 28 00:53:52.343072 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.342874    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-multus-cni-dir\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.343072 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.342914    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d-metrics-certs\") pod \"network-metrics-daemon-gpv8q\" (UID: \"532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d\") " pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:53:52.343072 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.342947    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-var-lib-openvswitch\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.343072 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.342974    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27305211-7ed9-4558-bd84-f703dcff265a-original-pull-secret\") pod \"global-pull-secret-syncer-lvd59\" (UID: \"27305211-7ed9-4558-bd84-f703dcff265a\") " pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:53:52.343072 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.342999    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/074938e2-4436-4f39-bf00-6333580a6916-system-cni-dir\") pod \"multus-additional-cni-plugins-7cgzs\" (UID: \"074938e2-4436-4f39-bf00-6333580a6916\") " pod="openshift-multus/multus-additional-cni-plugins-7cgzs"
Jul 28 00:53:52.343072 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343024    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-host-var-lib-cni-bin\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.343072 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343034    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-var-lib-openvswitch\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.343072 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343051    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b333573a-411c-4999-964f-daa8b82b48e4-multus-daemon-config\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.343656 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343080    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0d08553a-6760-43c3-bc66-4799fd934954-hosts-file\") pod \"node-resolver-pwmfh\" (UID: \"0d08553a-6760-43c3-bc66-4799fd934954\") " pod="openshift-dns/node-resolver-pwmfh"
Jul 28 00:53:52.343656 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343105    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-etc-sysctl-d\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.343656 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343121    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-host-var-lib-cni-bin\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.343656 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343131    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-etc-sysctl-conf\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.343656 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343081    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/074938e2-4436-4f39-bf00-6333580a6916-system-cni-dir\") pod \"multus-additional-cni-plugins-7cgzs\" (UID: \"074938e2-4436-4f39-bf00-6333580a6916\") " pod="openshift-multus/multus-additional-cni-plugins-7cgzs"
Jul 28 00:53:52.343656 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:52.343148    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 00:53:52.343656 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343157    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-host-run-netns\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.343656 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343182    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-system-cni-dir\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.343656 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:52.343198    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27305211-7ed9-4558-bd84-f703dcff265a-original-pull-secret podName:27305211-7ed9-4558-bd84-f703dcff265a nodeName:}" failed. No retries permitted until 2026-07-28 00:53:52.843184124 +0000 UTC m=+3.007992209 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/27305211-7ed9-4558-bd84-f703dcff265a-original-pull-secret") pod "global-pull-secret-syncer-lvd59" (UID: "27305211-7ed9-4558-bd84-f703dcff265a") : object "kube-system"/"original-pull-secret" not registered
Jul 28 00:53:52.343656 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343233    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-sys\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.343656 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343249    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd844814-e897-46e3-8d93-39785cd0712f-tmp\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.343656 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343253    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-system-cni-dir\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.343656 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343267    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f7b8125f-f60c-41f4-aca3-bc03804e41b6-host-slash\") pod \"iptables-alerter-6hxnm\" (UID: \"f7b8125f-f60c-41f4-aca3-bc03804e41b6\") " pod="openshift-network-operator/iptables-alerter-6hxnm"
Jul 28 00:53:52.343656 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343080    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d08553a-6760-43c3-bc66-4799fd934954-tmp-dir\") pod \"node-resolver-pwmfh\" (UID: \"0d08553a-6760-43c3-bc66-4799fd934954\") " pod="openshift-dns/node-resolver-pwmfh"
Jul 28 00:53:52.343656 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343284    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-host-kubelet\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.343656 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343300    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/27305211-7ed9-4558-bd84-f703dcff265a-kubelet-config\") pod \"global-pull-secret-syncer-lvd59\" (UID: \"27305211-7ed9-4558-bd84-f703dcff265a\") " pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:53:52.343656 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343309    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a2f32b69-b720-4c3c-b345-4cbd68398e2d-konnectivity-ca\") pod \"konnectivity-agent-xqfwc\" (UID: \"a2f32b69-b720-4c3c-b345-4cbd68398e2d\") " pod="kube-system/konnectivity-agent-xqfwc"
Jul 28 00:53:52.344416 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343320    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2e249ae8-06c0-4082-b90e-a7e54824bea1-sys-fs\") pod \"aws-ebs-csi-driver-node-64gsm\" (UID: \"2e249ae8-06c0-4082-b90e-a7e54824bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm"
Jul 28 00:53:52.344416 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343341    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/074938e2-4436-4f39-bf00-6333580a6916-os-release\") pod \"multus-additional-cni-plugins-7cgzs\" (UID: \"074938e2-4436-4f39-bf00-6333580a6916\") " pod="openshift-multus/multus-additional-cni-plugins-7cgzs"
Jul 28 00:53:52.344416 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343358    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-host-run-multus-certs\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.344416 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343373    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-run\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.344416 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343377    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-host-run-netns\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.344416 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343387    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-etc-sysconfig\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.344416 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343395    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-host-kubelet\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.344416 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343430    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2e249ae8-06c0-4082-b90e-a7e54824bea1-sys-fs\") pod \"aws-ebs-csi-driver-node-64gsm\" (UID: \"2e249ae8-06c0-4082-b90e-a7e54824bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm"
Jul 28 00:53:52.344416 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343402    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-var-lib-kubelet\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.344416 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343457    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/074938e2-4436-4f39-bf00-6333580a6916-os-release\") pod \"multus-additional-cni-plugins-7cgzs\" (UID: \"074938e2-4436-4f39-bf00-6333580a6916\") " pod="openshift-multus/multus-additional-cni-plugins-7cgzs"
Jul 28 00:53:52.344416 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343491    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-host-run-multus-certs\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.344416 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343503    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0d08553a-6760-43c3-bc66-4799fd934954-hosts-file\") pod \"node-resolver-pwmfh\" (UID: \"0d08553a-6760-43c3-bc66-4799fd934954\") " pod="openshift-dns/node-resolver-pwmfh"
Jul 28 00:53:52.344416 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343528    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/27305211-7ed9-4558-bd84-f703dcff265a-kubelet-config\") pod \"global-pull-secret-syncer-lvd59\" (UID: \"27305211-7ed9-4558-bd84-f703dcff265a\") " pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:53:52.344416 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343547    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f7b8125f-f60c-41f4-aca3-bc03804e41b6-host-slash\") pod \"iptables-alerter-6hxnm\" (UID: \"f7b8125f-f60c-41f4-aca3-bc03804e41b6\") " pod="openshift-network-operator/iptables-alerter-6hxnm"
Jul 28 00:53:52.344416 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343550    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-host-run-ovn-kubernetes\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.344416 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343587    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-host-run-ovn-kubernetes\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.344416 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343598    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-host-cni-bin\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.345184 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343619    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-host-cni-bin\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.345184 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343626    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vm25j\" (UniqueName: \"kubernetes.io/projected/2e249ae8-06c0-4082-b90e-a7e54824bea1-kube-api-access-vm25j\") pod \"aws-ebs-csi-driver-node-64gsm\" (UID: \"2e249ae8-06c0-4082-b90e-a7e54824bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm"
Jul 28 00:53:52.345184 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343654    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knsp9\" (UniqueName: \"kubernetes.io/projected/e5c337e6-650b-4ee1-a24e-3533292e41d1-kube-api-access-knsp9\") pod \"network-check-target-ff2f2\" (UID: \"e5c337e6-650b-4ee1-a24e-3533292e41d1\") " pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:53:52.345184 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343675    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-os-release\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.345184 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.343734    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b333573a-411c-4999-964f-daa8b82b48e4-multus-daemon-config\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.345184 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344007    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-cnibin\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.345184 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344027    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-os-release\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.345184 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344078    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-cnibin\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.345184 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344122    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lrw6\" (UniqueName: \"kubernetes.io/projected/f7b8125f-f60c-41f4-aca3-bc03804e41b6-kube-api-access-6lrw6\") pod \"iptables-alerter-6hxnm\" (UID: \"f7b8125f-f60c-41f4-aca3-bc03804e41b6\") " pod="openshift-network-operator/iptables-alerter-6hxnm"
Jul 28 00:53:52.345184 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344158    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-systemd-units\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.345184 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344189    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-etc-openvswitch\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.345184 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344237    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.345184 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344265    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a15efe45-733c-4a1c-b3d1-d22f75b18566-env-overrides\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.345184 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344295    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2e249ae8-06c0-4082-b90e-a7e54824bea1-device-dir\") pod \"aws-ebs-csi-driver-node-64gsm\" (UID: \"2e249ae8-06c0-4082-b90e-a7e54824bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm"
Jul 28 00:53:52.345184 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344323    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-host-run-k8s-cni-cncf-io\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.345184 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344380    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-host-run-k8s-cni-cncf-io\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.345184 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344419    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-host-slash\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.345953 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344445    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2e249ae8-06c0-4082-b90e-a7e54824bea1-device-dir\") pod \"aws-ebs-csi-driver-node-64gsm\" (UID: \"2e249ae8-06c0-4082-b90e-a7e54824bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm"
Jul 28 00:53:52.345953 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344457    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/074938e2-4436-4f39-bf00-6333580a6916-cnibin\") pod \"multus-additional-cni-plugins-7cgzs\" (UID: \"074938e2-4436-4f39-bf00-6333580a6916\") " pod="openshift-multus/multus-additional-cni-plugins-7cgzs"
Jul 28 00:53:52.345953 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344491    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/894bd991-2d81-4275-96c3-a6464fc1d74a-serviceca\") pod \"node-ca-cgdpl\" (UID: \"894bd991-2d81-4275-96c3-a6464fc1d74a\") " pod="openshift-image-registry/node-ca-cgdpl"
Jul 28 00:53:52.345953 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344494    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.345953 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344500    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-host-slash\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.345953 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344544    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-etc-openvswitch\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.345953 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344544    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-multus-conf-dir\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.345953 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344573    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/074938e2-4436-4f39-bf00-6333580a6916-cnibin\") pod \"multus-additional-cni-plugins-7cgzs\" (UID: \"074938e2-4436-4f39-bf00-6333580a6916\") " pod="openshift-multus/multus-additional-cni-plugins-7cgzs"
Jul 28 00:53:52.345953 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344590    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-run-systemd\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.345953 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344593    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-systemd-units\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.345953 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344625    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sztcl\" (UniqueName: \"kubernetes.io/projected/532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d-kube-api-access-sztcl\") pod \"network-metrics-daemon-gpv8q\" (UID: \"532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d\") " pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:53:52.345953 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344669    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-multus-conf-dir\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.345953 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344661    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f7b8125f-f60c-41f4-aca3-bc03804e41b6-iptables-alerter-script\") pod \"iptables-alerter-6hxnm\" (UID: \"f7b8125f-f60c-41f4-aca3-bc03804e41b6\") " pod="openshift-network-operator/iptables-alerter-6hxnm"
Jul 28 00:53:52.345953 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344712    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-log-socket\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.345953 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344750    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a15efe45-733c-4a1c-b3d1-d22f75b18566-ovnkube-script-lib\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.345953 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344785    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5n2m\" (UniqueName: \"kubernetes.io/projected/074938e2-4436-4f39-bf00-6333580a6916-kube-api-access-w5n2m\") pod \"multus-additional-cni-plugins-7cgzs\" (UID: \"074938e2-4436-4f39-bf00-6333580a6916\") " pod="openshift-multus/multus-additional-cni-plugins-7cgzs"
Jul 28 00:53:52.345953 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344813    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-etc-systemd\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.346848 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344818    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a15efe45-733c-4a1c-b3d1-d22f75b18566-env-overrides\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.346848 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344851    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a2f32b69-b720-4c3c-b345-4cbd68398e2d-agent-certs\") pod \"konnectivity-agent-xqfwc\" (UID: \"a2f32b69-b720-4c3c-b345-4cbd68398e2d\") " pod="kube-system/konnectivity-agent-xqfwc"
Jul 28 00:53:52.346848 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344882    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/074938e2-4436-4f39-bf00-6333580a6916-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7cgzs\" (UID: \"074938e2-4436-4f39-bf00-6333580a6916\") " pod="openshift-multus/multus-additional-cni-plugins-7cgzs"
Jul 28 00:53:52.346848 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344916    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-etc-kubernetes\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.346848 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344945    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-lib-modules\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.346848 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344974    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-log-socket\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.346848 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344917    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-run-systemd\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.346848 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.345074    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/894bd991-2d81-4275-96c3-a6464fc1d74a-serviceca\") pod \"node-ca-cgdpl\" (UID: \"894bd991-2d81-4275-96c3-a6464fc1d74a\") " pod="openshift-image-registry/node-ca-cgdpl"
Jul 28 00:53:52.346848 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.344976    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btcwg\" (UniqueName: \"kubernetes.io/projected/fd844814-e897-46e3-8d93-39785cd0712f-kube-api-access-btcwg\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.346848 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.345162    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/074938e2-4436-4f39-bf00-6333580a6916-cni-binary-copy\") pod \"multus-additional-cni-plugins-7cgzs\" (UID: \"074938e2-4436-4f39-bf00-6333580a6916\") " pod="openshift-multus/multus-additional-cni-plugins-7cgzs"
Jul 28 00:53:52.346848 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.345197    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-host-var-lib-kubelet\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.346848 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.345244    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-etc-modprobe-d\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.346848 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.345277    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-host\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.346848 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.345308    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-run-ovn\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.346848 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.345341    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/894bd991-2d81-4275-96c3-a6464fc1d74a-host\") pod \"node-ca-cgdpl\" (UID: \"894bd991-2d81-4275-96c3-a6464fc1d74a\") " pod="openshift-image-registry/node-ca-cgdpl"
Jul 28 00:53:52.346848 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.345382    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds7w2\" (UniqueName: \"kubernetes.io/projected/894bd991-2d81-4275-96c3-a6464fc1d74a-kube-api-access-ds7w2\") pod \"node-ca-cgdpl\" (UID: \"894bd991-2d81-4275-96c3-a6464fc1d74a\") " pod="openshift-image-registry/node-ca-cgdpl"
Jul 28 00:53:52.346848 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.345402    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f7b8125f-f60c-41f4-aca3-bc03804e41b6-iptables-alerter-script\") pod \"iptables-alerter-6hxnm\" (UID: \"f7b8125f-f60c-41f4-aca3-bc03804e41b6\") " pod="openshift-network-operator/iptables-alerter-6hxnm"
Jul 28 00:53:52.347639 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.345420    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-host-var-lib-cni-multus\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.347639 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.345478    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-host-var-lib-cni-multus\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.347639 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.345518    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-hostroot\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.347639 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.345535    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/074938e2-4436-4f39-bf00-6333580a6916-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7cgzs\" (UID: \"074938e2-4436-4f39-bf00-6333580a6916\") " pod="openshift-multus/multus-additional-cni-plugins-7cgzs"
Jul 28 00:53:52.347639 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.345546    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-etc-kubernetes\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.347639 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.345638    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-etc-kubernetes\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.347639 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.345661    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a15efe45-733c-4a1c-b3d1-d22f75b18566-ovnkube-script-lib\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.347639 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.345699    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-run-ovn\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.347639 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.345756    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/894bd991-2d81-4275-96c3-a6464fc1d74a-host\") pod \"node-ca-cgdpl\" (UID: \"894bd991-2d81-4275-96c3-a6464fc1d74a\") " pod="openshift-image-registry/node-ca-cgdpl"
Jul 28 00:53:52.347639 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.345810    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-hostroot\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.347639 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.345860    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b333573a-411c-4999-964f-daa8b82b48e4-host-var-lib-kubelet\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.347639 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.345887    2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 00:53:52.347639 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.345972    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h56lz\" (UniqueName: \"kubernetes.io/projected/b333573a-411c-4999-964f-daa8b82b48e4-kube-api-access-h56lz\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.347639 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.346008    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fd844814-e897-46e3-8d93-39785cd0712f-etc-tuned\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.347639 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.346043    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/27305211-7ed9-4558-bd84-f703dcff265a-dbus\") pod \"global-pull-secret-syncer-lvd59\" (UID: \"27305211-7ed9-4558-bd84-f703dcff265a\") " pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:53:52.347639 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.346078    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2e249ae8-06c0-4082-b90e-a7e54824bea1-socket-dir\") pod \"aws-ebs-csi-driver-node-64gsm\" (UID: \"2e249ae8-06c0-4082-b90e-a7e54824bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm"
Jul 28 00:53:52.347639 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.346109    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/074938e2-4436-4f39-bf00-6333580a6916-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7cgzs\" (UID: \"074938e2-4436-4f39-bf00-6333580a6916\") " pod="openshift-multus/multus-additional-cni-plugins-7cgzs"
Jul 28 00:53:52.347639 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.345972    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/074938e2-4436-4f39-bf00-6333580a6916-cni-binary-copy\") pod \"multus-additional-cni-plugins-7cgzs\" (UID: \"074938e2-4436-4f39-bf00-6333580a6916\") " pod="openshift-multus/multus-additional-cni-plugins-7cgzs"
Jul 28 00:53:52.348472 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.346176    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-run-openvswitch\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.348472 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.346235    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2e249ae8-06c0-4082-b90e-a7e54824bea1-registration-dir\") pod \"aws-ebs-csi-driver-node-64gsm\" (UID: \"2e249ae8-06c0-4082-b90e-a7e54824bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm"
Jul 28 00:53:52.348472 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.346271    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2e249ae8-06c0-4082-b90e-a7e54824bea1-socket-dir\") pod \"aws-ebs-csi-driver-node-64gsm\" (UID: \"2e249ae8-06c0-4082-b90e-a7e54824bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm"
Jul 28 00:53:52.348472 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.346301    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-run-openvswitch\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.348472 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.346322    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2e249ae8-06c0-4082-b90e-a7e54824bea1-registration-dir\") pod \"aws-ebs-csi-driver-node-64gsm\" (UID: \"2e249ae8-06c0-4082-b90e-a7e54824bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm"
Jul 28 00:53:52.348472 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.346358    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxpwg\" (UniqueName: \"kubernetes.io/projected/0d08553a-6760-43c3-bc66-4799fd934954-kube-api-access-fxpwg\") pod \"node-resolver-pwmfh\" (UID: \"0d08553a-6760-43c3-bc66-4799fd934954\") " pod="openshift-dns/node-resolver-pwmfh"
Jul 28 00:53:52.348472 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.346387    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b333573a-411c-4999-964f-daa8b82b48e4-cni-binary-copy\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.348472 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.346421    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-host-cni-netd\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.348472 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.346422    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/27305211-7ed9-4558-bd84-f703dcff265a-dbus\") pod \"global-pull-secret-syncer-lvd59\" (UID: \"27305211-7ed9-4558-bd84-f703dcff265a\") " pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:53:52.348472 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.346454    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a15efe45-733c-4a1c-b3d1-d22f75b18566-ovnkube-config\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.348472 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.346484    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a15efe45-733c-4a1c-b3d1-d22f75b18566-ovn-node-metrics-cert\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.348472 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.346667    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a15efe45-733c-4a1c-b3d1-d22f75b18566-host-cni-netd\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.348472 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.346687    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/074938e2-4436-4f39-bf00-6333580a6916-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7cgzs\" (UID: \"074938e2-4436-4f39-bf00-6333580a6916\") " pod="openshift-multus/multus-additional-cni-plugins-7cgzs"
Jul 28 00:53:52.348472 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.346840    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c5pc\" (UniqueName: \"kubernetes.io/projected/a15efe45-733c-4a1c-b3d1-d22f75b18566-kube-api-access-2c5pc\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.348472 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.346876    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/074938e2-4436-4f39-bf00-6333580a6916-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7cgzs\" (UID: \"074938e2-4436-4f39-bf00-6333580a6916\") " pod="openshift-multus/multus-additional-cni-plugins-7cgzs"
Jul 28 00:53:52.348472 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.347018    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b333573a-411c-4999-964f-daa8b82b48e4-cni-binary-copy\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.348472 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.347083    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/074938e2-4436-4f39-bf00-6333580a6916-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7cgzs\" (UID: \"074938e2-4436-4f39-bf00-6333580a6916\") " pod="openshift-multus/multus-additional-cni-plugins-7cgzs"
Jul 28 00:53:52.349194 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.347197    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a15efe45-733c-4a1c-b3d1-d22f75b18566-ovnkube-config\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.349727 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.349693    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a15efe45-733c-4a1c-b3d1-d22f75b18566-ovn-node-metrics-cert\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.349832 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.349819    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a2f32b69-b720-4c3c-b345-4cbd68398e2d-agent-certs\") pod \"konnectivity-agent-xqfwc\" (UID: \"a2f32b69-b720-4c3c-b345-4cbd68398e2d\") " pod="kube-system/konnectivity-agent-xqfwc"
Jul 28 00:53:52.350919 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:52.350898    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 00:53:52.350919 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:52.350922    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 00:53:52.351104 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:52.350936    2578 projected.go:196] Error preparing data for projected volume kube-api-access-knsp9 for pod openshift-network-diagnostics/network-check-target-ff2f2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 00:53:52.351104 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:52.350992    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e5c337e6-650b-4ee1-a24e-3533292e41d1-kube-api-access-knsp9 podName:e5c337e6-650b-4ee1-a24e-3533292e41d1 nodeName:}" failed. No retries permitted until 2026-07-28 00:53:52.85097466 +0000 UTC m=+3.015782768 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-knsp9" (UniqueName: "kubernetes.io/projected/e5c337e6-650b-4ee1-a24e-3533292e41d1-kube-api-access-knsp9") pod "network-check-target-ff2f2" (UID: "e5c337e6-650b-4ee1-a24e-3533292e41d1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 00:53:52.354113 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.354086    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds7w2\" (UniqueName: \"kubernetes.io/projected/894bd991-2d81-4275-96c3-a6464fc1d74a-kube-api-access-ds7w2\") pod \"node-ca-cgdpl\" (UID: \"894bd991-2d81-4275-96c3-a6464fc1d74a\") " pod="openshift-image-registry/node-ca-cgdpl"
Jul 28 00:53:52.354636 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.354613    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lrw6\" (UniqueName: \"kubernetes.io/projected/f7b8125f-f60c-41f4-aca3-bc03804e41b6-kube-api-access-6lrw6\") pod \"iptables-alerter-6hxnm\" (UID: \"f7b8125f-f60c-41f4-aca3-bc03804e41b6\") " pod="openshift-network-operator/iptables-alerter-6hxnm"
Jul 28 00:53:52.354849 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.354825    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5n2m\" (UniqueName: \"kubernetes.io/projected/074938e2-4436-4f39-bf00-6333580a6916-kube-api-access-w5n2m\") pod \"multus-additional-cni-plugins-7cgzs\" (UID: \"074938e2-4436-4f39-bf00-6333580a6916\") " pod="openshift-multus/multus-additional-cni-plugins-7cgzs"
Jul 28 00:53:52.354941 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.354833    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm25j\" (UniqueName: \"kubernetes.io/projected/2e249ae8-06c0-4082-b90e-a7e54824bea1-kube-api-access-vm25j\") pod \"aws-ebs-csi-driver-node-64gsm\" (UID: \"2e249ae8-06c0-4082-b90e-a7e54824bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm"
Jul 28 00:53:52.354941 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.354899    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h56lz\" (UniqueName: \"kubernetes.io/projected/b333573a-411c-4999-964f-daa8b82b48e4-kube-api-access-h56lz\") pod \"multus-28rt8\" (UID: \"b333573a-411c-4999-964f-daa8b82b48e4\") " pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.355505 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.355482    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxpwg\" (UniqueName: \"kubernetes.io/projected/0d08553a-6760-43c3-bc66-4799fd934954-kube-api-access-fxpwg\") pod \"node-resolver-pwmfh\" (UID: \"0d08553a-6760-43c3-bc66-4799fd934954\") " pod="openshift-dns/node-resolver-pwmfh"
Jul 28 00:53:52.356267 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.356247    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c5pc\" (UniqueName: \"kubernetes.io/projected/a15efe45-733c-4a1c-b3d1-d22f75b18566-kube-api-access-2c5pc\") pod \"ovnkube-node-t87lr\" (UID: \"a15efe45-733c-4a1c-b3d1-d22f75b18566\") " pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.447708 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.447671    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d-metrics-certs\") pod \"network-metrics-daemon-gpv8q\" (UID: \"532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d\") " pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:53:52.447941 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.447735    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-etc-sysctl-d\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.447941 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.447775    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-etc-sysctl-conf\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.447941 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.447792    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-sys\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.447941 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:52.447811    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 00:53:52.447941 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.447814    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd844814-e897-46e3-8d93-39785cd0712f-tmp\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.447941 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:52.447890    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d-metrics-certs podName:532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d nodeName:}" failed. No retries permitted until 2026-07-28 00:53:52.947870589 +0000 UTC m=+3.112678687 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d-metrics-certs") pod "network-metrics-daemon-gpv8q" (UID: "532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 00:53:52.448248 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.447950    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-run\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.448248 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.447980    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-etc-sysconfig\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.448248 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.448000    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-var-lib-kubelet\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.448248 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.448042    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sztcl\" (UniqueName: \"kubernetes.io/projected/532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d-kube-api-access-sztcl\") pod \"network-metrics-daemon-gpv8q\" (UID: \"532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d\") " pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:53:52.448248 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.447974    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-etc-sysctl-conf\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.448248 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.448069    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-etc-systemd\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.448248 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.448076    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-etc-sysconfig\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.448248 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.448091    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-etc-sysctl-d\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.448248 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.448098    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-var-lib-kubelet\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.448248 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.448054    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-run\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.448248 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.448109    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-etc-kubernetes\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.448248 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.448117    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-etc-systemd\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.448248 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.448139    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-lib-modules\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.448248 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.448150    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-etc-kubernetes\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.448248 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.448165    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btcwg\" (UniqueName: \"kubernetes.io/projected/fd844814-e897-46e3-8d93-39785cd0712f-kube-api-access-btcwg\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.448248 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.448252    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-sys\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.448861 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.448353    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-etc-modprobe-d\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.448861 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.448383    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-host\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.448861 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.448384    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-lib-modules\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.448861 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.448414    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fd844814-e897-46e3-8d93-39785cd0712f-etc-tuned\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.448861 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.448485    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-etc-modprobe-d\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.448861 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.448509    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fd844814-e897-46e3-8d93-39785cd0712f-host\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.450788 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.450752    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd844814-e897-46e3-8d93-39785cd0712f-tmp\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.450901 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.450882    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fd844814-e897-46e3-8d93-39785cd0712f-etc-tuned\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.456416 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.456365    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sztcl\" (UniqueName: \"kubernetes.io/projected/532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d-kube-api-access-sztcl\") pod \"network-metrics-daemon-gpv8q\" (UID: \"532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d\") " pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:53:52.456416 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.456376    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btcwg\" (UniqueName: \"kubernetes.io/projected/fd844814-e897-46e3-8d93-39785cd0712f-kube-api-access-btcwg\") pod \"tuned-62rn6\" (UID: \"fd844814-e897-46e3-8d93-39785cd0712f\") " pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.506899 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.506873    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 00:53:52.527798 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.527765    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cgdpl"
Jul 28 00:53:52.536908 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.536887    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm"
Jul 28 00:53:52.544569 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.544540    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-28rt8"
Jul 28 00:53:52.549883 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.549864    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xqfwc"
Jul 28 00:53:52.557671 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.557650    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pwmfh"
Jul 28 00:53:52.563979 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.563959    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7cgzs"
Jul 28 00:53:52.571161 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.571138    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6hxnm"
Jul 28 00:53:52.576801 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.576781    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:53:52.581516 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.581494    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-62rn6"
Jul 28 00:53:52.704689 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.704662    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 00:53:52.851316 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.851229    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27305211-7ed9-4558-bd84-f703dcff265a-original-pull-secret\") pod \"global-pull-secret-syncer-lvd59\" (UID: \"27305211-7ed9-4558-bd84-f703dcff265a\") " pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:53:52.851316 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.851292    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knsp9\" (UniqueName: \"kubernetes.io/projected/e5c337e6-650b-4ee1-a24e-3533292e41d1-kube-api-access-knsp9\") pod \"network-check-target-ff2f2\" (UID: \"e5c337e6-650b-4ee1-a24e-3533292e41d1\") " pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:53:52.851495 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:52.851395    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 00:53:52.851495 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:52.851410    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 00:53:52.851495 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:52.851427    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 00:53:52.851495 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:52.851436    2578 projected.go:196] Error preparing data for projected volume kube-api-access-knsp9 for pod openshift-network-diagnostics/network-check-target-ff2f2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 00:53:52.851495 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:52.851463    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27305211-7ed9-4558-bd84-f703dcff265a-original-pull-secret podName:27305211-7ed9-4558-bd84-f703dcff265a nodeName:}" failed. No retries permitted until 2026-07-28 00:53:53.851446653 +0000 UTC m=+4.016254738 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/27305211-7ed9-4558-bd84-f703dcff265a-original-pull-secret") pod "global-pull-secret-syncer-lvd59" (UID: "27305211-7ed9-4558-bd84-f703dcff265a") : object "kube-system"/"original-pull-secret" not registered
Jul 28 00:53:52.851495 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:52.851497    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e5c337e6-650b-4ee1-a24e-3533292e41d1-kube-api-access-knsp9 podName:e5c337e6-650b-4ee1-a24e-3533292e41d1 nodeName:}" failed. No retries permitted until 2026-07-28 00:53:53.851485061 +0000 UTC m=+4.016293146 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-knsp9" (UniqueName: "kubernetes.io/projected/e5c337e6-650b-4ee1-a24e-3533292e41d1-kube-api-access-knsp9") pod "network-check-target-ff2f2" (UID: "e5c337e6-650b-4ee1-a24e-3533292e41d1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 00:53:52.952361 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:52.952322    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d-metrics-certs\") pod \"network-metrics-daemon-gpv8q\" (UID: \"532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d\") " pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:53:52.952551 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:52.952480    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 00:53:52.952551 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:52.952545    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d-metrics-certs podName:532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d nodeName:}" failed. No retries permitted until 2026-07-28 00:53:53.952528329 +0000 UTC m=+4.117336420 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d-metrics-certs") pod "network-metrics-daemon-gpv8q" (UID: "532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 00:53:53.130659 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:53.130608    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d08553a_6760_43c3_bc66_4799fd934954.slice/crio-be4a5b9958e9866cbed674b6b9d0aedbc4096317e64ea53539a0067a2344005e WatchSource:0}: Error finding container be4a5b9958e9866cbed674b6b9d0aedbc4096317e64ea53539a0067a2344005e: Status 404 returned error can't find the container with id be4a5b9958e9866cbed674b6b9d0aedbc4096317e64ea53539a0067a2344005e
Jul 28 00:53:53.133863 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:53.133843    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod074938e2_4436_4f39_bf00_6333580a6916.slice/crio-01f5bee69164b1fea437d98d8f0dc6fe119574222443876633c53a4222d75ff0 WatchSource:0}: Error finding container 01f5bee69164b1fea437d98d8f0dc6fe119574222443876633c53a4222d75ff0: Status 404 returned error can't find the container with id 01f5bee69164b1fea437d98d8f0dc6fe119574222443876633c53a4222d75ff0
Jul 28 00:53:53.134696 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:53.134676    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd844814_e897_46e3_8d93_39785cd0712f.slice/crio-c0c7039032dd2e83fe69d693c06ba597300a6574be4ce8f4d6bbf0c6b3b7514f WatchSource:0}: Error finding container c0c7039032dd2e83fe69d693c06ba597300a6574be4ce8f4d6bbf0c6b3b7514f: Status 404 returned error can't find the container with id c0c7039032dd2e83fe69d693c06ba597300a6574be4ce8f4d6bbf0c6b3b7514f
Jul 28 00:53:53.135785 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:53.135715    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb333573a_411c_4999_964f_daa8b82b48e4.slice/crio-e79f2a4aad6847fdb56cd7d73708959aa34def1f38d133a6456b187abb2ce0f0 WatchSource:0}: Error finding container e79f2a4aad6847fdb56cd7d73708959aa34def1f38d133a6456b187abb2ce0f0: Status 404 returned error can't find the container with id e79f2a4aad6847fdb56cd7d73708959aa34def1f38d133a6456b187abb2ce0f0
Jul 28 00:53:53.136544 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:53.136451    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda15efe45_733c_4a1c_b3d1_d22f75b18566.slice/crio-61d5efaf366959835f41460d991682e0d0d50ce5d1474e4467274e804da5fa10 WatchSource:0}: Error finding container 61d5efaf366959835f41460d991682e0d0d50ce5d1474e4467274e804da5fa10: Status 404 returned error can't find the container with id 61d5efaf366959835f41460d991682e0d0d50ce5d1474e4467274e804da5fa10
Jul 28 00:53:53.138111 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:53.137605    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2f32b69_b720_4c3c_b345_4cbd68398e2d.slice/crio-4c6987569586f248aec876b6f0c29788cae666a45fc0c98cb25d0f9d5d837714 WatchSource:0}: Error finding container 4c6987569586f248aec876b6f0c29788cae666a45fc0c98cb25d0f9d5d837714: Status 404 returned error can't find the container with id 4c6987569586f248aec876b6f0c29788cae666a45fc0c98cb25d0f9d5d837714
Jul 28 00:53:53.138783 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:53.138412    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7b8125f_f60c_41f4_aca3_bc03804e41b6.slice/crio-49b6ab60753962bf396aaf5968c64c524ac1e004793de7951a85676d220e252d WatchSource:0}: Error finding container 49b6ab60753962bf396aaf5968c64c524ac1e004793de7951a85676d220e252d: Status 404 returned error can't find the container with id 49b6ab60753962bf396aaf5968c64c524ac1e004793de7951a85676d220e252d
Jul 28 00:53:53.139806 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:53.139720    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod894bd991_2d81_4275_96c3_a6464fc1d74a.slice/crio-75414d13b63c2148b600f68211afbdc6271fbf79a7ed925041b9fe36fe262c80 WatchSource:0}: Error finding container 75414d13b63c2148b600f68211afbdc6271fbf79a7ed925041b9fe36fe262c80: Status 404 returned error can't find the container with id 75414d13b63c2148b600f68211afbdc6271fbf79a7ed925041b9fe36fe262c80
Jul 28 00:53:53.141848 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:53:53.141798    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e249ae8_06c0_4082_b90e_a7e54824bea1.slice/crio-78cbc03997ea73ebee54295d72177fc47cd14c3ee2261c9e0aa067c2b91bbb04 WatchSource:0}: Error finding container 78cbc03997ea73ebee54295d72177fc47cd14c3ee2261c9e0aa067c2b91bbb04: Status 404 returned error can't find the container with id 78cbc03997ea73ebee54295d72177fc47cd14c3ee2261c9e0aa067c2b91bbb04
Jul 28 00:53:53.264466 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:53.264245    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 00:48:52 +0000 UTC" deadline="2028-03-31 19:59:58.61137012 +0000 UTC"
Jul 28 00:53:53.264466 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:53.264463    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14707h6m5.346911058s"
Jul 28 00:53:53.396626 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:53.396534    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:53:53.396755 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:53.396648    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvd59" podUID="27305211-7ed9-4558-bd84-f703dcff265a"
Jul 28 00:53:53.403819 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:53.403791    2578 generic.go:356] "Generic (PLEG): container finished" podID="4fb08da72a473db0ae4f0a5d5930c1dc" containerID="021002e50e2aaaf4d70fe039825ccdf52e1f73361423cea2a3cfd630e41fd028" exitCode=0
Jul 28 00:53:53.403923 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:53.403871    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-238.ec2.internal" event={"ID":"4fb08da72a473db0ae4f0a5d5930c1dc","Type":"ContainerDied","Data":"021002e50e2aaaf4d70fe039825ccdf52e1f73361423cea2a3cfd630e41fd028"}
Jul 28 00:53:53.405590 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:53.405565    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6hxnm" event={"ID":"f7b8125f-f60c-41f4-aca3-bc03804e41b6","Type":"ContainerStarted","Data":"49b6ab60753962bf396aaf5968c64c524ac1e004793de7951a85676d220e252d"}
Jul 28 00:53:53.406612 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:53.406588    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xqfwc" event={"ID":"a2f32b69-b720-4c3c-b345-4cbd68398e2d","Type":"ContainerStarted","Data":"4c6987569586f248aec876b6f0c29788cae666a45fc0c98cb25d0f9d5d837714"}
Jul 28 00:53:53.407686 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:53.407667    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-62rn6" event={"ID":"fd844814-e897-46e3-8d93-39785cd0712f","Type":"ContainerStarted","Data":"c0c7039032dd2e83fe69d693c06ba597300a6574be4ce8f4d6bbf0c6b3b7514f"}
Jul 28 00:53:53.411192 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:53.411168    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-238.ec2.internal" event={"ID":"7f112d9abde8daeb5837b42f75048eeb","Type":"ContainerStarted","Data":"9ea80c84675ea4c79dcbd4afb9a82df3a75872a5990093067871c8c15345570a"}
Jul 28 00:53:53.412238 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:53.412207    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm" event={"ID":"2e249ae8-06c0-4082-b90e-a7e54824bea1","Type":"ContainerStarted","Data":"78cbc03997ea73ebee54295d72177fc47cd14c3ee2261c9e0aa067c2b91bbb04"}
Jul 28 00:53:53.413178 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:53.413164    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cgdpl" event={"ID":"894bd991-2d81-4275-96c3-a6464fc1d74a","Type":"ContainerStarted","Data":"75414d13b63c2148b600f68211afbdc6271fbf79a7ed925041b9fe36fe262c80"}
Jul 28 00:53:53.414235 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:53.414202    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t87lr" event={"ID":"a15efe45-733c-4a1c-b3d1-d22f75b18566","Type":"ContainerStarted","Data":"61d5efaf366959835f41460d991682e0d0d50ce5d1474e4467274e804da5fa10"}
Jul 28 00:53:53.415169 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:53.415146    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-28rt8" event={"ID":"b333573a-411c-4999-964f-daa8b82b48e4","Type":"ContainerStarted","Data":"e79f2a4aad6847fdb56cd7d73708959aa34def1f38d133a6456b187abb2ce0f0"}
Jul 28 00:53:53.416140 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:53.416110    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7cgzs" event={"ID":"074938e2-4436-4f39-bf00-6333580a6916","Type":"ContainerStarted","Data":"01f5bee69164b1fea437d98d8f0dc6fe119574222443876633c53a4222d75ff0"}
Jul 28 00:53:53.417047 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:53.417030    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pwmfh" event={"ID":"0d08553a-6760-43c3-bc66-4799fd934954","Type":"ContainerStarted","Data":"be4a5b9958e9866cbed674b6b9d0aedbc4096317e64ea53539a0067a2344005e"}
Jul 28 00:53:53.428889 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:53.428837    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-238.ec2.internal" podStartSLOduration=2.428823863 podStartE2EDuration="2.428823863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 00:53:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:53:53.42839394 +0000 UTC m=+3.593202047" watchObservedRunningTime="2026-07-28 00:53:53.428823863 +0000 UTC m=+3.593631969"
Jul 28 00:53:53.860863 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:53.860772    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27305211-7ed9-4558-bd84-f703dcff265a-original-pull-secret\") pod \"global-pull-secret-syncer-lvd59\" (UID: \"27305211-7ed9-4558-bd84-f703dcff265a\") " pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:53:53.860863 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:53.860838    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knsp9\" (UniqueName: \"kubernetes.io/projected/e5c337e6-650b-4ee1-a24e-3533292e41d1-kube-api-access-knsp9\") pod \"network-check-target-ff2f2\" (UID: \"e5c337e6-650b-4ee1-a24e-3533292e41d1\") " pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:53:53.861099 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:53.860988    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 00:53:53.861099 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:53.861009    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 00:53:53.861099 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:53.861022    2578 projected.go:196] Error preparing data for projected volume kube-api-access-knsp9 for pod openshift-network-diagnostics/network-check-target-ff2f2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 00:53:53.861099 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:53.861088    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e5c337e6-650b-4ee1-a24e-3533292e41d1-kube-api-access-knsp9 podName:e5c337e6-650b-4ee1-a24e-3533292e41d1 nodeName:}" failed. No retries permitted until 2026-07-28 00:53:55.861069719 +0000 UTC m=+6.025877807 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-knsp9" (UniqueName: "kubernetes.io/projected/e5c337e6-650b-4ee1-a24e-3533292e41d1-kube-api-access-knsp9") pod "network-check-target-ff2f2" (UID: "e5c337e6-650b-4ee1-a24e-3533292e41d1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 00:53:53.861513 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:53.861493    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 00:53:53.861589 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:53.861550    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27305211-7ed9-4558-bd84-f703dcff265a-original-pull-secret podName:27305211-7ed9-4558-bd84-f703dcff265a nodeName:}" failed. No retries permitted until 2026-07-28 00:53:55.861535421 +0000 UTC m=+6.026343512 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/27305211-7ed9-4558-bd84-f703dcff265a-original-pull-secret") pod "global-pull-secret-syncer-lvd59" (UID: "27305211-7ed9-4558-bd84-f703dcff265a") : object "kube-system"/"original-pull-secret" not registered
Jul 28 00:53:53.961379 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:53.961342    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d-metrics-certs\") pod \"network-metrics-daemon-gpv8q\" (UID: \"532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d\") " pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:53:53.961554 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:53.961530    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 00:53:53.961665 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:53.961596    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d-metrics-certs podName:532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d nodeName:}" failed. No retries permitted until 2026-07-28 00:53:55.961576386 +0000 UTC m=+6.126384490 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d-metrics-certs") pod "network-metrics-daemon-gpv8q" (UID: "532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 00:53:54.265331 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:54.265241    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 00:48:52 +0000 UTC" deadline="2028-01-14 22:32:09.474164668 +0000 UTC"
Jul 28 00:53:54.265331 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:54.265281    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12861h38m15.20888726s"
Jul 28 00:53:54.396502 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:54.396138    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:53:54.396502 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:54.396288    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ff2f2" podUID="e5c337e6-650b-4ee1-a24e-3533292e41d1"
Jul 28 00:53:54.396502 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:54.396343    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:53:54.396502 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:54.396458    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpv8q" podUID="532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d"
Jul 28 00:53:54.434325 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:54.434284    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-238.ec2.internal" event={"ID":"4fb08da72a473db0ae4f0a5d5930c1dc","Type":"ContainerStarted","Data":"5f3b518767afcc389de66d4539d634e4fd321786018c25fc07257cc2401e46b1"}
Jul 28 00:53:54.449131 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:54.448789    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-238.ec2.internal" podStartSLOduration=3.44876775 podStartE2EDuration="3.44876775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 00:53:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:53:54.44778723 +0000 UTC m=+4.612595338" watchObservedRunningTime="2026-07-28 00:53:54.44876775 +0000 UTC m=+4.613575858"
Jul 28 00:53:55.397028 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:55.396555    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:53:55.397028 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:55.396685    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvd59" podUID="27305211-7ed9-4558-bd84-f703dcff265a"
Jul 28 00:53:55.878562 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:55.878461    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27305211-7ed9-4558-bd84-f703dcff265a-original-pull-secret\") pod \"global-pull-secret-syncer-lvd59\" (UID: \"27305211-7ed9-4558-bd84-f703dcff265a\") " pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:53:55.878562 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:55.878533    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knsp9\" (UniqueName: \"kubernetes.io/projected/e5c337e6-650b-4ee1-a24e-3533292e41d1-kube-api-access-knsp9\") pod \"network-check-target-ff2f2\" (UID: \"e5c337e6-650b-4ee1-a24e-3533292e41d1\") " pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:53:55.878765 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:55.878673    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 00:53:55.878765 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:55.878692    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 00:53:55.878765 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:55.878705    2578 projected.go:196] Error preparing data for projected volume kube-api-access-knsp9 for pod openshift-network-diagnostics/network-check-target-ff2f2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 00:53:55.878765 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:55.878762    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e5c337e6-650b-4ee1-a24e-3533292e41d1-kube-api-access-knsp9 podName:e5c337e6-650b-4ee1-a24e-3533292e41d1 nodeName:}" failed. No retries permitted until 2026-07-28 00:53:59.878743164 +0000 UTC m=+10.043551250 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-knsp9" (UniqueName: "kubernetes.io/projected/e5c337e6-650b-4ee1-a24e-3533292e41d1-kube-api-access-knsp9") pod "network-check-target-ff2f2" (UID: "e5c337e6-650b-4ee1-a24e-3533292e41d1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 00:53:55.879192 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:55.879172    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 00:53:55.879301 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:55.879240    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27305211-7ed9-4558-bd84-f703dcff265a-original-pull-secret podName:27305211-7ed9-4558-bd84-f703dcff265a nodeName:}" failed. No retries permitted until 2026-07-28 00:53:59.879224654 +0000 UTC m=+10.044032740 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/27305211-7ed9-4558-bd84-f703dcff265a-original-pull-secret") pod "global-pull-secret-syncer-lvd59" (UID: "27305211-7ed9-4558-bd84-f703dcff265a") : object "kube-system"/"original-pull-secret" not registered
Jul 28 00:53:55.979917 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:55.979310    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d-metrics-certs\") pod \"network-metrics-daemon-gpv8q\" (UID: \"532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d\") " pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:53:55.979917 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:55.979512    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 00:53:55.979917 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:55.979587    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d-metrics-certs podName:532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d nodeName:}" failed. No retries permitted until 2026-07-28 00:53:59.979568856 +0000 UTC m=+10.144376947 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d-metrics-certs") pod "network-metrics-daemon-gpv8q" (UID: "532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 00:53:56.396091 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:56.396057    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:53:56.396091 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:56.396081    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:53:56.396355 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:56.396248    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ff2f2" podUID="e5c337e6-650b-4ee1-a24e-3533292e41d1"
Jul 28 00:53:56.396355 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:56.396339    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpv8q" podUID="532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d"
Jul 28 00:53:57.396155 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:57.395668    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:53:57.396155 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:57.395798    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvd59" podUID="27305211-7ed9-4558-bd84-f703dcff265a"
Jul 28 00:53:58.397199 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:58.397169    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:53:58.397649 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:58.397326    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpv8q" podUID="532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d"
Jul 28 00:53:58.397649 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:58.397134    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:53:58.397649 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:58.397512    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ff2f2" podUID="e5c337e6-650b-4ee1-a24e-3533292e41d1"
Jul 28 00:53:59.396516 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:59.396484    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:53:59.396682 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:59.396589    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvd59" podUID="27305211-7ed9-4558-bd84-f703dcff265a"
Jul 28 00:53:59.914993 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:59.914957    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knsp9\" (UniqueName: \"kubernetes.io/projected/e5c337e6-650b-4ee1-a24e-3533292e41d1-kube-api-access-knsp9\") pod \"network-check-target-ff2f2\" (UID: \"e5c337e6-650b-4ee1-a24e-3533292e41d1\") " pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:53:59.915477 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:53:59.915056    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27305211-7ed9-4558-bd84-f703dcff265a-original-pull-secret\") pod \"global-pull-secret-syncer-lvd59\" (UID: \"27305211-7ed9-4558-bd84-f703dcff265a\") " pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:53:59.915477 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:59.915177    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 00:53:59.915477 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:59.915255    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27305211-7ed9-4558-bd84-f703dcff265a-original-pull-secret podName:27305211-7ed9-4558-bd84-f703dcff265a nodeName:}" failed. No retries permitted until 2026-07-28 00:54:07.915236483 +0000 UTC m=+18.080044572 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/27305211-7ed9-4558-bd84-f703dcff265a-original-pull-secret") pod "global-pull-secret-syncer-lvd59" (UID: "27305211-7ed9-4558-bd84-f703dcff265a") : object "kube-system"/"original-pull-secret" not registered
Jul 28 00:53:59.915674 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:59.915654    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 00:53:59.915723 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:59.915683    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 00:53:59.915723 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:59.915696    2578 projected.go:196] Error preparing data for projected volume kube-api-access-knsp9 for pod openshift-network-diagnostics/network-check-target-ff2f2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 00:53:59.915796 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:53:59.915751    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e5c337e6-650b-4ee1-a24e-3533292e41d1-kube-api-access-knsp9 podName:e5c337e6-650b-4ee1-a24e-3533292e41d1 nodeName:}" failed. No retries permitted until 2026-07-28 00:54:07.915735353 +0000 UTC m=+18.080543440 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-knsp9" (UniqueName: "kubernetes.io/projected/e5c337e6-650b-4ee1-a24e-3533292e41d1-kube-api-access-knsp9") pod "network-check-target-ff2f2" (UID: "e5c337e6-650b-4ee1-a24e-3533292e41d1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 00:54:00.016068 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:00.016033    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d-metrics-certs\") pod \"network-metrics-daemon-gpv8q\" (UID: \"532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d\") " pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:54:00.016253 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:00.016156    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 00:54:00.016253 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:00.016229    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d-metrics-certs podName:532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d nodeName:}" failed. No retries permitted until 2026-07-28 00:54:08.016197404 +0000 UTC m=+18.181005490 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d-metrics-certs") pod "network-metrics-daemon-gpv8q" (UID: "532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 00:54:00.398314 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:00.397676    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:54:00.398314 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:00.397805    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpv8q" podUID="532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d"
Jul 28 00:54:00.398314 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:00.397873    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:54:00.398314 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:00.397940    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ff2f2" podUID="e5c337e6-650b-4ee1-a24e-3533292e41d1"
Jul 28 00:54:01.396236 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:01.396174    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:54:01.396715 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:01.396329    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvd59" podUID="27305211-7ed9-4558-bd84-f703dcff265a"
Jul 28 00:54:02.396204 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:02.396171    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:54:02.396382 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:02.396323    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ff2f2" podUID="e5c337e6-650b-4ee1-a24e-3533292e41d1"
Jul 28 00:54:02.396382 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:02.396368    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:54:02.396816 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:02.396495    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpv8q" podUID="532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d"
Jul 28 00:54:03.396231 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:03.396199    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:54:03.396328 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:03.396309    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvd59" podUID="27305211-7ed9-4558-bd84-f703dcff265a"
Jul 28 00:54:04.396503 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:04.396471    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:54:04.396942 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:04.396589    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ff2f2" podUID="e5c337e6-650b-4ee1-a24e-3533292e41d1"
Jul 28 00:54:04.396942 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:04.396642    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:54:04.396942 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:04.396742    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpv8q" podUID="532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d"
Jul 28 00:54:05.395770 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:05.395740    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:54:05.395935 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:05.395848    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvd59" podUID="27305211-7ed9-4558-bd84-f703dcff265a"
Jul 28 00:54:06.396034 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:06.395994    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:54:06.396501 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:06.395995    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:54:06.396501 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:06.396125    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpv8q" podUID="532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d"
Jul 28 00:54:06.396501 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:06.396199    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ff2f2" podUID="e5c337e6-650b-4ee1-a24e-3533292e41d1"
Jul 28 00:54:07.396030 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:07.395987    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:54:07.396307 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:07.396124    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvd59" podUID="27305211-7ed9-4558-bd84-f703dcff265a"
Jul 28 00:54:07.974273 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:07.974232    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27305211-7ed9-4558-bd84-f703dcff265a-original-pull-secret\") pod \"global-pull-secret-syncer-lvd59\" (UID: \"27305211-7ed9-4558-bd84-f703dcff265a\") " pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:54:07.974460 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:07.974287    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knsp9\" (UniqueName: \"kubernetes.io/projected/e5c337e6-650b-4ee1-a24e-3533292e41d1-kube-api-access-knsp9\") pod \"network-check-target-ff2f2\" (UID: \"e5c337e6-650b-4ee1-a24e-3533292e41d1\") " pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:54:07.974460 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:07.974394    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 00:54:07.974460 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:07.974407    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 00:54:07.974460 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:07.974424    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 00:54:07.974460 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:07.974435    2578 projected.go:196] Error preparing data for projected volume kube-api-access-knsp9 for pod openshift-network-diagnostics/network-check-target-ff2f2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 00:54:07.974460 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:07.974462    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27305211-7ed9-4558-bd84-f703dcff265a-original-pull-secret podName:27305211-7ed9-4558-bd84-f703dcff265a nodeName:}" failed. No retries permitted until 2026-07-28 00:54:23.974442315 +0000 UTC m=+34.139250411 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/27305211-7ed9-4558-bd84-f703dcff265a-original-pull-secret") pod "global-pull-secret-syncer-lvd59" (UID: "27305211-7ed9-4558-bd84-f703dcff265a") : object "kube-system"/"original-pull-secret" not registered
Jul 28 00:54:07.974721 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:07.974478    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e5c337e6-650b-4ee1-a24e-3533292e41d1-kube-api-access-knsp9 podName:e5c337e6-650b-4ee1-a24e-3533292e41d1 nodeName:}" failed. No retries permitted until 2026-07-28 00:54:23.974470063 +0000 UTC m=+34.139278156 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-knsp9" (UniqueName: "kubernetes.io/projected/e5c337e6-650b-4ee1-a24e-3533292e41d1-kube-api-access-knsp9") pod "network-check-target-ff2f2" (UID: "e5c337e6-650b-4ee1-a24e-3533292e41d1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 00:54:08.074707 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:08.074665    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d-metrics-certs\") pod \"network-metrics-daemon-gpv8q\" (UID: \"532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d\") " pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:54:08.074891 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:08.074840    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 00:54:08.074955 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:08.074922    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d-metrics-certs podName:532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d nodeName:}" failed. No retries permitted until 2026-07-28 00:54:24.074901621 +0000 UTC m=+34.239709722 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d-metrics-certs") pod "network-metrics-daemon-gpv8q" (UID: "532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 00:54:08.396271 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:08.396233    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:54:08.396449 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:08.396233    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:54:08.396449 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:08.396380    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpv8q" podUID="532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d"
Jul 28 00:54:08.396840 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:08.396504    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ff2f2" podUID="e5c337e6-650b-4ee1-a24e-3533292e41d1"
Jul 28 00:54:09.396085 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:09.396050    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:54:09.396290 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:09.396192    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvd59" podUID="27305211-7ed9-4558-bd84-f703dcff265a"
Jul 28 00:54:10.396926 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:10.396892    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:54:10.397473 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:10.397015    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpv8q" podUID="532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d"
Jul 28 00:54:10.397473 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:10.397076    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:54:10.397473 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:10.397187    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ff2f2" podUID="e5c337e6-650b-4ee1-a24e-3533292e41d1"
Jul 28 00:54:11.396464 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:11.396434    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:54:11.396607 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:11.396525    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvd59" podUID="27305211-7ed9-4558-bd84-f703dcff265a"
Jul 28 00:54:12.396519 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:12.396205    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:54:12.397263 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:12.396225    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:54:12.397263 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:12.396621    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ff2f2" podUID="e5c337e6-650b-4ee1-a24e-3533292e41d1"
Jul 28 00:54:12.397263 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:12.396721    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpv8q" podUID="532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d"
Jul 28 00:54:12.473368 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:12.473287    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t87lr" event={"ID":"a15efe45-733c-4a1c-b3d1-d22f75b18566","Type":"ContainerStarted","Data":"1c51bc8e0c6fdb641b4a6a9fa689ba5d2dae2fbdd46f01f45e30b75b4432bac3"}
Jul 28 00:54:12.473517 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:12.473375    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t87lr" event={"ID":"a15efe45-733c-4a1c-b3d1-d22f75b18566","Type":"ContainerStarted","Data":"2be5a43a9a7791829f379fa56721ded85c8077e1845ab745c25bebe0a653cd62"}
Jul 28 00:54:12.473517 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:12.473415    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t87lr" event={"ID":"a15efe45-733c-4a1c-b3d1-d22f75b18566","Type":"ContainerStarted","Data":"5ad0c7f525976b8f83204f6bfc6bf1b87ee32b8798ab643de7d00e9f8619fea7"}
Jul 28 00:54:12.473517 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:12.473444    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t87lr" event={"ID":"a15efe45-733c-4a1c-b3d1-d22f75b18566","Type":"ContainerStarted","Data":"a2a6a61a7eec63925c3c89b0a276c7187bfa8bbb861ef8b631806af351d0afb5"}
Jul 28 00:54:12.473517 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:12.473490    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t87lr" event={"ID":"a15efe45-733c-4a1c-b3d1-d22f75b18566","Type":"ContainerStarted","Data":"be0b4fe1ed2d5df80a9d777b57592f32c9c3862505b4b97cb3ab836a280db62f"}
Jul 28 00:54:12.473760 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:12.473539    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t87lr" event={"ID":"a15efe45-733c-4a1c-b3d1-d22f75b18566","Type":"ContainerStarted","Data":"90134199b8a3a40f429c0f85cf71485ca4cc551d71196e1b8e1a45e1aab74543"}
Jul 28 00:54:12.474762 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:12.474740    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-28rt8" event={"ID":"b333573a-411c-4999-964f-daa8b82b48e4","Type":"ContainerStarted","Data":"6e31794985fc7e1dda46620467cbbc321ab4abaea45c6466ac05cc66159df2f6"}
Jul 28 00:54:12.476268 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:12.476240    2578 generic.go:356] "Generic (PLEG): container finished" podID="074938e2-4436-4f39-bf00-6333580a6916" containerID="f4aaae6fdc3ceb989f0434e432d3fb9bf86e676e074e2f8bf24584b4cc3e3cf8" exitCode=0
Jul 28 00:54:12.476352 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:12.476323    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7cgzs" event={"ID":"074938e2-4436-4f39-bf00-6333580a6916","Type":"ContainerDied","Data":"f4aaae6fdc3ceb989f0434e432d3fb9bf86e676e074e2f8bf24584b4cc3e3cf8"}
Jul 28 00:54:12.477821 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:12.477713    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pwmfh" event={"ID":"0d08553a-6760-43c3-bc66-4799fd934954","Type":"ContainerStarted","Data":"b5eb016c7183f351d6a47cc5857da1ed82b4ad2dc8015678a142a352006587b4"}
Jul 28 00:54:12.479091 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:12.479063    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xqfwc" event={"ID":"a2f32b69-b720-4c3c-b345-4cbd68398e2d","Type":"ContainerStarted","Data":"07135c9c5e8e0b5b2a69b0a2a8d2a250596eb4f9ed5527068df9bb7d3bb132e1"}
Jul 28 00:54:12.480529 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:12.480508    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-62rn6" event={"ID":"fd844814-e897-46e3-8d93-39785cd0712f","Type":"ContainerStarted","Data":"f48e0cc704e7748b16d5f3c9801bf32187f564842b1cb33d598e6080c47e16d0"}
Jul 28 00:54:12.482010 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:12.481990    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm" event={"ID":"2e249ae8-06c0-4082-b90e-a7e54824bea1","Type":"ContainerStarted","Data":"39fd4f9ab0bb6c65ea04fbb99fae6fd20cae8e966a04d59c25363ba55e39792d"}
Jul 28 00:54:12.483453 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:12.483432    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cgdpl" event={"ID":"894bd991-2d81-4275-96c3-a6464fc1d74a","Type":"ContainerStarted","Data":"14d3165cad6e06390e3e4829eb292c24381dfb0af8f890a2928767f6d59a62d2"}
Jul 28 00:54:12.493429 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:12.493394    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-28rt8" podStartSLOduration=3.6769945809999998 podStartE2EDuration="22.493383439s" totalImagesPullingTime="18.816388858s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 00:53:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:54:12.493248997 +0000 UTC m=+22.658057123" watchObservedRunningTime="2026-07-28 00:54:12.493383439 +0000 UTC m=+22.658191545"
Jul 28 00:54:12.507903 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:12.507862    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-62rn6" podStartSLOduration=3.91756965 podStartE2EDuration="22.507851171s" totalImagesPullingTime="18.590281521s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 00:53:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:54:12.507463889 +0000 UTC m=+22.672271992" watchObservedRunningTime="2026-07-28 00:54:12.507851171 +0000 UTC m=+22.672659275"
Jul 28 00:54:12.520138 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:12.520102    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-cgdpl" podStartSLOduration=3.936818966 podStartE2EDuration="22.520093631s" totalImagesPullingTime="18.583274665s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 00:53:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:54:12.519892961 +0000 UTC m=+22.684701068" watchObservedRunningTime="2026-07-28 00:54:12.520093631 +0000 UTC m=+22.684901738"
Jul 28 00:54:12.532855 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:12.532817    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xqfwc" podStartSLOduration=12.249637861 podStartE2EDuration="22.532801512s" totalImagesPullingTime="10.283163651s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:53:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:54:12.532665245 +0000 UTC m=+22.697473352" watchObservedRunningTime="2026-07-28 00:54:12.532801512 +0000 UTC m=+22.697609619"
Jul 28 00:54:12.548065 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:12.548023    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-pwmfh" podStartSLOduration=3.954564236 podStartE2EDuration="22.548006951s" totalImagesPullingTime="18.593442715s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 00:53:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:54:12.547837992 +0000 UTC m=+22.712646119" watchObservedRunningTime="2026-07-28 00:54:12.548006951 +0000 UTC m=+22.712815059"
Jul 28 00:54:13.001651 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:13.001576    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xqfwc"
Jul 28 00:54:13.256331 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:13.256180    2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 00:54:13.296669 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:13.296579    2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T00:54:13.256199894Z","UUID":"4343f752-e87d-4013-8cf3-03c5ba72d64d","Handler":null,"Name":"","Endpoint":""}
Jul 28 00:54:13.298051 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:13.298031    2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 00:54:13.298168 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:13.298072    2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 00:54:13.395882 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:13.395850    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:54:13.396068 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:13.395986    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvd59" podUID="27305211-7ed9-4558-bd84-f703dcff265a"
Jul 28 00:54:13.428043 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:13.428014    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xqfwc"
Jul 28 00:54:13.428654 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:13.428635    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xqfwc"
Jul 28 00:54:13.487665 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:13.487628    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6hxnm" event={"ID":"f7b8125f-f60c-41f4-aca3-bc03804e41b6","Type":"ContainerStarted","Data":"aa5098d74ec7f2eea8bdbc2c3ae4d875ae4f191e1b339be9ba48bde863315b15"}
Jul 28 00:54:13.491475 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:13.491345    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm" event={"ID":"2e249ae8-06c0-4082-b90e-a7e54824bea1","Type":"ContainerStarted","Data":"15a233b8ed8769421894c7f39d1bf00f839398110103ac95fbac9acad52b2e39"}
Jul 28 00:54:13.492621 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:13.492598    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xqfwc"
Jul 28 00:54:13.503319 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:13.503279    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6hxnm" podStartSLOduration=4.938032205 podStartE2EDuration="23.503263195s" totalImagesPullingTime="18.56523099s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 00:53:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:54:13.503159041 +0000 UTC m=+23.667967149" watchObservedRunningTime="2026-07-28 00:54:13.503263195 +0000 UTC m=+23.668071376"
Jul 28 00:54:14.397109 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:14.396626    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:54:14.397109 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:14.396675    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:54:14.397109 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:14.396766    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpv8q" podUID="532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d"
Jul 28 00:54:14.397109 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:14.396896    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ff2f2" podUID="e5c337e6-650b-4ee1-a24e-3533292e41d1"
Jul 28 00:54:15.396849 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:15.396666    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:54:15.397366 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:15.396957    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvd59" podUID="27305211-7ed9-4558-bd84-f703dcff265a"
Jul 28 00:54:15.498037 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:15.497997    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm" event={"ID":"2e249ae8-06c0-4082-b90e-a7e54824bea1","Type":"ContainerStarted","Data":"1b2ddbd40b6abc6e3ad6876e79b788758e1f4d479d5a811f2b4c44cb82523ace"}
Jul 28 00:54:15.501732 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:15.501694    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t87lr" event={"ID":"a15efe45-733c-4a1c-b3d1-d22f75b18566","Type":"ContainerStarted","Data":"ef525f1b6cdcaf6b772df3840c122efcded5e624613c5ede9742236b4b62ea05"}
Jul 28 00:54:15.514964 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:15.514909    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-64gsm" podStartSLOduration=4.541043144 podStartE2EDuration="25.514889113s" totalImagesPullingTime="20.973845969s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 00:53:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:54:15.514451917 +0000 UTC m=+25.679260031" watchObservedRunningTime="2026-07-28 00:54:15.514889113 +0000 UTC m=+25.679697264"
Jul 28 00:54:16.396242 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:16.396198    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:54:16.396467 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:16.396361    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:54:16.396467 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:16.396441    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ff2f2" podUID="e5c337e6-650b-4ee1-a24e-3533292e41d1"
Jul 28 00:54:16.396467 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:16.396351    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpv8q" podUID="532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d"
Jul 28 00:54:17.396736 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:17.396552    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:54:17.397437 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:17.396808    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvd59" podUID="27305211-7ed9-4558-bd84-f703dcff265a"
Jul 28 00:54:17.508343 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:17.508311    2578 generic.go:356] "Generic (PLEG): container finished" podID="074938e2-4436-4f39-bf00-6333580a6916" containerID="d77966c407479423b5c051ae3a506fa08d20e822716583dc779d22bb9c89bbcb" exitCode=0
Jul 28 00:54:17.508513 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:17.508399    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7cgzs" event={"ID":"074938e2-4436-4f39-bf00-6333580a6916","Type":"ContainerDied","Data":"d77966c407479423b5c051ae3a506fa08d20e822716583dc779d22bb9c89bbcb"}
Jul 28 00:54:17.511966 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:17.511921    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t87lr" event={"ID":"a15efe45-733c-4a1c-b3d1-d22f75b18566","Type":"ContainerStarted","Data":"60357e8cf1ebb423fc1ec75126450f0a0b2bfead4f0502b6fa9bf59e7803159c"}
Jul 28 00:54:17.512224 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:17.512194    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:54:17.512308 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:17.512282    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:54:17.527274 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:17.527248    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:54:17.559281 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:17.559231    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-t87lr" podStartSLOduration=8.921827226 podStartE2EDuration="27.559195809s" totalImagesPullingTime="18.637368583s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 00:53:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:54:17.557893164 +0000 UTC m=+27.722701291" watchObservedRunningTime="2026-07-28 00:54:17.559195809 +0000 UTC m=+27.724003916"
Jul 28 00:54:18.395659 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:18.395632    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:54:18.395894 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:18.395639    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:54:18.395894 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:18.395758    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpv8q" podUID="532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d"
Jul 28 00:54:18.395894 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:18.395791    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ff2f2" podUID="e5c337e6-650b-4ee1-a24e-3533292e41d1"
Jul 28 00:54:18.514338 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:18.514267    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:54:18.530032 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:18.530011    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:54:19.396265 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:19.396232    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:54:19.396423 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:19.396338    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvd59" podUID="27305211-7ed9-4558-bd84-f703dcff265a"
Jul 28 00:54:19.517372 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:19.517340    2578 generic.go:356] "Generic (PLEG): container finished" podID="074938e2-4436-4f39-bf00-6333580a6916" containerID="7f7095c0b1f4d9047583a6c23a030c94011a1dae65e1d4eedd13b2da283bb657" exitCode=0
Jul 28 00:54:19.517803 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:19.517419    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7cgzs" event={"ID":"074938e2-4436-4f39-bf00-6333580a6916","Type":"ContainerDied","Data":"7f7095c0b1f4d9047583a6c23a030c94011a1dae65e1d4eedd13b2da283bb657"}
Jul 28 00:54:20.397188 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:20.397153    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:54:20.397374 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:20.397279    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ff2f2" podUID="e5c337e6-650b-4ee1-a24e-3533292e41d1"
Jul 28 00:54:20.397374 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:20.397329    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:54:20.397456 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:20.397433    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpv8q" podUID="532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d"
Jul 28 00:54:21.396284 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:21.396248    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:54:21.396630 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:21.396351    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvd59" podUID="27305211-7ed9-4558-bd84-f703dcff265a"
Jul 28 00:54:21.524061 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:21.524025    2578 generic.go:356] "Generic (PLEG): container finished" podID="074938e2-4436-4f39-bf00-6333580a6916" containerID="5fccd5cf8ad2e9f8ba0bc711ed4f6c4c2b2ced0eaf20f0f47f44d5dc0efa1424" exitCode=0
Jul 28 00:54:21.524239 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:21.524105    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7cgzs" event={"ID":"074938e2-4436-4f39-bf00-6333580a6916","Type":"ContainerDied","Data":"5fccd5cf8ad2e9f8ba0bc711ed4f6c4c2b2ced0eaf20f0f47f44d5dc0efa1424"}
Jul 28 00:54:22.396668 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:22.396632    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:54:22.397134 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:22.396644    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:54:22.397134 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:22.396759    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ff2f2" podUID="e5c337e6-650b-4ee1-a24e-3533292e41d1"
Jul 28 00:54:22.397134 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:22.396864    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpv8q" podUID="532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d"
Jul 28 00:54:23.396319 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:23.396288    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:54:23.396511 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:23.396389    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvd59" podUID="27305211-7ed9-4558-bd84-f703dcff265a"
Jul 28 00:54:24.000321 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:24.000280    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knsp9\" (UniqueName: \"kubernetes.io/projected/e5c337e6-650b-4ee1-a24e-3533292e41d1-kube-api-access-knsp9\") pod \"network-check-target-ff2f2\" (UID: \"e5c337e6-650b-4ee1-a24e-3533292e41d1\") " pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:54:24.000892 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:24.000371    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27305211-7ed9-4558-bd84-f703dcff265a-original-pull-secret\") pod \"global-pull-secret-syncer-lvd59\" (UID: \"27305211-7ed9-4558-bd84-f703dcff265a\") " pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:54:24.000892 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:24.000458    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 00:54:24.000892 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:24.000480    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 00:54:24.000892 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:24.000492    2578 projected.go:196] Error preparing data for projected volume kube-api-access-knsp9 for pod openshift-network-diagnostics/network-check-target-ff2f2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 00:54:24.000892 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:24.000463    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 00:54:24.000892 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:24.000548    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e5c337e6-650b-4ee1-a24e-3533292e41d1-kube-api-access-knsp9 podName:e5c337e6-650b-4ee1-a24e-3533292e41d1 nodeName:}" failed. No retries permitted until 2026-07-28 00:54:56.000529897 +0000 UTC m=+66.165338009 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-knsp9" (UniqueName: "kubernetes.io/projected/e5c337e6-650b-4ee1-a24e-3533292e41d1-kube-api-access-knsp9") pod "network-check-target-ff2f2" (UID: "e5c337e6-650b-4ee1-a24e-3533292e41d1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 00:54:24.000892 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:24.000577    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27305211-7ed9-4558-bd84-f703dcff265a-original-pull-secret podName:27305211-7ed9-4558-bd84-f703dcff265a nodeName:}" failed. No retries permitted until 2026-07-28 00:54:56.000566182 +0000 UTC m=+66.165374269 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/27305211-7ed9-4558-bd84-f703dcff265a-original-pull-secret") pod "global-pull-secret-syncer-lvd59" (UID: "27305211-7ed9-4558-bd84-f703dcff265a") : object "kube-system"/"original-pull-secret" not registered
Jul 28 00:54:24.101075 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:24.101036    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d-metrics-certs\") pod \"network-metrics-daemon-gpv8q\" (UID: \"532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d\") " pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:54:24.101262 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:24.101183    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 00:54:24.101262 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:24.101252    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d-metrics-certs podName:532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d nodeName:}" failed. No retries permitted until 2026-07-28 00:54:56.101237431 +0000 UTC m=+66.266045519 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d-metrics-certs") pod "network-metrics-daemon-gpv8q" (UID: "532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 00:54:24.399867 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:24.399837    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:54:24.400057 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:24.399885    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:54:24.400057 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:24.399962    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ff2f2" podUID="e5c337e6-650b-4ee1-a24e-3533292e41d1"
Jul 28 00:54:24.400178 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:24.400110    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpv8q" podUID="532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d"
Jul 28 00:54:25.395754 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:25.395718    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:54:25.396098 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:25.395857    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvd59" podUID="27305211-7ed9-4558-bd84-f703dcff265a"
Jul 28 00:54:25.536400 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:25.536368    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7cgzs" event={"ID":"074938e2-4436-4f39-bf00-6333580a6916","Type":"ContainerStarted","Data":"49ace3d71bd0d6866bb76c9fab184b6e390fb625d537a2f229048e2a99f8aac0"}
Jul 28 00:54:26.396135 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:26.396102    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:54:26.396519 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:26.396234    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpv8q" podUID="532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d"
Jul 28 00:54:26.396519 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:26.396272    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:54:26.396519 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:26.396358    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ff2f2" podUID="e5c337e6-650b-4ee1-a24e-3533292e41d1"
Jul 28 00:54:26.540540 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:26.540507    2578 generic.go:356] "Generic (PLEG): container finished" podID="074938e2-4436-4f39-bf00-6333580a6916" containerID="49ace3d71bd0d6866bb76c9fab184b6e390fb625d537a2f229048e2a99f8aac0" exitCode=0
Jul 28 00:54:26.540695 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:26.540559    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7cgzs" event={"ID":"074938e2-4436-4f39-bf00-6333580a6916","Type":"ContainerDied","Data":"49ace3d71bd0d6866bb76c9fab184b6e390fb625d537a2f229048e2a99f8aac0"}
Jul 28 00:54:27.395945 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:27.395909    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:54:27.396123 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:27.396024    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvd59" podUID="27305211-7ed9-4558-bd84-f703dcff265a"
Jul 28 00:54:27.545487 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:27.545452    2578 generic.go:356] "Generic (PLEG): container finished" podID="074938e2-4436-4f39-bf00-6333580a6916" containerID="dfbc42d0815f5e89377107980d8d61d9dc4364606007b409b180fba89036e3a6" exitCode=0
Jul 28 00:54:27.545882 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:27.545507    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7cgzs" event={"ID":"074938e2-4436-4f39-bf00-6333580a6916","Type":"ContainerDied","Data":"dfbc42d0815f5e89377107980d8d61d9dc4364606007b409b180fba89036e3a6"}
Jul 28 00:54:28.395712 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:28.395674    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:54:28.395904 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:28.395815    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpv8q" podUID="532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d"
Jul 28 00:54:28.395904 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:28.395863    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:54:28.396082 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:28.395947    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ff2f2" podUID="e5c337e6-650b-4ee1-a24e-3533292e41d1"
Jul 28 00:54:28.550552 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:28.550521    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7cgzs" event={"ID":"074938e2-4436-4f39-bf00-6333580a6916","Type":"ContainerStarted","Data":"3aa63dfd91579cb966f75a4984e41ce298569f2ec9caf3b3b6602eef6d68f0d9"}
Jul 28 00:54:28.573123 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:28.573081    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7cgzs" podStartSLOduration=9.639060799 podStartE2EDuration="38.573057004s" totalImagesPullingTime="28.933996205s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 00:53:50 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:54:28.57142322 +0000 UTC m=+38.736231328" watchObservedRunningTime="2026-07-28 00:54:28.573057004 +0000 UTC m=+38.737865110"
Jul 28 00:54:29.395779 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:29.395749    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:54:29.395939 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:29.395853    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvd59" podUID="27305211-7ed9-4558-bd84-f703dcff265a"
Jul 28 00:54:30.396714 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:30.396681    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:54:30.397116 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:30.396770    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpv8q" podUID="532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d"
Jul 28 00:54:30.397116 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:30.396822    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:54:30.397116 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:30.396902    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ff2f2" podUID="e5c337e6-650b-4ee1-a24e-3533292e41d1"
Jul 28 00:54:31.396021 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:31.395992    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:54:31.396195 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:31.396096    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvd59" podUID="27305211-7ed9-4558-bd84-f703dcff265a"
Jul 28 00:54:32.396269 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:32.396240    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:54:32.396733 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:32.396284    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:54:32.396733 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:32.396378    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ff2f2" podUID="e5c337e6-650b-4ee1-a24e-3533292e41d1"
Jul 28 00:54:32.396733 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:32.396503    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpv8q" podUID="532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d"
Jul 28 00:54:33.161590 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:33.161557    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lvd59"]
Jul 28 00:54:33.161753 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:33.161705    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:54:33.161845 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:33.161823    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvd59" podUID="27305211-7ed9-4558-bd84-f703dcff265a"
Jul 28 00:54:33.169588 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:33.169561    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gpv8q"]
Jul 28 00:54:33.169859 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:33.169827    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:54:33.169967 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:33.169945    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpv8q" podUID="532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d"
Jul 28 00:54:33.182524 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:33.182497    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ff2f2"]
Jul 28 00:54:33.182643 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:33.182589    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:54:33.182695 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:33.182678    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ff2f2" podUID="e5c337e6-650b-4ee1-a24e-3533292e41d1"
Jul 28 00:54:34.396310 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:34.396274    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:54:34.397226 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:34.396402    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ff2f2" podUID="e5c337e6-650b-4ee1-a24e-3533292e41d1"
Jul 28 00:54:34.397226 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:34.396444    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:54:34.397226 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:34.396541    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpv8q" podUID="532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d"
Jul 28 00:54:35.395706 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:35.395664    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:54:35.395914 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:35.395778    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lvd59" podUID="27305211-7ed9-4558-bd84-f703dcff265a"
Jul 28 00:54:36.395753 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.395722    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:54:36.396185 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.395730    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:54:36.396185 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:36.395828    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ff2f2" podUID="e5c337e6-650b-4ee1-a24e-3533292e41d1"
Jul 28 00:54:36.396185 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:36.395896    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpv8q" podUID="532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d"
Jul 28 00:54:36.672358 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.672288    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-238.ec2.internal" event="NodeReady"
Jul 28 00:54:36.672489 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.672418    2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 00:54:36.713152 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.713117    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ptgdm"]
Jul 28 00:54:36.718294 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.718273    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ptgdm"
Jul 28 00:54:36.719324 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.719299    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-46nnl"]
Jul 28 00:54:36.720660 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.720642    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 00:54:36.720763 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.720728    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 00:54:36.720801 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.720773    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 00:54:36.720842 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.720825    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-s4cm9\""
Jul 28 00:54:36.734461 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.734433    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ptgdm"]
Jul 28 00:54:36.734590 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.734478    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-46nnl"]
Jul 28 00:54:36.734590 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.734569    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-46nnl"
Jul 28 00:54:36.734915 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.734852    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dbcvj"]
Jul 28 00:54:36.736842 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.736819    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 00:54:36.736974 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.736914    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 00:54:36.736974 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.736939    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 00:54:36.737281 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.737197    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-r566t\""
Jul 28 00:54:36.737434 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.737417    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 00:54:36.750556 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.750521    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dbcvj"]
Jul 28 00:54:36.750696 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.750675    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dbcvj"
Jul 28 00:54:36.754320 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.754299    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 00:54:36.754320 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.754312    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 00:54:36.754517 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.754443    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-w6lql\""
Jul 28 00:54:36.793603 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.793565    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/19425b8d-e929-41df-824d-49b9ec05ec72-config-volume\") pod \"dns-default-dbcvj\" (UID: \"19425b8d-e929-41df-824d-49b9ec05ec72\") " pod="openshift-dns/dns-default-dbcvj"
Jul 28 00:54:36.793603 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.793600    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b87dd1d9-c9dd-45c5-a9f5-f8eb70385708-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-46nnl\" (UID: \"b87dd1d9-c9dd-45c5-a9f5-f8eb70385708\") " pod="openshift-insights/insights-runtime-extractor-46nnl"
Jul 28 00:54:36.793816 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.793624    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c32f7a50-f215-4648-a7bc-8cf984ef6fb0-cert\") pod \"ingress-canary-ptgdm\" (UID: \"c32f7a50-f215-4648-a7bc-8cf984ef6fb0\") " pod="openshift-ingress-canary/ingress-canary-ptgdm"
Jul 28 00:54:36.793816 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.793639    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g25qc\" (UniqueName: \"kubernetes.io/projected/c32f7a50-f215-4648-a7bc-8cf984ef6fb0-kube-api-access-g25qc\") pod \"ingress-canary-ptgdm\" (UID: \"c32f7a50-f215-4648-a7bc-8cf984ef6fb0\") " pod="openshift-ingress-canary/ingress-canary-ptgdm"
Jul 28 00:54:36.793816 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.793719    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b87dd1d9-c9dd-45c5-a9f5-f8eb70385708-data-volume\") pod \"insights-runtime-extractor-46nnl\" (UID: \"b87dd1d9-c9dd-45c5-a9f5-f8eb70385708\") " pod="openshift-insights/insights-runtime-extractor-46nnl"
Jul 28 00:54:36.793816 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.793747    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19425b8d-e929-41df-824d-49b9ec05ec72-metrics-tls\") pod \"dns-default-dbcvj\" (UID: \"19425b8d-e929-41df-824d-49b9ec05ec72\") " pod="openshift-dns/dns-default-dbcvj"
Jul 28 00:54:36.793816 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.793805    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19425b8d-e929-41df-824d-49b9ec05ec72-tmp-dir\") pod \"dns-default-dbcvj\" (UID: \"19425b8d-e929-41df-824d-49b9ec05ec72\") " pod="openshift-dns/dns-default-dbcvj"
Jul 28 00:54:36.793977 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.793826    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b87dd1d9-c9dd-45c5-a9f5-f8eb70385708-crio-socket\") pod \"insights-runtime-extractor-46nnl\" (UID: \"b87dd1d9-c9dd-45c5-a9f5-f8eb70385708\") " pod="openshift-insights/insights-runtime-extractor-46nnl"
Jul 28 00:54:36.793977 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.793849    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b87dd1d9-c9dd-45c5-a9f5-f8eb70385708-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-46nnl\" (UID: \"b87dd1d9-c9dd-45c5-a9f5-f8eb70385708\") " pod="openshift-insights/insights-runtime-extractor-46nnl"
Jul 28 00:54:36.793977 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.793879    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvzzz\" (UniqueName: \"kubernetes.io/projected/b87dd1d9-c9dd-45c5-a9f5-f8eb70385708-kube-api-access-cvzzz\") pod \"insights-runtime-extractor-46nnl\" (UID: \"b87dd1d9-c9dd-45c5-a9f5-f8eb70385708\") " pod="openshift-insights/insights-runtime-extractor-46nnl"
Jul 28 00:54:36.793977 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.793895    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sd72\" (UniqueName: \"kubernetes.io/projected/19425b8d-e929-41df-824d-49b9ec05ec72-kube-api-access-4sd72\") pod \"dns-default-dbcvj\" (UID: \"19425b8d-e929-41df-824d-49b9ec05ec72\") " pod="openshift-dns/dns-default-dbcvj"
Jul 28 00:54:36.894512 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.894473    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19425b8d-e929-41df-824d-49b9ec05ec72-tmp-dir\") pod \"dns-default-dbcvj\" (UID: \"19425b8d-e929-41df-824d-49b9ec05ec72\") " pod="openshift-dns/dns-default-dbcvj"
Jul 28 00:54:36.894512 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.894515    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b87dd1d9-c9dd-45c5-a9f5-f8eb70385708-crio-socket\") pod \"insights-runtime-extractor-46nnl\" (UID: \"b87dd1d9-c9dd-45c5-a9f5-f8eb70385708\") " pod="openshift-insights/insights-runtime-extractor-46nnl"
Jul 28 00:54:36.894728 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.894531    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b87dd1d9-c9dd-45c5-a9f5-f8eb70385708-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-46nnl\" (UID: \"b87dd1d9-c9dd-45c5-a9f5-f8eb70385708\") " pod="openshift-insights/insights-runtime-extractor-46nnl"
Jul 28 00:54:36.894728 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.894558    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvzzz\" (UniqueName: \"kubernetes.io/projected/b87dd1d9-c9dd-45c5-a9f5-f8eb70385708-kube-api-access-cvzzz\") pod \"insights-runtime-extractor-46nnl\" (UID: \"b87dd1d9-c9dd-45c5-a9f5-f8eb70385708\") " pod="openshift-insights/insights-runtime-extractor-46nnl"
Jul 28 00:54:36.894728 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.894575    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sd72\" (UniqueName: \"kubernetes.io/projected/19425b8d-e929-41df-824d-49b9ec05ec72-kube-api-access-4sd72\") pod \"dns-default-dbcvj\" (UID: \"19425b8d-e929-41df-824d-49b9ec05ec72\") " pod="openshift-dns/dns-default-dbcvj"
Jul 28 00:54:36.894728 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.894607    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/19425b8d-e929-41df-824d-49b9ec05ec72-config-volume\") pod \"dns-default-dbcvj\" (UID: \"19425b8d-e929-41df-824d-49b9ec05ec72\") " pod="openshift-dns/dns-default-dbcvj"
Jul 28 00:54:36.894728 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.894631    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b87dd1d9-c9dd-45c5-a9f5-f8eb70385708-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-46nnl\" (UID: \"b87dd1d9-c9dd-45c5-a9f5-f8eb70385708\") " pod="openshift-insights/insights-runtime-extractor-46nnl"
Jul 28 00:54:36.894728 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.894661    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c32f7a50-f215-4648-a7bc-8cf984ef6fb0-cert\") pod \"ingress-canary-ptgdm\" (UID: \"c32f7a50-f215-4648-a7bc-8cf984ef6fb0\") " pod="openshift-ingress-canary/ingress-canary-ptgdm"
Jul 28 00:54:36.894728 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.894683    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g25qc\" (UniqueName: \"kubernetes.io/projected/c32f7a50-f215-4648-a7bc-8cf984ef6fb0-kube-api-access-g25qc\") pod \"ingress-canary-ptgdm\" (UID: \"c32f7a50-f215-4648-a7bc-8cf984ef6fb0\") " pod="openshift-ingress-canary/ingress-canary-ptgdm"
Jul 28 00:54:36.894728 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.894719    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b87dd1d9-c9dd-45c5-a9f5-f8eb70385708-data-volume\") pod \"insights-runtime-extractor-46nnl\" (UID: \"b87dd1d9-c9dd-45c5-a9f5-f8eb70385708\") " pod="openshift-insights/insights-runtime-extractor-46nnl"
Jul 28 00:54:36.895101 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.894734    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b87dd1d9-c9dd-45c5-a9f5-f8eb70385708-crio-socket\") pod \"insights-runtime-extractor-46nnl\" (UID: \"b87dd1d9-c9dd-45c5-a9f5-f8eb70385708\") " pod="openshift-insights/insights-runtime-extractor-46nnl"
Jul 28 00:54:36.895101 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.894893    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19425b8d-e929-41df-824d-49b9ec05ec72-tmp-dir\") pod \"dns-default-dbcvj\" (UID: \"19425b8d-e929-41df-824d-49b9ec05ec72\") " pod="openshift-dns/dns-default-dbcvj"
Jul 28 00:54:36.895101 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.894742    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19425b8d-e929-41df-824d-49b9ec05ec72-metrics-tls\") pod \"dns-default-dbcvj\" (UID: \"19425b8d-e929-41df-824d-49b9ec05ec72\") " pod="openshift-dns/dns-default-dbcvj"
Jul 28 00:54:36.895280 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.895135    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b87dd1d9-c9dd-45c5-a9f5-f8eb70385708-data-volume\") pod \"insights-runtime-extractor-46nnl\" (UID: \"b87dd1d9-c9dd-45c5-a9f5-f8eb70385708\") " pod="openshift-insights/insights-runtime-extractor-46nnl"
Jul 28 00:54:36.895434 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.895414    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b87dd1d9-c9dd-45c5-a9f5-f8eb70385708-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-46nnl\" (UID: \"b87dd1d9-c9dd-45c5-a9f5-f8eb70385708\") " pod="openshift-insights/insights-runtime-extractor-46nnl"
Jul 28 00:54:36.895514 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.895496    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/19425b8d-e929-41df-824d-49b9ec05ec72-config-volume\") pod \"dns-default-dbcvj\" (UID: \"19425b8d-e929-41df-824d-49b9ec05ec72\") " pod="openshift-dns/dns-default-dbcvj"
Jul 28 00:54:36.898728 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.898590    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19425b8d-e929-41df-824d-49b9ec05ec72-metrics-tls\") pod \"dns-default-dbcvj\" (UID: \"19425b8d-e929-41df-824d-49b9ec05ec72\") " pod="openshift-dns/dns-default-dbcvj"
Jul 28 00:54:36.898728 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.898714    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b87dd1d9-c9dd-45c5-a9f5-f8eb70385708-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-46nnl\" (UID: \"b87dd1d9-c9dd-45c5-a9f5-f8eb70385708\") " pod="openshift-insights/insights-runtime-extractor-46nnl"
Jul 28 00:54:36.898864 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.898763    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c32f7a50-f215-4648-a7bc-8cf984ef6fb0-cert\") pod \"ingress-canary-ptgdm\" (UID: \"c32f7a50-f215-4648-a7bc-8cf984ef6fb0\") " pod="openshift-ingress-canary/ingress-canary-ptgdm"
Jul 28 00:54:36.902138 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.902116    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvzzz\" (UniqueName: \"kubernetes.io/projected/b87dd1d9-c9dd-45c5-a9f5-f8eb70385708-kube-api-access-cvzzz\") pod \"insights-runtime-extractor-46nnl\" (UID: \"b87dd1d9-c9dd-45c5-a9f5-f8eb70385708\") " pod="openshift-insights/insights-runtime-extractor-46nnl"
Jul 28 00:54:36.902340 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.902320    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sd72\" (UniqueName: \"kubernetes.io/projected/19425b8d-e929-41df-824d-49b9ec05ec72-kube-api-access-4sd72\") pod \"dns-default-dbcvj\" (UID: \"19425b8d-e929-41df-824d-49b9ec05ec72\") " pod="openshift-dns/dns-default-dbcvj"
Jul 28 00:54:36.902385 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:36.902335    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g25qc\" (UniqueName: \"kubernetes.io/projected/c32f7a50-f215-4648-a7bc-8cf984ef6fb0-kube-api-access-g25qc\") pod \"ingress-canary-ptgdm\" (UID: \"c32f7a50-f215-4648-a7bc-8cf984ef6fb0\") " pod="openshift-ingress-canary/ingress-canary-ptgdm"
Jul 28 00:54:37.043620 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:37.043520    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-46nnl"
Jul 28 00:54:37.043620 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:37.043546    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ptgdm"
Jul 28 00:54:37.061728 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:37.061685    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dbcvj"
Jul 28 00:54:37.254315 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:37.254272    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-46nnl"]
Jul 28 00:54:37.259389 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:54:37.259364    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb87dd1d9_c9dd_45c5_a9f5_f8eb70385708.slice/crio-67d3a5773f707b5364e8c182c60ed5f2db9cf09296443b72c256e11c4fb8f99b WatchSource:0}: Error finding container 67d3a5773f707b5364e8c182c60ed5f2db9cf09296443b72c256e11c4fb8f99b: Status 404 returned error can't find the container with id 67d3a5773f707b5364e8c182c60ed5f2db9cf09296443b72c256e11c4fb8f99b
Jul 28 00:54:37.262060 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:37.262014    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ptgdm"]
Jul 28 00:54:37.265286 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:54:37.265266    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc32f7a50_f215_4648_a7bc_8cf984ef6fb0.slice/crio-7c264de4a1e20fdf79f74eb823cc1f3616776e29bdf5ba58304879759c4ee1a5 WatchSource:0}: Error finding container 7c264de4a1e20fdf79f74eb823cc1f3616776e29bdf5ba58304879759c4ee1a5: Status 404 returned error can't find the container with id 7c264de4a1e20fdf79f74eb823cc1f3616776e29bdf5ba58304879759c4ee1a5
Jul 28 00:54:37.277851 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:37.277828    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dbcvj"]
Jul 28 00:54:37.281173 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:54:37.281150    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19425b8d_e929_41df_824d_49b9ec05ec72.slice/crio-d4f83e28bfa6218fd493dde2a353b0a7b09a719eb9e6045263377f6bb89b54f4 WatchSource:0}: Error finding container d4f83e28bfa6218fd493dde2a353b0a7b09a719eb9e6045263377f6bb89b54f4: Status 404 returned error can't find the container with id d4f83e28bfa6218fd493dde2a353b0a7b09a719eb9e6045263377f6bb89b54f4
Jul 28 00:54:37.396245 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:37.396194    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:54:37.398606 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:37.398588    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 00:54:37.459986 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:37.459951    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9xv8h"]
Jul 28 00:54:37.479925 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:37.479899    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9xv8h"]
Jul 28 00:54:37.480073 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:37.480006    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9xv8h"
Jul 28 00:54:37.482289 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:37.482268    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 28 00:54:37.482424 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:37.482412    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-9z9cw\""
Jul 28 00:54:37.499026 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:37.499002    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c528c7c4-3a25-4163-8375-922fbe1c07e3-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-9xv8h\" (UID: \"c528c7c4-3a25-4163-8375-922fbe1c07e3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9xv8h"
Jul 28 00:54:37.572304 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:37.572270    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-46nnl" event={"ID":"b87dd1d9-c9dd-45c5-a9f5-f8eb70385708","Type":"ContainerStarted","Data":"67d3a5773f707b5364e8c182c60ed5f2db9cf09296443b72c256e11c4fb8f99b"}
Jul 28 00:54:37.573296 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:37.573268    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dbcvj" event={"ID":"19425b8d-e929-41df-824d-49b9ec05ec72","Type":"ContainerStarted","Data":"d4f83e28bfa6218fd493dde2a353b0a7b09a719eb9e6045263377f6bb89b54f4"}
Jul 28 00:54:37.574133 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:37.574112    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ptgdm" event={"ID":"c32f7a50-f215-4648-a7bc-8cf984ef6fb0","Type":"ContainerStarted","Data":"7c264de4a1e20fdf79f74eb823cc1f3616776e29bdf5ba58304879759c4ee1a5"}
Jul 28 00:54:37.600066 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:37.600034    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c528c7c4-3a25-4163-8375-922fbe1c07e3-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-9xv8h\" (UID: \"c528c7c4-3a25-4163-8375-922fbe1c07e3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9xv8h"
Jul 28 00:54:37.603274 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:37.603245    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c528c7c4-3a25-4163-8375-922fbe1c07e3-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-9xv8h\" (UID: \"c528c7c4-3a25-4163-8375-922fbe1c07e3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9xv8h"
Jul 28 00:54:37.789496 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:37.789413    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9xv8h"
Jul 28 00:54:37.957813 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:37.956661    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9xv8h"]
Jul 28 00:54:37.961143 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:54:37.961108    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc528c7c4_3a25_4163_8375_922fbe1c07e3.slice/crio-d600a27b4c93b2c367ffa08814d2d96af52dc8646ff1f43158d85a2d924b1dc8 WatchSource:0}: Error finding container d600a27b4c93b2c367ffa08814d2d96af52dc8646ff1f43158d85a2d924b1dc8: Status 404 returned error can't find the container with id d600a27b4c93b2c367ffa08814d2d96af52dc8646ff1f43158d85a2d924b1dc8
Jul 28 00:54:38.396225 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:38.396182    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:54:38.396702 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:38.396680    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:54:38.400325 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:38.400298    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 00:54:38.400451 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:38.400342    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cknlx\""
Jul 28 00:54:38.400451 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:38.400437    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 00:54:38.400581 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:38.400469    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wktlp\""
Jul 28 00:54:38.400581 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:38.400570    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 00:54:38.578501 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:38.578465    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9xv8h" event={"ID":"c528c7c4-3a25-4163-8375-922fbe1c07e3","Type":"ContainerStarted","Data":"d600a27b4c93b2c367ffa08814d2d96af52dc8646ff1f43158d85a2d924b1dc8"}
Jul 28 00:54:40.587132 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:40.586703    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9xv8h" event={"ID":"c528c7c4-3a25-4163-8375-922fbe1c07e3","Type":"ContainerStarted","Data":"0e3c016b20f850f45ae149e32f0112bc892acc2983c6adc743ac2d102298a6d0"}
Jul 28 00:54:40.587132 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:40.586981    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9xv8h"
Jul 28 00:54:40.589280 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:40.589252    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dbcvj" event={"ID":"19425b8d-e929-41df-824d-49b9ec05ec72","Type":"ContainerStarted","Data":"0f9dc794114d90f6547ed69f08e4d2afbf02975e67bc5aade33a1b4792d53996"}
Jul 28 00:54:40.589429 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:40.589305    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dbcvj" event={"ID":"19425b8d-e929-41df-824d-49b9ec05ec72","Type":"ContainerStarted","Data":"afa118299450e8420881ccc3777e800b4678710cc720fb6725ff3a9873f0075a"}
Jul 28 00:54:40.589429 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:40.589414    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dbcvj"
Jul 28 00:54:40.592654 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:40.592626    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9xv8h"
Jul 28 00:54:40.605704 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:40.605659    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9xv8h" podStartSLOduration=1.64605355 podStartE2EDuration="3.605641512s" totalImagesPullingTime="1.959587962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:54:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:54:40.604256737 +0000 UTC m=+50.769064845" watchObservedRunningTime="2026-07-28 00:54:40.605641512 +0000 UTC m=+50.770449599"
Jul 28 00:54:40.619796 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:40.619592    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dbcvj" podStartSLOduration=2.220698166 podStartE2EDuration="4.619572038s" totalImagesPullingTime="2.398873872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:54:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:54:40.618350482 +0000 UTC m=+50.783158604" watchObservedRunningTime="2026-07-28 00:54:40.619572038 +0000 UTC m=+50.784380153"
Jul 28 00:54:41.525993 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:41.525960    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-h9sks"]
Jul 28 00:54:41.639551 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:41.639513    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-h9sks"]
Jul 28 00:54:41.640035 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:41.639646    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-h9sks"
Jul 28 00:54:41.642159 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:41.642132    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 28 00:54:41.642159 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:41.642145    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 28 00:54:41.642159 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:41.642145    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 00:54:41.642644 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:41.642623    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 00:54:41.644170 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:41.643080    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-w8g2g\""
Jul 28 00:54:41.644170 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:41.644093    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 00:54:41.733935 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:41.733895    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f55f9612-aaa8-41e9-a785-4ec4cb622e64-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-h9sks\" (UID: \"f55f9612-aaa8-41e9-a785-4ec4cb622e64\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h9sks"
Jul 28 00:54:41.734089 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:41.733940    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f55f9612-aaa8-41e9-a785-4ec4cb622e64-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-h9sks\" (UID: \"f55f9612-aaa8-41e9-a785-4ec4cb622e64\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h9sks"
Jul 28 00:54:41.734163 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:41.734097    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f55f9612-aaa8-41e9-a785-4ec4cb622e64-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-h9sks\" (UID: \"f55f9612-aaa8-41e9-a785-4ec4cb622e64\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h9sks"
Jul 28 00:54:41.734163 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:41.734136    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh7qc\" (UniqueName: \"kubernetes.io/projected/f55f9612-aaa8-41e9-a785-4ec4cb622e64-kube-api-access-bh7qc\") pod \"prometheus-operator-84d69f794d-h9sks\" (UID: \"f55f9612-aaa8-41e9-a785-4ec4cb622e64\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h9sks"
Jul 28 00:54:41.835318 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:41.835225    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f55f9612-aaa8-41e9-a785-4ec4cb622e64-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-h9sks\" (UID: \"f55f9612-aaa8-41e9-a785-4ec4cb622e64\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h9sks"
Jul 28 00:54:41.835318 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:41.835284    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f55f9612-aaa8-41e9-a785-4ec4cb622e64-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-h9sks\" (UID: \"f55f9612-aaa8-41e9-a785-4ec4cb622e64\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h9sks"
Jul 28 00:54:41.835568 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:41.835372    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f55f9612-aaa8-41e9-a785-4ec4cb622e64-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-h9sks\" (UID: \"f55f9612-aaa8-41e9-a785-4ec4cb622e64\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h9sks"
Jul 28 00:54:41.835568 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:41.835410    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh7qc\" (UniqueName: \"kubernetes.io/projected/f55f9612-aaa8-41e9-a785-4ec4cb622e64-kube-api-access-bh7qc\") pod \"prometheus-operator-84d69f794d-h9sks\" (UID: \"f55f9612-aaa8-41e9-a785-4ec4cb622e64\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h9sks"
Jul 28 00:54:41.835678 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:41.835574    2578 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 28 00:54:41.835678 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:41.835659    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f55f9612-aaa8-41e9-a785-4ec4cb622e64-prometheus-operator-tls podName:f55f9612-aaa8-41e9-a785-4ec4cb622e64 nodeName:}" failed. No retries permitted until 2026-07-28 00:54:42.335635774 +0000 UTC m=+52.500443871 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/f55f9612-aaa8-41e9-a785-4ec4cb622e64-prometheus-operator-tls") pod "prometheus-operator-84d69f794d-h9sks" (UID: "f55f9612-aaa8-41e9-a785-4ec4cb622e64") : secret "prometheus-operator-tls" not found
Jul 28 00:54:41.836128 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:41.836100    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f55f9612-aaa8-41e9-a785-4ec4cb622e64-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-h9sks\" (UID: \"f55f9612-aaa8-41e9-a785-4ec4cb622e64\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h9sks"
Jul 28 00:54:41.838816 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:41.838796    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f55f9612-aaa8-41e9-a785-4ec4cb622e64-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-h9sks\" (UID: \"f55f9612-aaa8-41e9-a785-4ec4cb622e64\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h9sks"
Jul 28 00:54:41.843892 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:41.843874    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh7qc\" (UniqueName: \"kubernetes.io/projected/f55f9612-aaa8-41e9-a785-4ec4cb622e64-kube-api-access-bh7qc\") pod \"prometheus-operator-84d69f794d-h9sks\" (UID: \"f55f9612-aaa8-41e9-a785-4ec4cb622e64\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h9sks"
Jul 28 00:54:42.339598 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:42.339561    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f55f9612-aaa8-41e9-a785-4ec4cb622e64-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-h9sks\" (UID: \"f55f9612-aaa8-41e9-a785-4ec4cb622e64\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h9sks"
Jul 28 00:54:42.341844 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:42.341817    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f55f9612-aaa8-41e9-a785-4ec4cb622e64-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-h9sks\" (UID: \"f55f9612-aaa8-41e9-a785-4ec4cb622e64\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h9sks"
Jul 28 00:54:42.552406 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:42.552372    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-h9sks"
Jul 28 00:54:42.904450 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:42.904414    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57b5878bcc-6jdm7"]
Jul 28 00:54:42.917585 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:42.917561    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:54:42.919402 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:42.919355    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57b5878bcc-6jdm7"]
Jul 28 00:54:42.920194 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:42.920164    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 00:54:42.921399 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:42.921187    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-krj69\""
Jul 28 00:54:42.921399 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:42.921229    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 00:54:42.921399 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:42.921300    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 00:54:42.921399 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:42.921398    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 00:54:42.921668 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:42.921492    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 00:54:42.921668 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:42.921651    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 00:54:42.921876 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:42.921852    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 00:54:42.925828 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:42.925798    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 28 00:54:42.944081 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:42.944054    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e5723ff-4e24-4bcb-ae69-589b5150145f-service-ca\") pod \"console-57b5878bcc-6jdm7\" (UID: \"2e5723ff-4e24-4bcb-ae69-589b5150145f\") " pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:54:42.944182 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:42.944085    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e5723ff-4e24-4bcb-ae69-589b5150145f-trusted-ca-bundle\") pod \"console-57b5878bcc-6jdm7\" (UID: \"2e5723ff-4e24-4bcb-ae69-589b5150145f\") " pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:54:42.944182 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:42.944134    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6pjr\" (UniqueName: \"kubernetes.io/projected/2e5723ff-4e24-4bcb-ae69-589b5150145f-kube-api-access-x6pjr\") pod \"console-57b5878bcc-6jdm7\" (UID: \"2e5723ff-4e24-4bcb-ae69-589b5150145f\") " pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:54:42.944296 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:42.944184    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e5723ff-4e24-4bcb-ae69-589b5150145f-console-config\") pod \"console-57b5878bcc-6jdm7\" (UID: \"2e5723ff-4e24-4bcb-ae69-589b5150145f\") " pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:54:42.944296 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:42.944226    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e5723ff-4e24-4bcb-ae69-589b5150145f-oauth-serving-cert\") pod \"console-57b5878bcc-6jdm7\" (UID: \"2e5723ff-4e24-4bcb-ae69-589b5150145f\") " pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:54:42.944296 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:42.944260    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e5723ff-4e24-4bcb-ae69-589b5150145f-console-serving-cert\") pod \"console-57b5878bcc-6jdm7\" (UID: \"2e5723ff-4e24-4bcb-ae69-589b5150145f\") " pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:54:42.944296 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:42.944281    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e5723ff-4e24-4bcb-ae69-589b5150145f-console-oauth-config\") pod \"console-57b5878bcc-6jdm7\" (UID: \"2e5723ff-4e24-4bcb-ae69-589b5150145f\") " pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:54:43.045452 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:43.045412    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6pjr\" (UniqueName: \"kubernetes.io/projected/2e5723ff-4e24-4bcb-ae69-589b5150145f-kube-api-access-x6pjr\") pod \"console-57b5878bcc-6jdm7\" (UID: \"2e5723ff-4e24-4bcb-ae69-589b5150145f\") " pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:54:43.045711 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:43.045465    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e5723ff-4e24-4bcb-ae69-589b5150145f-console-config\") pod \"console-57b5878bcc-6jdm7\" (UID: \"2e5723ff-4e24-4bcb-ae69-589b5150145f\") " pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:54:43.045711 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:43.045496    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e5723ff-4e24-4bcb-ae69-589b5150145f-oauth-serving-cert\") pod \"console-57b5878bcc-6jdm7\" (UID: \"2e5723ff-4e24-4bcb-ae69-589b5150145f\") " pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:54:43.045711 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:43.045517    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e5723ff-4e24-4bcb-ae69-589b5150145f-console-serving-cert\") pod \"console-57b5878bcc-6jdm7\" (UID: \"2e5723ff-4e24-4bcb-ae69-589b5150145f\") " pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:54:43.045865 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:43.045537    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e5723ff-4e24-4bcb-ae69-589b5150145f-console-oauth-config\") pod \"console-57b5878bcc-6jdm7\" (UID: \"2e5723ff-4e24-4bcb-ae69-589b5150145f\") " pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:54:43.045865 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:43.045832    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e5723ff-4e24-4bcb-ae69-589b5150145f-service-ca\") pod \"console-57b5878bcc-6jdm7\" (UID: \"2e5723ff-4e24-4bcb-ae69-589b5150145f\") " pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:54:43.045865 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:43.045858    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e5723ff-4e24-4bcb-ae69-589b5150145f-trusted-ca-bundle\") pod \"console-57b5878bcc-6jdm7\" (UID: \"2e5723ff-4e24-4bcb-ae69-589b5150145f\") " pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:54:43.046314 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:43.046278    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e5723ff-4e24-4bcb-ae69-589b5150145f-oauth-serving-cert\") pod \"console-57b5878bcc-6jdm7\" (UID: \"2e5723ff-4e24-4bcb-ae69-589b5150145f\") " pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:54:43.046415 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:43.046385    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e5723ff-4e24-4bcb-ae69-589b5150145f-console-config\") pod \"console-57b5878bcc-6jdm7\" (UID: \"2e5723ff-4e24-4bcb-ae69-589b5150145f\") " pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:54:43.046528 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:43.046463    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e5723ff-4e24-4bcb-ae69-589b5150145f-service-ca\") pod \"console-57b5878bcc-6jdm7\" (UID: \"2e5723ff-4e24-4bcb-ae69-589b5150145f\") " pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:54:43.046781 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:43.046755    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e5723ff-4e24-4bcb-ae69-589b5150145f-trusted-ca-bundle\") pod \"console-57b5878bcc-6jdm7\" (UID: \"2e5723ff-4e24-4bcb-ae69-589b5150145f\") " pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:54:43.048427 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:43.048406    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e5723ff-4e24-4bcb-ae69-589b5150145f-console-oauth-config\") pod \"console-57b5878bcc-6jdm7\" (UID: \"2e5723ff-4e24-4bcb-ae69-589b5150145f\") " pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:54:43.048648 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:43.048632    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e5723ff-4e24-4bcb-ae69-589b5150145f-console-serving-cert\") pod \"console-57b5878bcc-6jdm7\" (UID: \"2e5723ff-4e24-4bcb-ae69-589b5150145f\") " pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:54:43.053235 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:43.053192    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6pjr\" (UniqueName: \"kubernetes.io/projected/2e5723ff-4e24-4bcb-ae69-589b5150145f-kube-api-access-x6pjr\") pod \"console-57b5878bcc-6jdm7\" (UID: \"2e5723ff-4e24-4bcb-ae69-589b5150145f\") " pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:54:43.229225 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:43.229132    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:54:44.039724 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:44.039654    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57b5878bcc-6jdm7"]
Jul 28 00:54:44.046275 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:54:44.046196    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e5723ff_4e24_4bcb_ae69_589b5150145f.slice/crio-09e9e4d44a3c775e2f5fb89b6dbfa0e104ef7eb13dda9fbb025c0d3c6d6adda6 WatchSource:0}: Error finding container 09e9e4d44a3c775e2f5fb89b6dbfa0e104ef7eb13dda9fbb025c0d3c6d6adda6: Status 404 returned error can't find the container with id 09e9e4d44a3c775e2f5fb89b6dbfa0e104ef7eb13dda9fbb025c0d3c6d6adda6
Jul 28 00:54:44.055775 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:44.054097    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-h9sks"]
Jul 28 00:54:44.062372 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:54:44.062346    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf55f9612_aaa8_41e9_a785_4ec4cb622e64.slice/crio-a15854484669bb181218d0d8cc6666e9a68b21297f5ba773d4b038bd41bfcf90 WatchSource:0}: Error finding container a15854484669bb181218d0d8cc6666e9a68b21297f5ba773d4b038bd41bfcf90: Status 404 returned error can't find the container with id a15854484669bb181218d0d8cc6666e9a68b21297f5ba773d4b038bd41bfcf90
Jul 28 00:54:44.602674 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:44.602589    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57b5878bcc-6jdm7" event={"ID":"2e5723ff-4e24-4bcb-ae69-589b5150145f","Type":"ContainerStarted","Data":"09e9e4d44a3c775e2f5fb89b6dbfa0e104ef7eb13dda9fbb025c0d3c6d6adda6"}
Jul 28 00:54:44.605241 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:44.605181    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-46nnl" event={"ID":"b87dd1d9-c9dd-45c5-a9f5-f8eb70385708","Type":"ContainerStarted","Data":"4260e3b5535165394c5d279c4e45285b7258bd1d9775ff0a3e438ab6b73cf0c0"}
Jul 28 00:54:44.607526 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:44.607501    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ptgdm" event={"ID":"c32f7a50-f215-4648-a7bc-8cf984ef6fb0","Type":"ContainerStarted","Data":"b29af018895466a24eb940eb80b71a73979e0bf6a374232641030bf022641eec"}
Jul 28 00:54:44.610183 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:44.610155    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-h9sks" event={"ID":"f55f9612-aaa8-41e9-a785-4ec4cb622e64","Type":"ContainerStarted","Data":"a15854484669bb181218d0d8cc6666e9a68b21297f5ba773d4b038bd41bfcf90"}
Jul 28 00:54:46.619901 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:46.619708    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-h9sks" event={"ID":"f55f9612-aaa8-41e9-a785-4ec4cb622e64","Type":"ContainerStarted","Data":"e91389e138dfd3aafc91896b0612ceb8e435525fe0c46b5a18686481fe4bb5ac"}
Jul 28 00:54:46.620396 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:46.619923    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-h9sks" event={"ID":"f55f9612-aaa8-41e9-a785-4ec4cb622e64","Type":"ContainerStarted","Data":"d268521ae77e63d76a0d414727174ae266677827921751c827423db9369683aa"}
Jul 28 00:54:46.635715 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:46.635665    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ptgdm" podStartSLOduration=4.01685854 podStartE2EDuration="10.635641564s" totalImagesPullingTime="6.618783024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:54:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:54:44.623712829 +0000 UTC m=+54.788520937" watchObservedRunningTime="2026-07-28 00:54:46.635641564 +0000 UTC m=+56.800449673"
Jul 28 00:54:46.636133 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:46.636090    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-h9sks" podStartSLOduration=3.8809214990000003 podStartE2EDuration="5.636079335s" totalImagesPullingTime="1.755157836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:54:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:54:46.634010629 +0000 UTC m=+56.798818735" watchObservedRunningTime="2026-07-28 00:54:46.636079335 +0000 UTC m=+56.800887439"
Jul 28 00:54:48.872469 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.872437    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-s2ng7"]
Jul 28 00:54:48.876412 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.876391    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s2ng7"
Jul 28 00:54:48.878722 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.878701    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 28 00:54:48.878829 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.878795    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-6zkgf\""
Jul 28 00:54:48.878902 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.878889    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 28 00:54:48.886673 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.886653    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-s2ng7"]
Jul 28 00:54:48.895338 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.895320    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-fnjzx"]
Jul 28 00:54:48.899359 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.899340    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fnjzx"
Jul 28 00:54:48.901810 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.901786    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 28 00:54:48.901900 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.901843    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-tgr4v\""
Jul 28 00:54:48.901900 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.901858    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 28 00:54:48.902286 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.902267    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 28 00:54:48.912415 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.912393    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-fnjzx"]
Jul 28 00:54:48.920767 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.920746    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qz2nn"]
Jul 28 00:54:48.924265 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.924248    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:48.926813 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.926796    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9zh8t\""
Jul 28 00:54:48.927294 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.927055    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 00:54:48.927294 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.927064    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 00:54:48.927294 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.927082    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 00:54:48.998295 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.998269    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hntx\" (UniqueName: \"kubernetes.io/projected/8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1-kube-api-access-5hntx\") pod \"openshift-state-metrics-b8f49f5c6-s2ng7\" (UID: \"8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s2ng7"
Jul 28 00:54:48.998467 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.998303    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd8976a7-8e1e-45b9-851c-7312c17fab2d-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-fnjzx\" (UID: \"bd8976a7-8e1e-45b9-851c-7312c17fab2d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fnjzx"
Jul 28 00:54:48.998467 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.998322    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-s2ng7\" (UID: \"8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s2ng7"
Jul 28 00:54:48.998467 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.998341    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/bd8976a7-8e1e-45b9-851c-7312c17fab2d-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-fnjzx\" (UID: \"bd8976a7-8e1e-45b9-851c-7312c17fab2d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fnjzx"
Jul 28 00:54:48.998467 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.998398    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/bd8976a7-8e1e-45b9-851c-7312c17fab2d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-fnjzx\" (UID: \"bd8976a7-8e1e-45b9-851c-7312c17fab2d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fnjzx"
Jul 28 00:54:48.998467 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.998439    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/24ea961e-dcc1-471c-841d-f851ad0b3969-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qz2nn\" (UID: \"24ea961e-dcc1-471c-841d-f851ad0b3969\") " pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:48.998719 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.998484    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/24ea961e-dcc1-471c-841d-f851ad0b3969-sys\") pod \"node-exporter-qz2nn\" (UID: \"24ea961e-dcc1-471c-841d-f851ad0b3969\") " pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:48.998719 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.998513    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z26vk\" (UniqueName: \"kubernetes.io/projected/24ea961e-dcc1-471c-841d-f851ad0b3969-kube-api-access-z26vk\") pod \"node-exporter-qz2nn\" (UID: \"24ea961e-dcc1-471c-841d-f851ad0b3969\") " pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:48.998719 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.998552    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bd8976a7-8e1e-45b9-851c-7312c17fab2d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-fnjzx\" (UID: \"bd8976a7-8e1e-45b9-851c-7312c17fab2d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fnjzx"
Jul 28 00:54:48.998719 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.998587    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd8976a7-8e1e-45b9-851c-7312c17fab2d-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-fnjzx\" (UID: \"bd8976a7-8e1e-45b9-851c-7312c17fab2d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fnjzx"
Jul 28 00:54:48.998719 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.998615    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/24ea961e-dcc1-471c-841d-f851ad0b3969-node-exporter-textfile\") pod \"node-exporter-qz2nn\" (UID: \"24ea961e-dcc1-471c-841d-f851ad0b3969\") " pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:48.998719 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.998655    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/24ea961e-dcc1-471c-841d-f851ad0b3969-node-exporter-wtmp\") pod \"node-exporter-qz2nn\" (UID: \"24ea961e-dcc1-471c-841d-f851ad0b3969\") " pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:48.998719 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.998691    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-s2ng7\" (UID: \"8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s2ng7"
Jul 28 00:54:48.998991 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.998759    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/24ea961e-dcc1-471c-841d-f851ad0b3969-metrics-client-ca\") pod \"node-exporter-qz2nn\" (UID: \"24ea961e-dcc1-471c-841d-f851ad0b3969\") " pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:48.998991 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.998788    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/24ea961e-dcc1-471c-841d-f851ad0b3969-node-exporter-tls\") pod \"node-exporter-qz2nn\" (UID: \"24ea961e-dcc1-471c-841d-f851ad0b3969\") " pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:48.998991 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.998825    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-s2ng7\" (UID: \"8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s2ng7"
Jul 28 00:54:48.998991 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.998845    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvs5w\" (UniqueName: \"kubernetes.io/projected/bd8976a7-8e1e-45b9-851c-7312c17fab2d-kube-api-access-mvs5w\") pod \"kube-state-metrics-56d9f944b4-fnjzx\" (UID: \"bd8976a7-8e1e-45b9-851c-7312c17fab2d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fnjzx"
Jul 28 00:54:48.998991 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.998863    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/24ea961e-dcc1-471c-841d-f851ad0b3969-root\") pod \"node-exporter-qz2nn\" (UID: \"24ea961e-dcc1-471c-841d-f851ad0b3969\") " pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:48.998991 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:48.998880    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/24ea961e-dcc1-471c-841d-f851ad0b3969-node-exporter-accelerators-collector-config\") pod \"node-exporter-qz2nn\" (UID: \"24ea961e-dcc1-471c-841d-f851ad0b3969\") " pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:49.099701 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.099672    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/24ea961e-dcc1-471c-841d-f851ad0b3969-sys\") pod \"node-exporter-qz2nn\" (UID: \"24ea961e-dcc1-471c-841d-f851ad0b3969\") " pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:49.099829 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.099718    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z26vk\" (UniqueName: \"kubernetes.io/projected/24ea961e-dcc1-471c-841d-f851ad0b3969-kube-api-access-z26vk\") pod \"node-exporter-qz2nn\" (UID: \"24ea961e-dcc1-471c-841d-f851ad0b3969\") " pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:49.099829 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.099756    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bd8976a7-8e1e-45b9-851c-7312c17fab2d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-fnjzx\" (UID: \"bd8976a7-8e1e-45b9-851c-7312c17fab2d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fnjzx"
Jul 28 00:54:49.099829 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.099794    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd8976a7-8e1e-45b9-851c-7312c17fab2d-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-fnjzx\" (UID: \"bd8976a7-8e1e-45b9-851c-7312c17fab2d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fnjzx"
Jul 28 00:54:49.099829 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.099818    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/24ea961e-dcc1-471c-841d-f851ad0b3969-node-exporter-textfile\") pod \"node-exporter-qz2nn\" (UID: \"24ea961e-dcc1-471c-841d-f851ad0b3969\") " pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:49.100026 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.099794    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/24ea961e-dcc1-471c-841d-f851ad0b3969-sys\") pod \"node-exporter-qz2nn\" (UID: \"24ea961e-dcc1-471c-841d-f851ad0b3969\") " pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:49.100026 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.099945    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/24ea961e-dcc1-471c-841d-f851ad0b3969-node-exporter-wtmp\") pod \"node-exporter-qz2nn\" (UID: \"24ea961e-dcc1-471c-841d-f851ad0b3969\") " pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:49.100026 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.099986    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-s2ng7\" (UID: \"8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s2ng7"
Jul 28 00:54:49.100177 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.100030    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/24ea961e-dcc1-471c-841d-f851ad0b3969-metrics-client-ca\") pod \"node-exporter-qz2nn\" (UID: \"24ea961e-dcc1-471c-841d-f851ad0b3969\") " pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:49.100177 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.100065    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/24ea961e-dcc1-471c-841d-f851ad0b3969-node-exporter-tls\") pod \"node-exporter-qz2nn\" (UID: \"24ea961e-dcc1-471c-841d-f851ad0b3969\") " pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:49.100177 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.100099    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-s2ng7\" (UID: \"8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s2ng7"
Jul 28 00:54:49.100177 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.100133    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvs5w\" (UniqueName: \"kubernetes.io/projected/bd8976a7-8e1e-45b9-851c-7312c17fab2d-kube-api-access-mvs5w\") pod \"kube-state-metrics-56d9f944b4-fnjzx\" (UID: \"bd8976a7-8e1e-45b9-851c-7312c17fab2d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fnjzx"
Jul 28 00:54:49.100177 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.100135    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/24ea961e-dcc1-471c-841d-f851ad0b3969-node-exporter-wtmp\") pod \"node-exporter-qz2nn\" (UID: \"24ea961e-dcc1-471c-841d-f851ad0b3969\") " pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:49.100177 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.100153    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/24ea961e-dcc1-471c-841d-f851ad0b3969-root\") pod \"node-exporter-qz2nn\" (UID: \"24ea961e-dcc1-471c-841d-f851ad0b3969\") " pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:49.100177 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.100157    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/24ea961e-dcc1-471c-841d-f851ad0b3969-node-exporter-textfile\") pod \"node-exporter-qz2nn\" (UID: \"24ea961e-dcc1-471c-841d-f851ad0b3969\") " pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:49.100545 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.100196    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/24ea961e-dcc1-471c-841d-f851ad0b3969-node-exporter-accelerators-collector-config\") pod \"node-exporter-qz2nn\" (UID: \"24ea961e-dcc1-471c-841d-f851ad0b3969\") " pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:49.100545 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.100244    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/24ea961e-dcc1-471c-841d-f851ad0b3969-root\") pod \"node-exporter-qz2nn\" (UID: \"24ea961e-dcc1-471c-841d-f851ad0b3969\") " pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:49.100545 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.100262    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hntx\" (UniqueName: \"kubernetes.io/projected/8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1-kube-api-access-5hntx\") pod \"openshift-state-metrics-b8f49f5c6-s2ng7\" (UID: \"8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s2ng7"
Jul 28 00:54:49.100545 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.100308    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd8976a7-8e1e-45b9-851c-7312c17fab2d-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-fnjzx\" (UID: \"bd8976a7-8e1e-45b9-851c-7312c17fab2d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fnjzx"
Jul 28 00:54:49.100545 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.100334    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-s2ng7\" (UID: \"8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s2ng7"
Jul 28 00:54:49.100545 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.100360    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/bd8976a7-8e1e-45b9-851c-7312c17fab2d-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-fnjzx\" (UID: \"bd8976a7-8e1e-45b9-851c-7312c17fab2d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fnjzx"
Jul 28 00:54:49.100545 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.100388    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/bd8976a7-8e1e-45b9-851c-7312c17fab2d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-fnjzx\" (UID: \"bd8976a7-8e1e-45b9-851c-7312c17fab2d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fnjzx"
Jul 28 00:54:49.100545 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.100412    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/24ea961e-dcc1-471c-841d-f851ad0b3969-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qz2nn\" (UID: \"24ea961e-dcc1-471c-841d-f851ad0b3969\") " pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:49.100909 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.100553    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd8976a7-8e1e-45b9-851c-7312c17fab2d-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-fnjzx\" (UID: \"bd8976a7-8e1e-45b9-851c-7312c17fab2d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fnjzx"
Jul 28 00:54:49.100909 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.100739    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-s2ng7\" (UID: \"8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s2ng7"
Jul 28 00:54:49.100909 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.100892    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/24ea961e-dcc1-471c-841d-f851ad0b3969-metrics-client-ca\") pod \"node-exporter-qz2nn\" (UID: \"24ea961e-dcc1-471c-841d-f851ad0b3969\") " pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:49.101096 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.101042    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/bd8976a7-8e1e-45b9-851c-7312c17fab2d-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-fnjzx\" (UID: \"bd8976a7-8e1e-45b9-851c-7312c17fab2d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fnjzx"
Jul 28 00:54:49.101318 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:49.101246    2578 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 28 00:54:49.101318 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.101272    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/24ea961e-dcc1-471c-841d-f851ad0b3969-node-exporter-accelerators-collector-config\") pod \"node-exporter-qz2nn\" (UID: \"24ea961e-dcc1-471c-841d-f851ad0b3969\") " pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:49.101790 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.101347    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/bd8976a7-8e1e-45b9-851c-7312c17fab2d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-fnjzx\" (UID: \"bd8976a7-8e1e-45b9-851c-7312c17fab2d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fnjzx"
Jul 28 00:54:49.101790 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:49.101656    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24ea961e-dcc1-471c-841d-f851ad0b3969-node-exporter-tls podName:24ea961e-dcc1-471c-841d-f851ad0b3969 nodeName:}" failed. No retries permitted until 2026-07-28 00:54:49.601289891 +0000 UTC m=+59.766097995 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/24ea961e-dcc1-471c-841d-f851ad0b3969-node-exporter-tls") pod "node-exporter-qz2nn" (UID: "24ea961e-dcc1-471c-841d-f851ad0b3969") : secret "node-exporter-tls" not found
Jul 28 00:54:49.104701 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.104675    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/24ea961e-dcc1-471c-841d-f851ad0b3969-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qz2nn\" (UID: \"24ea961e-dcc1-471c-841d-f851ad0b3969\") " pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:49.104817 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.104795    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-s2ng7\" (UID: \"8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s2ng7"
Jul 28 00:54:49.104874 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.104795    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-s2ng7\" (UID: \"8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s2ng7"
Jul 28 00:54:49.104874 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.104848    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd8976a7-8e1e-45b9-851c-7312c17fab2d-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-fnjzx\" (UID: \"bd8976a7-8e1e-45b9-851c-7312c17fab2d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fnjzx"
Jul 28 00:54:49.105145 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.105128    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bd8976a7-8e1e-45b9-851c-7312c17fab2d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-fnjzx\" (UID: \"bd8976a7-8e1e-45b9-851c-7312c17fab2d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fnjzx"
Jul 28 00:54:49.110027 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.110005    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z26vk\" (UniqueName: \"kubernetes.io/projected/24ea961e-dcc1-471c-841d-f851ad0b3969-kube-api-access-z26vk\") pod \"node-exporter-qz2nn\" (UID: \"24ea961e-dcc1-471c-841d-f851ad0b3969\") " pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:49.110141 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.110124    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvs5w\" (UniqueName: \"kubernetes.io/projected/bd8976a7-8e1e-45b9-851c-7312c17fab2d-kube-api-access-mvs5w\") pod \"kube-state-metrics-56d9f944b4-fnjzx\" (UID: \"bd8976a7-8e1e-45b9-851c-7312c17fab2d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fnjzx"
Jul 28 00:54:49.110247 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.110232    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hntx\" (UniqueName: \"kubernetes.io/projected/8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1-kube-api-access-5hntx\") pod \"openshift-state-metrics-b8f49f5c6-s2ng7\" (UID: \"8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s2ng7"
Jul 28 00:54:49.185261 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.185235    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s2ng7"
Jul 28 00:54:49.208324 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.208299    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fnjzx"
Jul 28 00:54:49.472199 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.472164    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-s2ng7"]
Jul 28 00:54:49.475604 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:54:49.475579    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fd3d0c3_7107_4334_8e7d_2448d4c8b6e1.slice/crio-e4c73bb97224fb54618967970101ae903c0afed071f84f5577024532377d71a8 WatchSource:0}: Error finding container e4c73bb97224fb54618967970101ae903c0afed071f84f5577024532377d71a8: Status 404 returned error can't find the container with id e4c73bb97224fb54618967970101ae903c0afed071f84f5577024532377d71a8
Jul 28 00:54:49.484616 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.484591    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-fnjzx"]
Jul 28 00:54:49.486843 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:54:49.486821    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd8976a7_8e1e_45b9_851c_7312c17fab2d.slice/crio-1672447a79cfac796dfe85eb893000720796b1bc8f97092ca21ce81a94084034 WatchSource:0}: Error finding container 1672447a79cfac796dfe85eb893000720796b1bc8f97092ca21ce81a94084034: Status 404 returned error can't find the container with id 1672447a79cfac796dfe85eb893000720796b1bc8f97092ca21ce81a94084034
Jul 28 00:54:49.605389 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.605355    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/24ea961e-dcc1-471c-841d-f851ad0b3969-node-exporter-tls\") pod \"node-exporter-qz2nn\" (UID: \"24ea961e-dcc1-471c-841d-f851ad0b3969\") " pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:49.607403 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.607377    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/24ea961e-dcc1-471c-841d-f851ad0b3969-node-exporter-tls\") pod \"node-exporter-qz2nn\" (UID: \"24ea961e-dcc1-471c-841d-f851ad0b3969\") " pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:49.634055 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.634005    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-46nnl" event={"ID":"b87dd1d9-c9dd-45c5-a9f5-f8eb70385708","Type":"ContainerStarted","Data":"cb31c72c89b751387905cc12521612d6584438848ffbfcf84ad2f08f6b55083c"}
Jul 28 00:54:49.635111 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.635088    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fnjzx" event={"ID":"bd8976a7-8e1e-45b9-851c-7312c17fab2d","Type":"ContainerStarted","Data":"1672447a79cfac796dfe85eb893000720796b1bc8f97092ca21ce81a94084034"}
Jul 28 00:54:49.636519 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.636497    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57b5878bcc-6jdm7" event={"ID":"2e5723ff-4e24-4bcb-ae69-589b5150145f","Type":"ContainerStarted","Data":"765c2ffbdae6e1955cd69a681b7bfcecdf9fe37c0a878dfc5146a36ebb02562e"}
Jul 28 00:54:49.638279 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.638251    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s2ng7" event={"ID":"8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1","Type":"ContainerStarted","Data":"f6b8e9d5721e336d0d339767564927dd3b79b97c59c743c1fefee91cdb070ede"}
Jul 28 00:54:49.638348 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.638311    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s2ng7" event={"ID":"8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1","Type":"ContainerStarted","Data":"e71524011b54b3105f6ff1877e9ac5f67309c3d2ff25603d440e268cd682ef07"}
Jul 28 00:54:49.638348 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.638340    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s2ng7" event={"ID":"8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1","Type":"ContainerStarted","Data":"e4c73bb97224fb54618967970101ae903c0afed071f84f5577024532377d71a8"}
Jul 28 00:54:49.655180 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.655091    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57b5878bcc-6jdm7" podStartSLOduration=2.637224675 podStartE2EDuration="7.655071295s" totalImagesPullingTime="5.01784662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:54:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:54:49.654375073 +0000 UTC m=+59.819183182" watchObservedRunningTime="2026-07-28 00:54:49.655071295 +0000 UTC m=+59.819879403"
Jul 28 00:54:49.834538 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.834505    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qz2nn"
Jul 28 00:54:49.860291 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:54:49.860266    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24ea961e_dcc1_471c_841d_f851ad0b3969.slice/crio-e3387fe33d77d357203ff0c7af36ab0b744e6018f27b32ec1a054a99a71b816b WatchSource:0}: Error finding container e3387fe33d77d357203ff0c7af36ab0b744e6018f27b32ec1a054a99a71b816b: Status 404 returned error can't find the container with id e3387fe33d77d357203ff0c7af36ab0b744e6018f27b32ec1a054a99a71b816b
Jul 28 00:54:49.935871 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.935790    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 00:54:49.958587 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.958558    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 00:54:49.958772 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.958688    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:49.961455 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.961431    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 28 00:54:49.961601 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.961434    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 28 00:54:49.961601 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.961435    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 28 00:54:49.961711 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.961667    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-spd2s\""
Jul 28 00:54:49.961782 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.961766    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 28 00:54:49.961848 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.961782    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 28 00:54:49.961848 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.961783    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 28 00:54:49.961947 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.961770    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 28 00:54:49.962110 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.962095    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 28 00:54:49.962182 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:49.962098    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 28 00:54:50.008423 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.008385    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/37112156-d7b1-42c5-9f56-08e5ccb8472b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.008423 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.008426    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/37112156-d7b1-42c5-9f56-08e5ccb8472b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.008610 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.008456    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37112156-d7b1-42c5-9f56-08e5ccb8472b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.008610 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.008509    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/37112156-d7b1-42c5-9f56-08e5ccb8472b-web-config\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.008610 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.008565    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-644zg\" (UniqueName: \"kubernetes.io/projected/37112156-d7b1-42c5-9f56-08e5ccb8472b-kube-api-access-644zg\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.008756 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.008616    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/37112156-d7b1-42c5-9f56-08e5ccb8472b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.008756 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.008641    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37112156-d7b1-42c5-9f56-08e5ccb8472b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.008756 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.008711    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/37112156-d7b1-42c5-9f56-08e5ccb8472b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.008911 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.008760    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/37112156-d7b1-42c5-9f56-08e5ccb8472b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.008911 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.008790    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/37112156-d7b1-42c5-9f56-08e5ccb8472b-config-out\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.008911 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.008813    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/37112156-d7b1-42c5-9f56-08e5ccb8472b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.008911 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.008855    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/37112156-d7b1-42c5-9f56-08e5ccb8472b-config-volume\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.008911 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.008897    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/37112156-d7b1-42c5-9f56-08e5ccb8472b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.109884 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.109838    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37112156-d7b1-42c5-9f56-08e5ccb8472b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.110051 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.109898    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/37112156-d7b1-42c5-9f56-08e5ccb8472b-web-config\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.110051 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.109930    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-644zg\" (UniqueName: \"kubernetes.io/projected/37112156-d7b1-42c5-9f56-08e5ccb8472b-kube-api-access-644zg\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.110051 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.109968    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/37112156-d7b1-42c5-9f56-08e5ccb8472b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.110051 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:50.109998    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37112156-d7b1-42c5-9f56-08e5ccb8472b-alertmanager-trusted-ca-bundle podName:37112156-d7b1-42c5-9f56-08e5ccb8472b nodeName:}" failed. No retries permitted until 2026-07-28 00:54:50.609980006 +0000 UTC m=+60.774788091 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/37112156-d7b1-42c5-9f56-08e5ccb8472b-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "37112156-d7b1-42c5-9f56-08e5ccb8472b") : configmap references non-existent config key: ca-bundle.crt
Jul 28 00:54:50.110051 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.110027    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37112156-d7b1-42c5-9f56-08e5ccb8472b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.110353 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.110058    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/37112156-d7b1-42c5-9f56-08e5ccb8472b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.110353 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.110108    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/37112156-d7b1-42c5-9f56-08e5ccb8472b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.110353 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.110139    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/37112156-d7b1-42c5-9f56-08e5ccb8472b-config-out\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.110353 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.110167    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/37112156-d7b1-42c5-9f56-08e5ccb8472b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.110353 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.110230    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/37112156-d7b1-42c5-9f56-08e5ccb8472b-config-volume\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.110353 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.110279    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/37112156-d7b1-42c5-9f56-08e5ccb8472b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.110618 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.110317    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/37112156-d7b1-42c5-9f56-08e5ccb8472b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.110618 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.110372    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/37112156-d7b1-42c5-9f56-08e5ccb8472b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.110618 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:50.110478    2578 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 28 00:54:50.110618 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:50.110525    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37112156-d7b1-42c5-9f56-08e5ccb8472b-secret-alertmanager-main-tls podName:37112156-d7b1-42c5-9f56-08e5ccb8472b nodeName:}" failed. No retries permitted until 2026-07-28 00:54:50.610510469 +0000 UTC m=+60.775318553 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/37112156-d7b1-42c5-9f56-08e5ccb8472b-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "37112156-d7b1-42c5-9f56-08e5ccb8472b") : secret "alertmanager-main-tls" not found
Jul 28 00:54:50.111450 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.111049    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37112156-d7b1-42c5-9f56-08e5ccb8472b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.111450 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.111106    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/37112156-d7b1-42c5-9f56-08e5ccb8472b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.113004 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.112926    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/37112156-d7b1-42c5-9f56-08e5ccb8472b-config-out\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.113004 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.112991    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/37112156-d7b1-42c5-9f56-08e5ccb8472b-web-config\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.113168 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.113112    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/37112156-d7b1-42c5-9f56-08e5ccb8472b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.113711 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.113659    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/37112156-d7b1-42c5-9f56-08e5ccb8472b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.113809 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.113789    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/37112156-d7b1-42c5-9f56-08e5ccb8472b-config-volume\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.114368 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.114343    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/37112156-d7b1-42c5-9f56-08e5ccb8472b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.114578 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.114558    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/37112156-d7b1-42c5-9f56-08e5ccb8472b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.114983 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.114945    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/37112156-d7b1-42c5-9f56-08e5ccb8472b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.118687 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.118603    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-644zg\" (UniqueName: \"kubernetes.io/projected/37112156-d7b1-42c5-9f56-08e5ccb8472b-kube-api-access-644zg\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.540536 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.540504    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t87lr"
Jul 28 00:54:50.594535 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.594504    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dbcvj"
Jul 28 00:54:50.614788 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.614746    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/37112156-d7b1-42c5-9f56-08e5ccb8472b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.614928 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.614914    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37112156-d7b1-42c5-9f56-08e5ccb8472b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.616059 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.615865    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37112156-d7b1-42c5-9f56-08e5ccb8472b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:50.616059 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:50.615925    2578 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 28 00:54:50.616059 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:50.615991    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37112156-d7b1-42c5-9f56-08e5ccb8472b-secret-alertmanager-main-tls podName:37112156-d7b1-42c5-9f56-08e5ccb8472b nodeName:}" failed. No retries permitted until 2026-07-28 00:54:51.615972974 +0000 UTC m=+61.780781061 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/37112156-d7b1-42c5-9f56-08e5ccb8472b-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "37112156-d7b1-42c5-9f56-08e5ccb8472b") : secret "alertmanager-main-tls" not found
Jul 28 00:54:50.645082 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.645001    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qz2nn" event={"ID":"24ea961e-dcc1-471c-841d-f851ad0b3969","Type":"ContainerStarted","Data":"e3387fe33d77d357203ff0c7af36ab0b744e6018f27b32ec1a054a99a71b816b"}
Jul 28 00:54:50.858273 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.857619    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"]
Jul 28 00:54:50.860934 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.860914    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"
Jul 28 00:54:50.864073 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.863642    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 28 00:54:50.864073 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.863901    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 28 00:54:50.864366 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.864343    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 28 00:54:50.864472 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.864369    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-75k5f\""
Jul 28 00:54:50.864548 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.864529    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 28 00:54:50.864632 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.864617    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-24fn0i4iqquvl\""
Jul 28 00:54:50.864695 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.864659    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 28 00:54:50.872985 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.872897    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"]
Jul 28 00:54:50.919116 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.918671    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8bf4b440-aaa7-4750-ac41-f1be52a77a51-secret-thanos-querier-tls\") pod \"thanos-querier-78977f5d4b-pl7n7\" (UID: \"8bf4b440-aaa7-4750-ac41-f1be52a77a51\") " pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"
Jul 28 00:54:50.919116 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.918733    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8bf4b440-aaa7-4750-ac41-f1be52a77a51-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78977f5d4b-pl7n7\" (UID: \"8bf4b440-aaa7-4750-ac41-f1be52a77a51\") " pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"
Jul 28 00:54:50.919116 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.918814    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8bf4b440-aaa7-4750-ac41-f1be52a77a51-metrics-client-ca\") pod \"thanos-querier-78977f5d4b-pl7n7\" (UID: \"8bf4b440-aaa7-4750-ac41-f1be52a77a51\") " pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"
Jul 28 00:54:50.919116 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.918842    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8bf4b440-aaa7-4750-ac41-f1be52a77a51-secret-grpc-tls\") pod \"thanos-querier-78977f5d4b-pl7n7\" (UID: \"8bf4b440-aaa7-4750-ac41-f1be52a77a51\") " pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"
Jul 28 00:54:50.919116 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.918869    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8bf4b440-aaa7-4750-ac41-f1be52a77a51-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78977f5d4b-pl7n7\" (UID: \"8bf4b440-aaa7-4750-ac41-f1be52a77a51\") " pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"
Jul 28 00:54:50.919116 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.918894    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8bf4b440-aaa7-4750-ac41-f1be52a77a51-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78977f5d4b-pl7n7\" (UID: \"8bf4b440-aaa7-4750-ac41-f1be52a77a51\") " pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"
Jul 28 00:54:50.919116 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.918964    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nstjb\" (UniqueName: \"kubernetes.io/projected/8bf4b440-aaa7-4750-ac41-f1be52a77a51-kube-api-access-nstjb\") pod \"thanos-querier-78977f5d4b-pl7n7\" (UID: \"8bf4b440-aaa7-4750-ac41-f1be52a77a51\") " pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"
Jul 28 00:54:50.919116 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:50.918999    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8bf4b440-aaa7-4750-ac41-f1be52a77a51-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78977f5d4b-pl7n7\" (UID: \"8bf4b440-aaa7-4750-ac41-f1be52a77a51\") " pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"
Jul 28 00:54:51.020467 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:51.020418    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8bf4b440-aaa7-4750-ac41-f1be52a77a51-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78977f5d4b-pl7n7\" (UID: \"8bf4b440-aaa7-4750-ac41-f1be52a77a51\") " pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"
Jul 28 00:54:51.020945 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:51.020518    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8bf4b440-aaa7-4750-ac41-f1be52a77a51-metrics-client-ca\") pod \"thanos-querier-78977f5d4b-pl7n7\" (UID: \"8bf4b440-aaa7-4750-ac41-f1be52a77a51\") " pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"
Jul 28 00:54:51.020945 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:51.020545    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8bf4b440-aaa7-4750-ac41-f1be52a77a51-secret-grpc-tls\") pod \"thanos-querier-78977f5d4b-pl7n7\" (UID: \"8bf4b440-aaa7-4750-ac41-f1be52a77a51\") " pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"
Jul 28 00:54:51.020945 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:51.020574    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8bf4b440-aaa7-4750-ac41-f1be52a77a51-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78977f5d4b-pl7n7\" (UID: \"8bf4b440-aaa7-4750-ac41-f1be52a77a51\") " pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"
Jul 28 00:54:51.020945 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:51.020605    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8bf4b440-aaa7-4750-ac41-f1be52a77a51-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78977f5d4b-pl7n7\" (UID: \"8bf4b440-aaa7-4750-ac41-f1be52a77a51\") " pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"
Jul 28 00:54:51.020945 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:51.020672    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nstjb\" (UniqueName: \"kubernetes.io/projected/8bf4b440-aaa7-4750-ac41-f1be52a77a51-kube-api-access-nstjb\") pod \"thanos-querier-78977f5d4b-pl7n7\" (UID: \"8bf4b440-aaa7-4750-ac41-f1be52a77a51\") " pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"
Jul 28 00:54:51.020945 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:51.020706    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8bf4b440-aaa7-4750-ac41-f1be52a77a51-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78977f5d4b-pl7n7\" (UID: \"8bf4b440-aaa7-4750-ac41-f1be52a77a51\") " pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"
Jul 28 00:54:51.020945 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:51.020765    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8bf4b440-aaa7-4750-ac41-f1be52a77a51-secret-thanos-querier-tls\") pod \"thanos-querier-78977f5d4b-pl7n7\" (UID: \"8bf4b440-aaa7-4750-ac41-f1be52a77a51\") " pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"
Jul 28 00:54:51.021761 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:51.021728    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8bf4b440-aaa7-4750-ac41-f1be52a77a51-metrics-client-ca\") pod \"thanos-querier-78977f5d4b-pl7n7\" (UID: \"8bf4b440-aaa7-4750-ac41-f1be52a77a51\") " pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"
Jul 28 00:54:51.024709 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:51.024686    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8bf4b440-aaa7-4750-ac41-f1be52a77a51-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78977f5d4b-pl7n7\" (UID: \"8bf4b440-aaa7-4750-ac41-f1be52a77a51\") " pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"
Jul 28 00:54:51.024807 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:51.024688    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8bf4b440-aaa7-4750-ac41-f1be52a77a51-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78977f5d4b-pl7n7\" (UID: \"8bf4b440-aaa7-4750-ac41-f1be52a77a51\") " pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"
Jul 28 00:54:51.024952 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:51.024905    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8bf4b440-aaa7-4750-ac41-f1be52a77a51-secret-grpc-tls\") pod \"thanos-querier-78977f5d4b-pl7n7\" (UID: \"8bf4b440-aaa7-4750-ac41-f1be52a77a51\") " pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"
Jul 28 00:54:51.026352 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:51.026307    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8bf4b440-aaa7-4750-ac41-f1be52a77a51-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78977f5d4b-pl7n7\" (UID: \"8bf4b440-aaa7-4750-ac41-f1be52a77a51\") " pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"
Jul 28 00:54:51.026685 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:51.026660    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8bf4b440-aaa7-4750-ac41-f1be52a77a51-secret-thanos-querier-tls\") pod \"thanos-querier-78977f5d4b-pl7n7\" (UID: \"8bf4b440-aaa7-4750-ac41-f1be52a77a51\") " pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"
Jul 28 00:54:51.026776 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:51.026685    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8bf4b440-aaa7-4750-ac41-f1be52a77a51-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78977f5d4b-pl7n7\" (UID: \"8bf4b440-aaa7-4750-ac41-f1be52a77a51\") " pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"
Jul 28 00:54:51.028394 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:51.028359    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nstjb\" (UniqueName: \"kubernetes.io/projected/8bf4b440-aaa7-4750-ac41-f1be52a77a51-kube-api-access-nstjb\") pod \"thanos-querier-78977f5d4b-pl7n7\" (UID: \"8bf4b440-aaa7-4750-ac41-f1be52a77a51\") " pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"
Jul 28 00:54:51.174987 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:51.174895    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"
Jul 28 00:54:51.626421 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:51.626387    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/37112156-d7b1-42c5-9f56-08e5ccb8472b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:51.632799 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:51.632736    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/37112156-d7b1-42c5-9f56-08e5ccb8472b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"37112156-d7b1-42c5-9f56-08e5ccb8472b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:51.651443 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:51.651409    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qz2nn" event={"ID":"24ea961e-dcc1-471c-841d-f851ad0b3969","Type":"ContainerStarted","Data":"4421fcdb3eb6dde634a0e6a127d1863fd0ce4784e62e04cb8a224150ec4d7211"}
Jul 28 00:54:51.653826 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:51.653792    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fnjzx" event={"ID":"bd8976a7-8e1e-45b9-851c-7312c17fab2d","Type":"ContainerStarted","Data":"b03706dd970aaa99eb05e2cd8f2a16570099d0697383af30f0164c3bec30938c"}
Jul 28 00:54:51.655528 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:51.655323    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"]
Jul 28 00:54:51.659040 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:51.659000    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s2ng7" event={"ID":"8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1","Type":"ContainerStarted","Data":"0ca5ae355270acc435becd7e4fc0481efde3e4f3fa9b923d0d2b1cf919be0d96"}
Jul 28 00:54:51.694851 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:51.694795    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s2ng7" podStartSLOduration=1.832402449 podStartE2EDuration="3.694776375s" totalImagesPullingTime="1.862373926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:54:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:54:51.694063627 +0000 UTC m=+61.858871734" watchObservedRunningTime="2026-07-28 00:54:51.694776375 +0000 UTC m=+61.859584482"
Jul 28 00:54:51.792443 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:51.792374    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-spd2s\""
Jul 28 00:54:51.800478 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:51.800254    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 00:54:53.229914 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.229877    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:54:53.230394 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.229957    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:54:53.235823 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.235799    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6f59fc7c69-d2796"]
Jul 28 00:54:53.257153 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.257115    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6f59fc7c69-d2796"]
Jul 28 00:54:53.257323 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.257244    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:54:53.257399 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.257386    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796"
Jul 28 00:54:53.259752 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.259730    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 28 00:54:53.259870 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.259730    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 28 00:54:53.260449 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.260432    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-bsvd5\""
Jul 28 00:54:53.260897 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.260851    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 28 00:54:53.260897 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.260878    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ddo0llh01t7f6\""
Jul 28 00:54:53.261149 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.261133    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 28 00:54:53.326359 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:54:53.326330    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bf4b440_aaa7_4750_ac41_f1be52a77a51.slice/crio-bb5fce4da1466d3078e07d6253d6211665e60c2bb16516b02cc5a7889d079120 WatchSource:0}: Error finding container bb5fce4da1466d3078e07d6253d6211665e60c2bb16516b02cc5a7889d079120: Status 404 returned error can't find the container with id bb5fce4da1466d3078e07d6253d6211665e60c2bb16516b02cc5a7889d079120
Jul 28 00:54:53.343053 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.343027    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1fe546e-f927-4a9d-ac62-273236c89387-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6f59fc7c69-d2796\" (UID: \"b1fe546e-f927-4a9d-ac62-273236c89387\") " pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796"
Jul 28 00:54:53.343158 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.343123    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b1fe546e-f927-4a9d-ac62-273236c89387-client-ca-bundle\") pod \"metrics-server-6f59fc7c69-d2796\" (UID: \"b1fe546e-f927-4a9d-ac62-273236c89387\") " pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796"
Jul 28 00:54:53.343276 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.343249    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b1fe546e-f927-4a9d-ac62-273236c89387-metrics-server-audit-profiles\") pod \"metrics-server-6f59fc7c69-d2796\" (UID: \"b1fe546e-f927-4a9d-ac62-273236c89387\") " pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796"
Jul 28 00:54:53.343724 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.343321    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b1fe546e-f927-4a9d-ac62-273236c89387-audit-log\") pod \"metrics-server-6f59fc7c69-d2796\" (UID: \"b1fe546e-f927-4a9d-ac62-273236c89387\") " pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796"
Jul 28 00:54:53.343724 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.343482    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mh7k\" (UniqueName: \"kubernetes.io/projected/b1fe546e-f927-4a9d-ac62-273236c89387-kube-api-access-2mh7k\") pod \"metrics-server-6f59fc7c69-d2796\" (UID: \"b1fe546e-f927-4a9d-ac62-273236c89387\") " pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796"
Jul 28 00:54:53.343724 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.343526    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b1fe546e-f927-4a9d-ac62-273236c89387-secret-metrics-server-client-certs\") pod \"metrics-server-6f59fc7c69-d2796\" (UID: \"b1fe546e-f927-4a9d-ac62-273236c89387\") " pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796"
Jul 28 00:54:53.343724 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.343590    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b1fe546e-f927-4a9d-ac62-273236c89387-secret-metrics-server-tls\") pod \"metrics-server-6f59fc7c69-d2796\" (UID: \"b1fe546e-f927-4a9d-ac62-273236c89387\") " pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796"
Jul 28 00:54:53.444248 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.444183    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b1fe546e-f927-4a9d-ac62-273236c89387-client-ca-bundle\") pod \"metrics-server-6f59fc7c69-d2796\" (UID: \"b1fe546e-f927-4a9d-ac62-273236c89387\") " pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796"
Jul 28 00:54:53.444369 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.444293    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b1fe546e-f927-4a9d-ac62-273236c89387-metrics-server-audit-profiles\") pod \"metrics-server-6f59fc7c69-d2796\" (UID: \"b1fe546e-f927-4a9d-ac62-273236c89387\") " pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796"
Jul 28 00:54:53.444369 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.444327    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b1fe546e-f927-4a9d-ac62-273236c89387-audit-log\") pod \"metrics-server-6f59fc7c69-d2796\" (UID: \"b1fe546e-f927-4a9d-ac62-273236c89387\") " pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796"
Jul 28 00:54:53.444476 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.444386    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mh7k\" (UniqueName: \"kubernetes.io/projected/b1fe546e-f927-4a9d-ac62-273236c89387-kube-api-access-2mh7k\") pod \"metrics-server-6f59fc7c69-d2796\" (UID: \"b1fe546e-f927-4a9d-ac62-273236c89387\") " pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796"
Jul 28 00:54:53.444476 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.444416    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b1fe546e-f927-4a9d-ac62-273236c89387-secret-metrics-server-client-certs\") pod \"metrics-server-6f59fc7c69-d2796\" (UID: \"b1fe546e-f927-4a9d-ac62-273236c89387\") " pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796"
Jul 28 00:54:53.444476 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.444460    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b1fe546e-f927-4a9d-ac62-273236c89387-secret-metrics-server-tls\") pod \"metrics-server-6f59fc7c69-d2796\" (UID: \"b1fe546e-f927-4a9d-ac62-273236c89387\") " pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796"
Jul 28 00:54:53.444632 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.444486    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1fe546e-f927-4a9d-ac62-273236c89387-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6f59fc7c69-d2796\" (UID: \"b1fe546e-f927-4a9d-ac62-273236c89387\") " pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796"
Jul 28 00:54:53.445628 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.445184    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1fe546e-f927-4a9d-ac62-273236c89387-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6f59fc7c69-d2796\" (UID: \"b1fe546e-f927-4a9d-ac62-273236c89387\") " pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796"
Jul 28 00:54:53.445628 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.445498    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b1fe546e-f927-4a9d-ac62-273236c89387-audit-log\") pod \"metrics-server-6f59fc7c69-d2796\" (UID: \"b1fe546e-f927-4a9d-ac62-273236c89387\") " pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796"
Jul 28 00:54:53.445801 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.445726    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b1fe546e-f927-4a9d-ac62-273236c89387-metrics-server-audit-profiles\") pod \"metrics-server-6f59fc7c69-d2796\" (UID: \"b1fe546e-f927-4a9d-ac62-273236c89387\") " pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796"
Jul 28 00:54:53.446947 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.446924    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b1fe546e-f927-4a9d-ac62-273236c89387-client-ca-bundle\") pod \"metrics-server-6f59fc7c69-d2796\" (UID: \"b1fe546e-f927-4a9d-ac62-273236c89387\") " pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796"
Jul 28 00:54:53.447373 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.447351    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b1fe546e-f927-4a9d-ac62-273236c89387-secret-metrics-server-client-certs\") pod \"metrics-server-6f59fc7c69-d2796\" (UID: \"b1fe546e-f927-4a9d-ac62-273236c89387\") " pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796"
Jul 28 00:54:53.447641 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.447621    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b1fe546e-f927-4a9d-ac62-273236c89387-secret-metrics-server-tls\") pod \"metrics-server-6f59fc7c69-d2796\" (UID: \"b1fe546e-f927-4a9d-ac62-273236c89387\") " pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796"
Jul 28 00:54:53.452185 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.452165    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mh7k\" (UniqueName: \"kubernetes.io/projected/b1fe546e-f927-4a9d-ac62-273236c89387-kube-api-access-2mh7k\") pod \"metrics-server-6f59fc7c69-d2796\" (UID: \"b1fe546e-f927-4a9d-ac62-273236c89387\") " pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796"
Jul 28 00:54:53.478844 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.478817    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 00:54:53.482441 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:54:53.482410    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37112156_d7b1_42c5_9f56_08e5ccb8472b.slice/crio-f451d31b5abab5b78f1a0d61f3fff599e2e6f25c4ff71cae58d2aa445778f56c WatchSource:0}: Error finding container f451d31b5abab5b78f1a0d61f3fff599e2e6f25c4ff71cae58d2aa445778f56c: Status 404 returned error can't find the container with id f451d31b5abab5b78f1a0d61f3fff599e2e6f25c4ff71cae58d2aa445778f56c
Jul 28 00:54:53.567673 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.567642    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796"
Jul 28 00:54:53.669266 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.669239    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-slx94"]
Jul 28 00:54:53.673262 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.673236    2578 generic.go:356] "Generic (PLEG): container finished" podID="24ea961e-dcc1-471c-841d-f851ad0b3969" containerID="4421fcdb3eb6dde634a0e6a127d1863fd0ce4784e62e04cb8a224150ec4d7211" exitCode=0
Jul 28 00:54:53.673583 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.673549    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-46nnl" event={"ID":"b87dd1d9-c9dd-45c5-a9f5-f8eb70385708","Type":"ContainerStarted","Data":"c7bfab385131a0ef14c936a15f578824380d05bc3291703fbb4e03e34faa230b"}
Jul 28 00:54:53.673662 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.673590    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qz2nn" event={"ID":"24ea961e-dcc1-471c-841d-f851ad0b3969","Type":"ContainerDied","Data":"4421fcdb3eb6dde634a0e6a127d1863fd0ce4784e62e04cb8a224150ec4d7211"}
Jul 28 00:54:53.673662 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.673654    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-slx94"
Jul 28 00:54:53.676005 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.675960    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 28 00:54:53.676133 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.676114    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-5d2z2\""
Jul 28 00:54:53.676568 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.676534    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fnjzx" event={"ID":"bd8976a7-8e1e-45b9-851c-7312c17fab2d","Type":"ContainerStarted","Data":"5945f414ce75d2f5d18e2bcb095bcf02e90b0527cb3d24ce796e43e507a0d928"}
Jul 28 00:54:53.676677 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.676583    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fnjzx" event={"ID":"bd8976a7-8e1e-45b9-851c-7312c17fab2d","Type":"ContainerStarted","Data":"b1c64655b17f4ca49895d5e1140fd9d1efc563b6c054a03757da3eba7785e93a"}
Jul 28 00:54:53.678248 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.678108    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"37112156-d7b1-42c5-9f56-08e5ccb8472b","Type":"ContainerStarted","Data":"f451d31b5abab5b78f1a0d61f3fff599e2e6f25c4ff71cae58d2aa445778f56c"}
Jul 28 00:54:53.679475 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.679453    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7" event={"ID":"8bf4b440-aaa7-4750-ac41-f1be52a77a51","Type":"ContainerStarted","Data":"bb5fce4da1466d3078e07d6253d6211665e60c2bb16516b02cc5a7889d079120"}
Jul 28 00:54:53.681961 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.681938    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-slx94"]
Jul 28 00:54:53.684751 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.684733    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:54:53.693705 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.693663    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-46nnl" podStartSLOduration=1.79144306 podStartE2EDuration="17.693649029s" totalImagesPullingTime="15.902205969s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 00:54:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:54:53.693304182 +0000 UTC m=+63.858112320" watchObservedRunningTime="2026-07-28 00:54:53.693649029 +0000 UTC m=+63.858457135"
Jul 28 00:54:53.701428 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.701403    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6f59fc7c69-d2796"]
Jul 28 00:54:53.704740 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:54:53.704717    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1fe546e_f927_4a9d_ac62_273236c89387.slice/crio-b94f2a6b9fc7cc6127dd3bd6cd062a7e41281cb2962c3ae2821b24a4fbd18a76 WatchSource:0}: Error finding container b94f2a6b9fc7cc6127dd3bd6cd062a7e41281cb2962c3ae2821b24a4fbd18a76: Status 404 returned error can't find the container with id b94f2a6b9fc7cc6127dd3bd6cd062a7e41281cb2962c3ae2821b24a4fbd18a76
Jul 28 00:54:53.716118 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.716097    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7684df49c5-5q4bb"]
Jul 28 00:54:53.720561 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.720543    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:54:53.728615 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.728593    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7684df49c5-5q4bb"]
Jul 28 00:54:53.747442 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.747398    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/365ce136-2a8b-4d7f-a5b5-8dbb810ed6a2-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-slx94\" (UID: \"365ce136-2a8b-4d7f-a5b5-8dbb810ed6a2\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-slx94"
Jul 28 00:54:53.748023 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.747997    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nnjm\" (UniqueName: \"kubernetes.io/projected/365ce136-2a8b-4d7f-a5b5-8dbb810ed6a2-kube-api-access-2nnjm\") pod \"monitoring-plugin-68b7fc9574-slx94\" (UID: \"365ce136-2a8b-4d7f-a5b5-8dbb810ed6a2\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-slx94"
Jul 28 00:54:53.772533 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.772481    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-fnjzx" podStartSLOduration=3.758055542 podStartE2EDuration="5.772460522s" totalImagesPullingTime="2.01440498s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:54:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:54:53.770777041 +0000 UTC m=+63.935585174" watchObservedRunningTime="2026-07-28 00:54:53.772460522 +0000 UTC m=+63.937268637"
Jul 28 00:54:53.849351 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.849316    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nnjm\" (UniqueName: \"kubernetes.io/projected/365ce136-2a8b-4d7f-a5b5-8dbb810ed6a2-kube-api-access-2nnjm\") pod \"monitoring-plugin-68b7fc9574-slx94\" (UID: \"365ce136-2a8b-4d7f-a5b5-8dbb810ed6a2\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-slx94"
Jul 28 00:54:53.849510 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.849366    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5d905293-5622-4ad4-ba6a-aece516c7ea6-console-config\") pod \"console-7684df49c5-5q4bb\" (UID: \"5d905293-5622-4ad4-ba6a-aece516c7ea6\") " pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:54:53.849510 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.849400    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d905293-5622-4ad4-ba6a-aece516c7ea6-trusted-ca-bundle\") pod \"console-7684df49c5-5q4bb\" (UID: \"5d905293-5622-4ad4-ba6a-aece516c7ea6\") " pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:54:53.849510 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.849462    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lfvz\" (UniqueName: \"kubernetes.io/projected/5d905293-5622-4ad4-ba6a-aece516c7ea6-kube-api-access-4lfvz\") pod \"console-7684df49c5-5q4bb\" (UID: \"5d905293-5622-4ad4-ba6a-aece516c7ea6\") " pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:54:53.849655 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.849521    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/365ce136-2a8b-4d7f-a5b5-8dbb810ed6a2-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-slx94\" (UID: \"365ce136-2a8b-4d7f-a5b5-8dbb810ed6a2\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-slx94"
Jul 28 00:54:53.849655 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.849560    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5d905293-5622-4ad4-ba6a-aece516c7ea6-service-ca\") pod \"console-7684df49c5-5q4bb\" (UID: \"5d905293-5622-4ad4-ba6a-aece516c7ea6\") " pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:54:53.849655 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.849608    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5d905293-5622-4ad4-ba6a-aece516c7ea6-oauth-serving-cert\") pod \"console-7684df49c5-5q4bb\" (UID: \"5d905293-5622-4ad4-ba6a-aece516c7ea6\") " pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:54:53.849790 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.849664    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d905293-5622-4ad4-ba6a-aece516c7ea6-console-serving-cert\") pod \"console-7684df49c5-5q4bb\" (UID: \"5d905293-5622-4ad4-ba6a-aece516c7ea6\") " pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:54:53.849790 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:53.849689    2578 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 28 00:54:53.849790 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.849705    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5d905293-5622-4ad4-ba6a-aece516c7ea6-console-oauth-config\") pod \"console-7684df49c5-5q4bb\" (UID: \"5d905293-5622-4ad4-ba6a-aece516c7ea6\") " pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:54:53.849790 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:54:53.849770    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/365ce136-2a8b-4d7f-a5b5-8dbb810ed6a2-monitoring-plugin-cert podName:365ce136-2a8b-4d7f-a5b5-8dbb810ed6a2 nodeName:}" failed. No retries permitted until 2026-07-28 00:54:54.349748645 +0000 UTC m=+64.514556734 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/365ce136-2a8b-4d7f-a5b5-8dbb810ed6a2-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-slx94" (UID: "365ce136-2a8b-4d7f-a5b5-8dbb810ed6a2") : secret "monitoring-plugin-cert" not found
Jul 28 00:54:53.857242 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.857205    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nnjm\" (UniqueName: \"kubernetes.io/projected/365ce136-2a8b-4d7f-a5b5-8dbb810ed6a2-kube-api-access-2nnjm\") pod \"monitoring-plugin-68b7fc9574-slx94\" (UID: \"365ce136-2a8b-4d7f-a5b5-8dbb810ed6a2\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-slx94"
Jul 28 00:54:53.950544 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.950508    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5d905293-5622-4ad4-ba6a-aece516c7ea6-console-config\") pod \"console-7684df49c5-5q4bb\" (UID: \"5d905293-5622-4ad4-ba6a-aece516c7ea6\") " pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:54:53.950544 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.950541    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d905293-5622-4ad4-ba6a-aece516c7ea6-trusted-ca-bundle\") pod \"console-7684df49c5-5q4bb\" (UID: \"5d905293-5622-4ad4-ba6a-aece516c7ea6\") " pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:54:53.950744 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.950584    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lfvz\" (UniqueName: \"kubernetes.io/projected/5d905293-5622-4ad4-ba6a-aece516c7ea6-kube-api-access-4lfvz\") pod \"console-7684df49c5-5q4bb\" (UID: \"5d905293-5622-4ad4-ba6a-aece516c7ea6\") " pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:54:53.950744 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.950641    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5d905293-5622-4ad4-ba6a-aece516c7ea6-service-ca\") pod \"console-7684df49c5-5q4bb\" (UID: \"5d905293-5622-4ad4-ba6a-aece516c7ea6\") " pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:54:53.950744 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.950693    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5d905293-5622-4ad4-ba6a-aece516c7ea6-oauth-serving-cert\") pod \"console-7684df49c5-5q4bb\" (UID: \"5d905293-5622-4ad4-ba6a-aece516c7ea6\") " pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:54:53.950864 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.950745    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d905293-5622-4ad4-ba6a-aece516c7ea6-console-serving-cert\") pod \"console-7684df49c5-5q4bb\" (UID: \"5d905293-5622-4ad4-ba6a-aece516c7ea6\") " pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:54:53.950864 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.950771    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5d905293-5622-4ad4-ba6a-aece516c7ea6-console-oauth-config\") pod \"console-7684df49c5-5q4bb\" (UID: \"5d905293-5622-4ad4-ba6a-aece516c7ea6\") " pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:54:53.951400 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.951367    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5d905293-5622-4ad4-ba6a-aece516c7ea6-console-config\") pod \"console-7684df49c5-5q4bb\" (UID: \"5d905293-5622-4ad4-ba6a-aece516c7ea6\") " pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:54:53.951514 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.951412    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5d905293-5622-4ad4-ba6a-aece516c7ea6-oauth-serving-cert\") pod \"console-7684df49c5-5q4bb\" (UID: \"5d905293-5622-4ad4-ba6a-aece516c7ea6\") " pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:54:53.951514 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.951460    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5d905293-5622-4ad4-ba6a-aece516c7ea6-service-ca\") pod \"console-7684df49c5-5q4bb\" (UID: \"5d905293-5622-4ad4-ba6a-aece516c7ea6\") " pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:54:53.951636 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.951613    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d905293-5622-4ad4-ba6a-aece516c7ea6-trusted-ca-bundle\") pod \"console-7684df49c5-5q4bb\" (UID: \"5d905293-5622-4ad4-ba6a-aece516c7ea6\") " pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:54:53.953168 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.953146    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d905293-5622-4ad4-ba6a-aece516c7ea6-console-serving-cert\") pod \"console-7684df49c5-5q4bb\" (UID: \"5d905293-5622-4ad4-ba6a-aece516c7ea6\") " pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:54:53.953276 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.953204    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5d905293-5622-4ad4-ba6a-aece516c7ea6-console-oauth-config\") pod \"console-7684df49c5-5q4bb\" (UID: \"5d905293-5622-4ad4-ba6a-aece516c7ea6\") " pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:54:53.957696 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:53.957678    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lfvz\" (UniqueName: \"kubernetes.io/projected/5d905293-5622-4ad4-ba6a-aece516c7ea6-kube-api-access-4lfvz\") pod \"console-7684df49c5-5q4bb\" (UID: \"5d905293-5622-4ad4-ba6a-aece516c7ea6\") " pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:54:54.033612 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.033541    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:54:54.082378 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.082005    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"]
Jul 28 00:54:54.086361 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.086335    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"
Jul 28 00:54:54.089150 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.088921    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 28 00:54:54.089150 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.088931    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 28 00:54:54.089150 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.088970    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 28 00:54:54.089150 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.088993    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 28 00:54:54.089150 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.089021    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-2tq2g\""
Jul 28 00:54:54.089150 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.088921    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 28 00:54:54.106113 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.105949    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 28 00:54:54.109796 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.109770    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"]
Jul 28 00:54:54.152828 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.152794    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/627ac9a1-6f65-49d9-9cad-58858ea00f26-telemeter-client-tls\") pod \"telemeter-client-7497f8b8d6-25h9t\" (UID: \"627ac9a1-6f65-49d9-9cad-58858ea00f26\") " pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"
Jul 28 00:54:54.152960 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.152841    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/627ac9a1-6f65-49d9-9cad-58858ea00f26-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7497f8b8d6-25h9t\" (UID: \"627ac9a1-6f65-49d9-9cad-58858ea00f26\") " pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"
Jul 28 00:54:54.152960 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.152894    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm4kw\" (UniqueName: \"kubernetes.io/projected/627ac9a1-6f65-49d9-9cad-58858ea00f26-kube-api-access-vm4kw\") pod \"telemeter-client-7497f8b8d6-25h9t\" (UID: \"627ac9a1-6f65-49d9-9cad-58858ea00f26\") " pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"
Jul 28 00:54:54.153085 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.153003    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/627ac9a1-6f65-49d9-9cad-58858ea00f26-secret-telemeter-client\") pod \"telemeter-client-7497f8b8d6-25h9t\" (UID: \"627ac9a1-6f65-49d9-9cad-58858ea00f26\") " pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"
Jul 28 00:54:54.153085 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.153039    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/627ac9a1-6f65-49d9-9cad-58858ea00f26-federate-client-tls\") pod \"telemeter-client-7497f8b8d6-25h9t\" (UID: \"627ac9a1-6f65-49d9-9cad-58858ea00f26\") " pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"
Jul 28 00:54:54.153165 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.153105    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/627ac9a1-6f65-49d9-9cad-58858ea00f26-serving-certs-ca-bundle\") pod \"telemeter-client-7497f8b8d6-25h9t\" (UID: \"627ac9a1-6f65-49d9-9cad-58858ea00f26\") " pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"
Jul 28 00:54:54.153237 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.153192    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/627ac9a1-6f65-49d9-9cad-58858ea00f26-metrics-client-ca\") pod \"telemeter-client-7497f8b8d6-25h9t\" (UID: \"627ac9a1-6f65-49d9-9cad-58858ea00f26\") " pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"
Jul 28 00:54:54.153328 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.153284    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/627ac9a1-6f65-49d9-9cad-58858ea00f26-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7497f8b8d6-25h9t\" (UID: \"627ac9a1-6f65-49d9-9cad-58858ea00f26\") " pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"
Jul 28 00:54:54.189507 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.189473    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7684df49c5-5q4bb"]
Jul 28 00:54:54.192753 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:54:54.192725    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d905293_5622_4ad4_ba6a_aece516c7ea6.slice/crio-266749483dba489d55e11ded41664b0374b379e034a64931323c4862cc92169e WatchSource:0}: Error finding container 266749483dba489d55e11ded41664b0374b379e034a64931323c4862cc92169e: Status 404 returned error can't find the container with id 266749483dba489d55e11ded41664b0374b379e034a64931323c4862cc92169e
Jul 28 00:54:54.256318 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.254123    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/627ac9a1-6f65-49d9-9cad-58858ea00f26-secret-telemeter-client\") pod \"telemeter-client-7497f8b8d6-25h9t\" (UID: \"627ac9a1-6f65-49d9-9cad-58858ea00f26\") " pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"
Jul 28 00:54:54.256318 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.254170    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/627ac9a1-6f65-49d9-9cad-58858ea00f26-federate-client-tls\") pod \"telemeter-client-7497f8b8d6-25h9t\" (UID: \"627ac9a1-6f65-49d9-9cad-58858ea00f26\") " pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"
Jul 28 00:54:54.256318 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.254204    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/627ac9a1-6f65-49d9-9cad-58858ea00f26-serving-certs-ca-bundle\") pod \"telemeter-client-7497f8b8d6-25h9t\" (UID: \"627ac9a1-6f65-49d9-9cad-58858ea00f26\") " pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"
Jul 28 00:54:54.256318 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.254300    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/627ac9a1-6f65-49d9-9cad-58858ea00f26-metrics-client-ca\") pod \"telemeter-client-7497f8b8d6-25h9t\" (UID: \"627ac9a1-6f65-49d9-9cad-58858ea00f26\") " pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"
Jul 28 00:54:54.256318 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.254363    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/627ac9a1-6f65-49d9-9cad-58858ea00f26-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7497f8b8d6-25h9t\" (UID: \"627ac9a1-6f65-49d9-9cad-58858ea00f26\") " pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"
Jul 28 00:54:54.256318 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.254404    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/627ac9a1-6f65-49d9-9cad-58858ea00f26-telemeter-client-tls\") pod \"telemeter-client-7497f8b8d6-25h9t\" (UID: \"627ac9a1-6f65-49d9-9cad-58858ea00f26\") " pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"
Jul 28 00:54:54.256318 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.254431    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/627ac9a1-6f65-49d9-9cad-58858ea00f26-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7497f8b8d6-25h9t\" (UID: \"627ac9a1-6f65-49d9-9cad-58858ea00f26\") " pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"
Jul 28 00:54:54.256318 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.254460    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vm4kw\" (UniqueName: \"kubernetes.io/projected/627ac9a1-6f65-49d9-9cad-58858ea00f26-kube-api-access-vm4kw\") pod \"telemeter-client-7497f8b8d6-25h9t\" (UID: \"627ac9a1-6f65-49d9-9cad-58858ea00f26\") " pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"
Jul 28 00:54:54.256318 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.256033    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/627ac9a1-6f65-49d9-9cad-58858ea00f26-metrics-client-ca\") pod \"telemeter-client-7497f8b8d6-25h9t\" (UID: \"627ac9a1-6f65-49d9-9cad-58858ea00f26\") " pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"
Jul 28 00:54:54.257073 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.256945    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/627ac9a1-6f65-49d9-9cad-58858ea00f26-serving-certs-ca-bundle\") pod \"telemeter-client-7497f8b8d6-25h9t\" (UID: \"627ac9a1-6f65-49d9-9cad-58858ea00f26\") " pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"
Jul 28 00:54:54.258059 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.257980    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/627ac9a1-6f65-49d9-9cad-58858ea00f26-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7497f8b8d6-25h9t\" (UID: \"627ac9a1-6f65-49d9-9cad-58858ea00f26\") " pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"
Jul 28 00:54:54.259634 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.259543    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/627ac9a1-6f65-49d9-9cad-58858ea00f26-telemeter-client-tls\") pod \"telemeter-client-7497f8b8d6-25h9t\" (UID: \"627ac9a1-6f65-49d9-9cad-58858ea00f26\") " pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"
Jul 28 00:54:54.259634 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.259589    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/627ac9a1-6f65-49d9-9cad-58858ea00f26-federate-client-tls\") pod \"telemeter-client-7497f8b8d6-25h9t\" (UID: \"627ac9a1-6f65-49d9-9cad-58858ea00f26\") " pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"
Jul 28 00:54:54.261047 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.261004    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/627ac9a1-6f65-49d9-9cad-58858ea00f26-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7497f8b8d6-25h9t\" (UID: \"627ac9a1-6f65-49d9-9cad-58858ea00f26\") " pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"
Jul 28 00:54:54.262551 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.262514    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/627ac9a1-6f65-49d9-9cad-58858ea00f26-secret-telemeter-client\") pod \"telemeter-client-7497f8b8d6-25h9t\" (UID: \"627ac9a1-6f65-49d9-9cad-58858ea00f26\") " pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"
Jul 28 00:54:54.263313 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.263295    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm4kw\" (UniqueName: \"kubernetes.io/projected/627ac9a1-6f65-49d9-9cad-58858ea00f26-kube-api-access-vm4kw\") pod \"telemeter-client-7497f8b8d6-25h9t\" (UID: \"627ac9a1-6f65-49d9-9cad-58858ea00f26\") " pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"
Jul 28 00:54:54.356555 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.355899    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/365ce136-2a8b-4d7f-a5b5-8dbb810ed6a2-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-slx94\" (UID: \"365ce136-2a8b-4d7f-a5b5-8dbb810ed6a2\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-slx94"
Jul 28 00:54:54.358863 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.358837    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/365ce136-2a8b-4d7f-a5b5-8dbb810ed6a2-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-slx94\" (UID: \"365ce136-2a8b-4d7f-a5b5-8dbb810ed6a2\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-slx94"
Jul 28 00:54:54.403257 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.403126    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"
Jul 28 00:54:54.577771 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.577724    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7497f8b8d6-25h9t"]
Jul 28 00:54:54.586149 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.585980    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-slx94"
Jul 28 00:54:54.691288 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.691018    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qz2nn" event={"ID":"24ea961e-dcc1-471c-841d-f851ad0b3969","Type":"ContainerStarted","Data":"e60b32cee4fbedae36098452d27793eb4a040623a204503aecd9737e935717b3"}
Jul 28 00:54:54.691288 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.691080    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qz2nn" event={"ID":"24ea961e-dcc1-471c-841d-f851ad0b3969","Type":"ContainerStarted","Data":"b0441c2cc0e7b449546fd3bd813d1e55c712af59e60226c315e907c4a9924eb7"}
Jul 28 00:54:54.697537 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.697447    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7684df49c5-5q4bb" event={"ID":"5d905293-5622-4ad4-ba6a-aece516c7ea6","Type":"ContainerStarted","Data":"ac9827b995b3b6cb1fb94ac244da5eb8849d414d226fae0b4bd8933223c61112"}
Jul 28 00:54:54.697537 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.697501    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7684df49c5-5q4bb" event={"ID":"5d905293-5622-4ad4-ba6a-aece516c7ea6","Type":"ContainerStarted","Data":"266749483dba489d55e11ded41664b0374b379e034a64931323c4862cc92169e"}
Jul 28 00:54:54.701041 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.699842    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796" event={"ID":"b1fe546e-f927-4a9d-ac62-273236c89387","Type":"ContainerStarted","Data":"b94f2a6b9fc7cc6127dd3bd6cd062a7e41281cb2962c3ae2821b24a4fbd18a76"}
Jul 28 00:54:54.729370 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.729102    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qz2nn" podStartSLOduration=4.995700788 podStartE2EDuration="6.729022127s" totalImagesPullingTime="1.63917739s" totalInitContainerRuntime="94.143949ms" isStatefulPod=true podCreationTimestamp="2026-07-28 00:54:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:54:54.728474788 +0000 UTC m=+64.893282895" watchObservedRunningTime="2026-07-28 00:54:54.729022127 +0000 UTC m=+64.893830234"
Jul 28 00:54:54.746459 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:54.746403    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7684df49c5-5q4bb" podStartSLOduration=1.746385703 podStartE2EDuration="1.746385703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:54:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:54:54.745990236 +0000 UTC m=+64.910798343" watchObservedRunningTime="2026-07-28 00:54:54.746385703 +0000 UTC m=+64.911193810"
Jul 28 00:54:55.028499 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.028367    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-slx94"]
Jul 28 00:54:55.120578 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.120528    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 00:54:55.128150 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.128122    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.131071 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.130962    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 28 00:54:55.131071 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.130982    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 28 00:54:55.131540 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.131520    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 28 00:54:55.131650 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.131542    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 28 00:54:55.131650 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.131579    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 28 00:54:55.131748 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.131522    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 28 00:54:55.131748 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.131742    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-m9szw\""
Jul 28 00:54:55.131862 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.131777    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 28 00:54:55.131960 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.131942    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 28 00:54:55.132531 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.131986    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 28 00:54:55.132531 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.132138    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 28 00:54:55.132531 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.132201    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8gjuvrvjnal77\""
Jul 28 00:54:55.132531 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.132469    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 28 00:54:55.132723 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.132657    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 28 00:54:55.133299 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.133203    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 28 00:54:55.142101 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.142078    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 00:54:55.142474 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.142151    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 28 00:54:55.269001 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.268256    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6pm4\" (UniqueName: \"kubernetes.io/projected/e345eadb-4917-47d1-81d0-3ab515d785d8-kube-api-access-g6pm4\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.269001 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.268332    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e345eadb-4917-47d1-81d0-3ab515d785d8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.269001 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.268361    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e345eadb-4917-47d1-81d0-3ab515d785d8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.269001 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.268423    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e345eadb-4917-47d1-81d0-3ab515d785d8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.269001 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.268456    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e345eadb-4917-47d1-81d0-3ab515d785d8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.269001 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.268491    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e345eadb-4917-47d1-81d0-3ab515d785d8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.269001 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.268512    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e345eadb-4917-47d1-81d0-3ab515d785d8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.269001 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.268539    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e345eadb-4917-47d1-81d0-3ab515d785d8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.269001 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.268566    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e345eadb-4917-47d1-81d0-3ab515d785d8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.269001 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.268613    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e345eadb-4917-47d1-81d0-3ab515d785d8-config-out\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.269001 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.268644    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e345eadb-4917-47d1-81d0-3ab515d785d8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.269001 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.268672    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e345eadb-4917-47d1-81d0-3ab515d785d8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.269001 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.268696    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e345eadb-4917-47d1-81d0-3ab515d785d8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.269001 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.268721    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e345eadb-4917-47d1-81d0-3ab515d785d8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.269001 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.268747    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e345eadb-4917-47d1-81d0-3ab515d785d8-config\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.269001 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.268832    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e345eadb-4917-47d1-81d0-3ab515d785d8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.270107 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.268888    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e345eadb-4917-47d1-81d0-3ab515d785d8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.270107 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.268916    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e345eadb-4917-47d1-81d0-3ab515d785d8-web-config\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.270107 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.268995    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e345eadb-4917-47d1-81d0-3ab515d785d8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.270107 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.269070    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e345eadb-4917-47d1-81d0-3ab515d785d8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.369851 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.369815    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e345eadb-4917-47d1-81d0-3ab515d785d8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.370022 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.369864    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e345eadb-4917-47d1-81d0-3ab515d785d8-config\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.370022 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.369891    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e345eadb-4917-47d1-81d0-3ab515d785d8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.370022 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.369917    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e345eadb-4917-47d1-81d0-3ab515d785d8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.370022 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.369939    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e345eadb-4917-47d1-81d0-3ab515d785d8-web-config\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.370022 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.369971    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e345eadb-4917-47d1-81d0-3ab515d785d8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.370022 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.370000    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e345eadb-4917-47d1-81d0-3ab515d785d8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.370357 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.370029    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6pm4\" (UniqueName: \"kubernetes.io/projected/e345eadb-4917-47d1-81d0-3ab515d785d8-kube-api-access-g6pm4\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.370357 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.370077    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e345eadb-4917-47d1-81d0-3ab515d785d8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.370357 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.370101    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e345eadb-4917-47d1-81d0-3ab515d785d8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.370357 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.370188    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e345eadb-4917-47d1-81d0-3ab515d785d8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.370357 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.370250    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e345eadb-4917-47d1-81d0-3ab515d785d8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.370357 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.370289    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e345eadb-4917-47d1-81d0-3ab515d785d8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.370357 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.370313    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e345eadb-4917-47d1-81d0-3ab515d785d8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.370357 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.370319    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e345eadb-4917-47d1-81d0-3ab515d785d8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.370357 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.370343    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e345eadb-4917-47d1-81d0-3ab515d785d8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.370757 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.370373    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e345eadb-4917-47d1-81d0-3ab515d785d8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.370757 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.370424    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e345eadb-4917-47d1-81d0-3ab515d785d8-config-out\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.370757 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.370459    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e345eadb-4917-47d1-81d0-3ab515d785d8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.370757 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.370488    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e345eadb-4917-47d1-81d0-3ab515d785d8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.370757 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.370519    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e345eadb-4917-47d1-81d0-3ab515d785d8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.370999 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.370838    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e345eadb-4917-47d1-81d0-3ab515d785d8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.372510 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.371060    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e345eadb-4917-47d1-81d0-3ab515d785d8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.372510 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.371822    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e345eadb-4917-47d1-81d0-3ab515d785d8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.372510 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.371921    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e345eadb-4917-47d1-81d0-3ab515d785d8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.372510 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.372160    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e345eadb-4917-47d1-81d0-3ab515d785d8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.373397 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.373369    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e345eadb-4917-47d1-81d0-3ab515d785d8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.375328 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.375298    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e345eadb-4917-47d1-81d0-3ab515d785d8-config\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.375814 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.375741    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e345eadb-4917-47d1-81d0-3ab515d785d8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.376874 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.376807    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e345eadb-4917-47d1-81d0-3ab515d785d8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.376874 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.376853    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e345eadb-4917-47d1-81d0-3ab515d785d8-web-config\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.378080 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.378050    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e345eadb-4917-47d1-81d0-3ab515d785d8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.378332 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.378303    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e345eadb-4917-47d1-81d0-3ab515d785d8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.378449 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.378432    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e345eadb-4917-47d1-81d0-3ab515d785d8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.378663 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.378619    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e345eadb-4917-47d1-81d0-3ab515d785d8-config-out\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.379539 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.379498    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e345eadb-4917-47d1-81d0-3ab515d785d8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.379886 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.379867    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6pm4\" (UniqueName: \"kubernetes.io/projected/e345eadb-4917-47d1-81d0-3ab515d785d8-kube-api-access-g6pm4\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.379971 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.379918    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e345eadb-4917-47d1-81d0-3ab515d785d8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.380570 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.380548    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e345eadb-4917-47d1-81d0-3ab515d785d8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.381602 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.381574    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e345eadb-4917-47d1-81d0-3ab515d785d8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e345eadb-4917-47d1-81d0-3ab515d785d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.442753 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:54:55.442721    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod365ce136_2a8b_4d7f_a5b5_8dbb810ed6a2.slice/crio-c761d8c0614cf1baa7a4382dd25f4e99c2e11f14fdffce45735984cb10daf13b WatchSource:0}: Error finding container c761d8c0614cf1baa7a4382dd25f4e99c2e11f14fdffce45735984cb10daf13b: Status 404 returned error can't find the container with id c761d8c0614cf1baa7a4382dd25f4e99c2e11f14fdffce45735984cb10daf13b
Jul 28 00:54:55.445525 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.445312    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:54:55.704964 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.704916    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-slx94" event={"ID":"365ce136-2a8b-4d7f-a5b5-8dbb810ed6a2","Type":"ContainerStarted","Data":"c761d8c0614cf1baa7a4382dd25f4e99c2e11f14fdffce45735984cb10daf13b"}
Jul 28 00:54:55.706321 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:55.706240    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t" event={"ID":"627ac9a1-6f65-49d9-9cad-58858ea00f26","Type":"ContainerStarted","Data":"e274262e8190707039b952ca925abfcd8aa4341ffb4bc99079738dbf061a0613"}
Jul 28 00:54:56.077159 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.077135    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27305211-7ed9-4558-bd84-f703dcff265a-original-pull-secret\") pod \"global-pull-secret-syncer-lvd59\" (UID: \"27305211-7ed9-4558-bd84-f703dcff265a\") " pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:54:56.077293 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.077182    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knsp9\" (UniqueName: \"kubernetes.io/projected/e5c337e6-650b-4ee1-a24e-3533292e41d1-kube-api-access-knsp9\") pod \"network-check-target-ff2f2\" (UID: \"e5c337e6-650b-4ee1-a24e-3533292e41d1\") " pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:54:56.079999 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.079908    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 00:54:56.080360 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.080338    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 00:54:56.090495 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.090473    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 00:54:56.091901 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.091854    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27305211-7ed9-4558-bd84-f703dcff265a-original-pull-secret\") pod \"global-pull-secret-syncer-lvd59\" (UID: \"27305211-7ed9-4558-bd84-f703dcff265a\") " pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:54:56.102382 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.102352    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knsp9\" (UniqueName: \"kubernetes.io/projected/e5c337e6-650b-4ee1-a24e-3533292e41d1-kube-api-access-knsp9\") pod \"network-check-target-ff2f2\" (UID: \"e5c337e6-650b-4ee1-a24e-3533292e41d1\") " pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:54:56.117403 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.117182    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cknlx\""
Jul 28 00:54:56.125228 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.125186    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:54:56.177710 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.177675    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d-metrics-certs\") pod \"network-metrics-daemon-gpv8q\" (UID: \"532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d\") " pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:54:56.182338 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.181492    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 00:54:56.202651 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.202627    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d-metrics-certs\") pod \"network-metrics-daemon-gpv8q\" (UID: \"532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d\") " pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:54:56.235901 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.235872    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 00:54:56.305277 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.305012    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lvd59"
Jul 28 00:54:56.319229 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.319186    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ff2f2"]
Jul 28 00:54:56.324492 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:54:56.324462    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5c337e6_650b_4ee1_a24e_3533292e41d1.slice/crio-b1ee35a8abc310e2d5b6eeb419a9f3313f38d93b07a5d909101357ca548bf25e WatchSource:0}: Error finding container b1ee35a8abc310e2d5b6eeb419a9f3313f38d93b07a5d909101357ca548bf25e: Status 404 returned error can't find the container with id b1ee35a8abc310e2d5b6eeb419a9f3313f38d93b07a5d909101357ca548bf25e
Jul 28 00:54:56.410251 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.410203    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wktlp\""
Jul 28 00:54:56.418761 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.418557    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpv8q"
Jul 28 00:54:56.510415 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.510074    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lvd59"]
Jul 28 00:54:56.611012 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.610935    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gpv8q"]
Jul 28 00:54:56.615290 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:54:56.615260    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod532f058b_6b5d_4c84_82c0_d2b2ec6e9c7d.slice/crio-5c3c06adb9fe8cf148015b6a39bf4daceb4d5a864a8ea38375c7b0939d7cb1ba WatchSource:0}: Error finding container 5c3c06adb9fe8cf148015b6a39bf4daceb4d5a864a8ea38375c7b0939d7cb1ba: Status 404 returned error can't find the container with id 5c3c06adb9fe8cf148015b6a39bf4daceb4d5a864a8ea38375c7b0939d7cb1ba
Jul 28 00:54:56.712511 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.712474    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gpv8q" event={"ID":"532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d","Type":"ContainerStarted","Data":"5c3c06adb9fe8cf148015b6a39bf4daceb4d5a864a8ea38375c7b0939d7cb1ba"}
Jul 28 00:54:56.714204 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.714180    2578 generic.go:356] "Generic (PLEG): container finished" podID="37112156-d7b1-42c5-9f56-08e5ccb8472b" containerID="e17e577658ab9e1ed7bdc1766d7e9077fcae78523cbab04de0adb2a0f7d19f73" exitCode=0
Jul 28 00:54:56.714350 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.714248    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"37112156-d7b1-42c5-9f56-08e5ccb8472b","Type":"ContainerDied","Data":"e17e577658ab9e1ed7bdc1766d7e9077fcae78523cbab04de0adb2a0f7d19f73"}
Jul 28 00:54:56.715576 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.715548    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ff2f2" event={"ID":"e5c337e6-650b-4ee1-a24e-3533292e41d1","Type":"ContainerStarted","Data":"b1ee35a8abc310e2d5b6eeb419a9f3313f38d93b07a5d909101357ca548bf25e"}
Jul 28 00:54:56.718426 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.717858    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796" event={"ID":"b1fe546e-f927-4a9d-ac62-273236c89387","Type":"ContainerStarted","Data":"5318e5bed454968dd9b6a06ffab300d11d9b4bdf6b0f78f61f62a5bc05c6c838"}
Jul 28 00:54:56.725135 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.725106    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7" event={"ID":"8bf4b440-aaa7-4750-ac41-f1be52a77a51","Type":"ContainerStarted","Data":"a3d146d750e5808edaa072ccf3d36bcaeb8092042df1f5ecf0c4b54b6d1e4038"}
Jul 28 00:54:56.725272 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.725164    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7" event={"ID":"8bf4b440-aaa7-4750-ac41-f1be52a77a51","Type":"ContainerStarted","Data":"61549c9515904ce7474a6f2cf27a02bdca30cb60fdd3510263c42de5b698f0db"}
Jul 28 00:54:56.725272 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.725204    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7" event={"ID":"8bf4b440-aaa7-4750-ac41-f1be52a77a51","Type":"ContainerStarted","Data":"45a90edd6513e81a923ecdb6e2315d2158c52ce7281e36eb81da4938bad8b60b"}
Jul 28 00:54:56.726408 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.726382    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lvd59" event={"ID":"27305211-7ed9-4558-bd84-f703dcff265a","Type":"ContainerStarted","Data":"38d901ad17dbb16694bede858e9cbf933f122fc726696eea67d78b4effd598b3"}
Jul 28 00:54:56.728069 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.728044    2578 generic.go:356] "Generic (PLEG): container finished" podID="e345eadb-4917-47d1-81d0-3ab515d785d8" containerID="48e9600d18b1999e025a378808fda2685548be1a72306fd02ab3602750686f68" exitCode=0
Jul 28 00:54:56.728169 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.728079    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e345eadb-4917-47d1-81d0-3ab515d785d8","Type":"ContainerDied","Data":"48e9600d18b1999e025a378808fda2685548be1a72306fd02ab3602750686f68"}
Jul 28 00:54:56.728169 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:54:56.728133    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e345eadb-4917-47d1-81d0-3ab515d785d8","Type":"ContainerStarted","Data":"cd2ea4bce657d5e3ae426f84dbad7ed535d273bae8c02bdd35d8afc6cf94bf2b"}
Jul 28 00:55:04.034367 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.034290    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:55:04.035568 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.034978    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:55:04.041017 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.040994    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:55:04.062111 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.061908    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796" podStartSLOduration=8.699843487999999 podStartE2EDuration="11.061887077s" totalImagesPullingTime="2.362043589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:54:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:54:56.77805063 +0000 UTC m=+66.942858737" watchObservedRunningTime="2026-07-28 00:55:04.061887077 +0000 UTC m=+74.226695185"
Jul 28 00:55:04.771875 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.771779    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t" event={"ID":"627ac9a1-6f65-49d9-9cad-58858ea00f26","Type":"ContainerStarted","Data":"0f35cb56ae421c194f2d5be872d4e92d53f93945b52d1946b30bd40755198319"}
Jul 28 00:55:04.772020 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.771885    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t" event={"ID":"627ac9a1-6f65-49d9-9cad-58858ea00f26","Type":"ContainerStarted","Data":"520bf6f39fc8e1423860b67f87c8c8caccee00b6fb6f099c17e7055f250d2771"}
Jul 28 00:55:04.772020 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.771909    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t" event={"ID":"627ac9a1-6f65-49d9-9cad-58858ea00f26","Type":"ContainerStarted","Data":"847c0dbe506ca1ec769d2ccad3c142df1e7edb182c1fff25e2e4436222a03b68"}
Jul 28 00:55:04.774826 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.774798    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7" event={"ID":"8bf4b440-aaa7-4750-ac41-f1be52a77a51","Type":"ContainerStarted","Data":"a3220ecd3e8cd11a69a8d5782f3bd510d1e00a01d012be0c46b9ee0da02d2525"}
Jul 28 00:55:04.774969 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.774856    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7" event={"ID":"8bf4b440-aaa7-4750-ac41-f1be52a77a51","Type":"ContainerStarted","Data":"c7a26fd95f80936dd2729f373d5bd1e58e1f90db033ac63864e9b48f05127ea1"}
Jul 28 00:55:04.774969 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.774887    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7" event={"ID":"8bf4b440-aaa7-4750-ac41-f1be52a77a51","Type":"ContainerStarted","Data":"ce1463d07d4fe5e47bc33cc1f3d387a6651f83d911e7192b8c4f0ec7aca8347c"}
Jul 28 00:55:04.774969 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.774934    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"
Jul 28 00:55:04.776424 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.776401    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-slx94" event={"ID":"365ce136-2a8b-4d7f-a5b5-8dbb810ed6a2","Type":"ContainerStarted","Data":"bb6d507c155bbc516da4b80d142347721530baec5d7e51cb9f49c4ca10625e38"}
Jul 28 00:55:04.776633 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.776612    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-slx94"
Jul 28 00:55:04.778223 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.778095    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lvd59" event={"ID":"27305211-7ed9-4558-bd84-f703dcff265a","Type":"ContainerStarted","Data":"3428cb83f953e3f9abc492a38b46eed2e2758a56d18132b497c800cd3473dd1e"}
Jul 28 00:55:04.781741 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.781723    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7"
Jul 28 00:55:04.782009 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.781980    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-slx94"
Jul 28 00:55:04.782254 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.782235    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e345eadb-4917-47d1-81d0-3ab515d785d8","Type":"ContainerStarted","Data":"706ca44792736c676a2619c48d516dd3b680a0484b80eac3fd04c76144418a51"}
Jul 28 00:55:04.782334 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.782285    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e345eadb-4917-47d1-81d0-3ab515d785d8","Type":"ContainerStarted","Data":"cfcf09f185880f01490a1b350e5af9240ed69afb46431cc582bcf322b80e7c11"}
Jul 28 00:55:04.782334 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.782319    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e345eadb-4917-47d1-81d0-3ab515d785d8","Type":"ContainerStarted","Data":"e9ebe6216eae1e615b236b234e8aa36d6b09f5df0d2e615e27631f69481ecf3f"}
Jul 28 00:55:04.782422 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.782345    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e345eadb-4917-47d1-81d0-3ab515d785d8","Type":"ContainerStarted","Data":"231c845ea959f83ebac1026eb4071f7cd8552e21cbeca428bd3100e8dd4dfccd"}
Jul 28 00:55:04.782422 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.782380    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e345eadb-4917-47d1-81d0-3ab515d785d8","Type":"ContainerStarted","Data":"cf6faf0360b3afecea6961cf128bc026ab9d9199e5e902a3b1b3266c2170c9d1"}
Jul 28 00:55:04.782422 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.782413    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e345eadb-4917-47d1-81d0-3ab515d785d8","Type":"ContainerStarted","Data":"916d6cf63a631701c23799eb9d8219823c0ba7d95e8b29ea9ba0a99d25fb64e0"}
Jul 28 00:55:04.784081 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.784059    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gpv8q" event={"ID":"532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d","Type":"ContainerStarted","Data":"32ce2209e1b449f4ec0a42dbcabc89153bfc986964b8566364a7b4a5e65e99a1"}
Jul 28 00:55:04.784172 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.784106    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gpv8q" event={"ID":"532f058b-6b5d-4c84-82c0-d2b2ec6e9c7d","Type":"ContainerStarted","Data":"5554841d083df25c98fd8c8af3bb8c1f8a21b1c35f2445d7f90e10b999f382a8"}
Jul 28 00:55:04.787014 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.786997    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"37112156-d7b1-42c5-9f56-08e5ccb8472b","Type":"ContainerStarted","Data":"47109ec79ff1f6dc79fbb50c72c0723492709572ebfe04f145f4248b8a899341"}
Jul 28 00:55:04.787114 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.787039    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"37112156-d7b1-42c5-9f56-08e5ccb8472b","Type":"ContainerStarted","Data":"c1318072c4ce1c1d7a958e955c6d63c60f56404519a81e9f85fa65034c819d33"}
Jul 28 00:55:04.787114 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.787072    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"37112156-d7b1-42c5-9f56-08e5ccb8472b","Type":"ContainerStarted","Data":"05c8e031e588474677cbfa0216d3a33a8433daf37921e2276fd753bc33c3a782"}
Jul 28 00:55:04.787114 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.787099    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"37112156-d7b1-42c5-9f56-08e5ccb8472b","Type":"ContainerStarted","Data":"78aadca62d26277f6d5e8724f1bef58421a150d7a8d0058ef7d9a023b8bd6c22"}
Jul 28 00:55:04.787246 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.787141    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"37112156-d7b1-42c5-9f56-08e5ccb8472b","Type":"ContainerStarted","Data":"e8d329d5026e6cedd5d138de70bb517e9f1f3011cfa848a8deb56e7d55bd933c"}
Jul 28 00:55:04.787246 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.787170    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"37112156-d7b1-42c5-9f56-08e5ccb8472b","Type":"ContainerStarted","Data":"c8147dc7b42f8a208e7b313244d1faaa32082e027b834ae0154e330a01630a10"}
Jul 28 00:55:04.788506 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.788486    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ff2f2" event={"ID":"e5c337e6-650b-4ee1-a24e-3533292e41d1","Type":"ContainerStarted","Data":"b668e2589ee00ee7f48127e16d42c95d854407fea668e3cd83601fb9a30f5d8e"}
Jul 28 00:55:04.788603 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.788522    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:55:04.792231 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.792198    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:55:04.800781 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.800741    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7497f8b8d6-25h9t" podStartSLOduration=1.6529645039999998 podStartE2EDuration="10.80072737s" totalImagesPullingTime="9.147762866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:54:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:55:04.795073905 +0000 UTC m=+74.959882051" watchObservedRunningTime="2026-07-28 00:55:04.80072737 +0000 UTC m=+74.965535518"
Jul 28 00:55:04.815416 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.815368    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-gpv8q" podStartSLOduration=67.417802213 podStartE2EDuration="1m14.8153515s" totalImagesPullingTime="7.397549287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:53:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:55:04.815289538 +0000 UTC m=+74.980097646" watchObservedRunningTime="2026-07-28 00:55:04.8153515 +0000 UTC m=+74.980159606"
Jul 28 00:55:04.840588 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.840541    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=5.923243017 podStartE2EDuration="15.840529535s" totalImagesPullingTime="9.88501938s" totalInitContainerRuntime="32.267138ms" isStatefulPod=false podCreationTimestamp="2026-07-28 00:54:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:55:04.839205783 +0000 UTC m=+75.004013892" watchObservedRunningTime="2026-07-28 00:55:04.840529535 +0000 UTC m=+75.005337642"
Jul 28 00:55:04.857550 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.857503    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-slx94" podStartSLOduration=8.126834948 podStartE2EDuration="11.857486046s" totalImagesPullingTime="3.730651098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:54:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:55:04.857061139 +0000 UTC m=+75.021869246" watchObservedRunningTime="2026-07-28 00:55:04.857486046 +0000 UTC m=+75.022294156"
Jul 28 00:55:04.907753 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.907720    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57b5878bcc-6jdm7"]
Jul 28 00:55:04.910285 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.910239    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.565596507 podStartE2EDuration="9.910204174s" totalImagesPullingTime="7.30161141s" totalInitContainerRuntime="42.996257ms" isStatefulPod=false podCreationTimestamp="2026-07-28 00:54:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:55:04.908878904 +0000 UTC m=+75.073687012" watchObservedRunningTime="2026-07-28 00:55:04.910204174 +0000 UTC m=+75.075012277"
Jul 28 00:55:04.930248 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.930172    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-ff2f2" podStartSLOduration=67.237328118 podStartE2EDuration="1m14.930159612s" totalImagesPullingTime="7.692831494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:53:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:55:04.928785603 +0000 UTC m=+75.093593710" watchObservedRunningTime="2026-07-28 00:55:04.930159612 +0000 UTC m=+75.094967719"
Jul 28 00:55:04.951964 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.951918    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-78977f5d4b-pl7n7" podStartSLOduration=6.5548561880000005 podStartE2EDuration="14.95190103s" totalImagesPullingTime="8.397044842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:54:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:55:04.950059314 +0000 UTC m=+75.114867445" watchObservedRunningTime="2026-07-28 00:55:04.95190103 +0000 UTC m=+75.116709136"
Jul 28 00:55:04.972879 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:04.972830    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-lvd59" podStartSLOduration=67.453883342 podStartE2EDuration="1m14.972814417s" totalImagesPullingTime="7.518931075s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 00:53:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:55:04.971856458 +0000 UTC m=+75.136664566" watchObservedRunningTime="2026-07-28 00:55:04.972814417 +0000 UTC m=+75.137622545"
Jul 28 00:55:05.446092 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:05.446056    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:55:13.568411 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:13.568281    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796"
Jul 28 00:55:13.568411 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:13.568373    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796"
Jul 28 00:55:16.303200 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:16.303170    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7684df49c5-5q4bb"]
Jul 28 00:55:29.927626 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:29.927563    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-57b5878bcc-6jdm7" podUID="2e5723ff-4e24-4bcb-ae69-589b5150145f" containerName="console" containerID="cri-o://765c2ffbdae6e1955cd69a681b7bfcecdf9fe37c0a878dfc5146a36ebb02562e" gracePeriod=15
Jul 28 00:55:30.176284 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.176260    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:55:30.298323 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.298229    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e5723ff-4e24-4bcb-ae69-589b5150145f-oauth-serving-cert\") pod \"2e5723ff-4e24-4bcb-ae69-589b5150145f\" (UID: \"2e5723ff-4e24-4bcb-ae69-589b5150145f\") "
Jul 28 00:55:30.298323 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.298312    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e5723ff-4e24-4bcb-ae69-589b5150145f-console-config\") pod \"2e5723ff-4e24-4bcb-ae69-589b5150145f\" (UID: \"2e5723ff-4e24-4bcb-ae69-589b5150145f\") "
Jul 28 00:55:30.298542 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.298341    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e5723ff-4e24-4bcb-ae69-589b5150145f-service-ca\") pod \"2e5723ff-4e24-4bcb-ae69-589b5150145f\" (UID: \"2e5723ff-4e24-4bcb-ae69-589b5150145f\") "
Jul 28 00:55:30.298542 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.298365    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e5723ff-4e24-4bcb-ae69-589b5150145f-console-oauth-config\") pod \"2e5723ff-4e24-4bcb-ae69-589b5150145f\" (UID: \"2e5723ff-4e24-4bcb-ae69-589b5150145f\") "
Jul 28 00:55:30.298542 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.298398    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e5723ff-4e24-4bcb-ae69-589b5150145f-console-serving-cert\") pod \"2e5723ff-4e24-4bcb-ae69-589b5150145f\" (UID: \"2e5723ff-4e24-4bcb-ae69-589b5150145f\") "
Jul 28 00:55:30.298542 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.298436    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e5723ff-4e24-4bcb-ae69-589b5150145f-trusted-ca-bundle\") pod \"2e5723ff-4e24-4bcb-ae69-589b5150145f\" (UID: \"2e5723ff-4e24-4bcb-ae69-589b5150145f\") "
Jul 28 00:55:30.298542 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.298465    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6pjr\" (UniqueName: \"kubernetes.io/projected/2e5723ff-4e24-4bcb-ae69-589b5150145f-kube-api-access-x6pjr\") pod \"2e5723ff-4e24-4bcb-ae69-589b5150145f\" (UID: \"2e5723ff-4e24-4bcb-ae69-589b5150145f\") "
Jul 28 00:55:30.298791 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.298675    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e5723ff-4e24-4bcb-ae69-589b5150145f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2e5723ff-4e24-4bcb-ae69-589b5150145f" (UID: "2e5723ff-4e24-4bcb-ae69-589b5150145f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 00:55:30.298791 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.298752    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e5723ff-4e24-4bcb-ae69-589b5150145f-console-config" (OuterVolumeSpecName: "console-config") pod "2e5723ff-4e24-4bcb-ae69-589b5150145f" (UID: "2e5723ff-4e24-4bcb-ae69-589b5150145f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 00:55:30.298898 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.298827    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e5723ff-4e24-4bcb-ae69-589b5150145f-service-ca" (OuterVolumeSpecName: "service-ca") pod "2e5723ff-4e24-4bcb-ae69-589b5150145f" (UID: "2e5723ff-4e24-4bcb-ae69-589b5150145f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 00:55:30.298950 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.298905    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e5723ff-4e24-4bcb-ae69-589b5150145f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2e5723ff-4e24-4bcb-ae69-589b5150145f" (UID: "2e5723ff-4e24-4bcb-ae69-589b5150145f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 00:55:30.299082 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.299060    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e5723ff-4e24-4bcb-ae69-589b5150145f-console-config\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 00:55:30.299082 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.299084    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e5723ff-4e24-4bcb-ae69-589b5150145f-service-ca\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 00:55:30.299303 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.299093    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e5723ff-4e24-4bcb-ae69-589b5150145f-trusted-ca-bundle\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 00:55:30.299303 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.299103    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e5723ff-4e24-4bcb-ae69-589b5150145f-oauth-serving-cert\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 00:55:30.300815 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.300784    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e5723ff-4e24-4bcb-ae69-589b5150145f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2e5723ff-4e24-4bcb-ae69-589b5150145f" (UID: "2e5723ff-4e24-4bcb-ae69-589b5150145f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 00:55:30.300912 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.300784    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e5723ff-4e24-4bcb-ae69-589b5150145f-kube-api-access-x6pjr" (OuterVolumeSpecName: "kube-api-access-x6pjr") pod "2e5723ff-4e24-4bcb-ae69-589b5150145f" (UID: "2e5723ff-4e24-4bcb-ae69-589b5150145f"). InnerVolumeSpecName "kube-api-access-x6pjr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 00:55:30.300912 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.300838    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e5723ff-4e24-4bcb-ae69-589b5150145f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2e5723ff-4e24-4bcb-ae69-589b5150145f" (UID: "2e5723ff-4e24-4bcb-ae69-589b5150145f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 00:55:30.399594 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.399562    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e5723ff-4e24-4bcb-ae69-589b5150145f-console-oauth-config\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 00:55:30.399594 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.399590    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e5723ff-4e24-4bcb-ae69-589b5150145f-console-serving-cert\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 00:55:30.399821 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.399603    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6pjr\" (UniqueName: \"kubernetes.io/projected/2e5723ff-4e24-4bcb-ae69-589b5150145f-kube-api-access-x6pjr\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 00:55:30.901500 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.901464    2578 generic.go:356] "Generic (PLEG): container finished" podID="2e5723ff-4e24-4bcb-ae69-589b5150145f" containerID="765c2ffbdae6e1955cd69a681b7bfcecdf9fe37c0a878dfc5146a36ebb02562e" exitCode=0
Jul 28 00:55:30.901667 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.901509    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57b5878bcc-6jdm7"
Jul 28 00:55:30.901667 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.901521    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57b5878bcc-6jdm7" event={"ID":"2e5723ff-4e24-4bcb-ae69-589b5150145f","Type":"ContainerDied","Data":"765c2ffbdae6e1955cd69a681b7bfcecdf9fe37c0a878dfc5146a36ebb02562e"}
Jul 28 00:55:30.901667 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.901571    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57b5878bcc-6jdm7" event={"ID":"2e5723ff-4e24-4bcb-ae69-589b5150145f","Type":"ContainerDied","Data":"09e9e4d44a3c775e2f5fb89b6dbfa0e104ef7eb13dda9fbb025c0d3c6d6adda6"}
Jul 28 00:55:30.901667 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.901599    2578 scope.go:117] "RemoveContainer" containerID="765c2ffbdae6e1955cd69a681b7bfcecdf9fe37c0a878dfc5146a36ebb02562e"
Jul 28 00:55:30.910488 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.910469    2578 scope.go:117] "RemoveContainer" containerID="765c2ffbdae6e1955cd69a681b7bfcecdf9fe37c0a878dfc5146a36ebb02562e"
Jul 28 00:55:30.910764 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:55:30.910744    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"765c2ffbdae6e1955cd69a681b7bfcecdf9fe37c0a878dfc5146a36ebb02562e\": container with ID starting with 765c2ffbdae6e1955cd69a681b7bfcecdf9fe37c0a878dfc5146a36ebb02562e not found: ID does not exist" containerID="765c2ffbdae6e1955cd69a681b7bfcecdf9fe37c0a878dfc5146a36ebb02562e"
Jul 28 00:55:30.910833 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.910777    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"765c2ffbdae6e1955cd69a681b7bfcecdf9fe37c0a878dfc5146a36ebb02562e"} err="failed to get container status \"765c2ffbdae6e1955cd69a681b7bfcecdf9fe37c0a878dfc5146a36ebb02562e\": rpc error: code = NotFound desc = could not find container \"765c2ffbdae6e1955cd69a681b7bfcecdf9fe37c0a878dfc5146a36ebb02562e\": container with ID starting with 765c2ffbdae6e1955cd69a681b7bfcecdf9fe37c0a878dfc5146a36ebb02562e not found: ID does not exist"
Jul 28 00:55:30.915990 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.915967    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57b5878bcc-6jdm7"]
Jul 28 00:55:30.920018 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:30.919997    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57b5878bcc-6jdm7"]
Jul 28 00:55:32.405500 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:32.405467    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e5723ff-4e24-4bcb-ae69-589b5150145f" path="/var/lib/kubelet/pods/2e5723ff-4e24-4bcb-ae69-589b5150145f/volumes"
Jul 28 00:55:33.573033 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:33.573005    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796"
Jul 28 00:55:33.576950 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:33.576930    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6f59fc7c69-d2796"
Jul 28 00:55:35.799892 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:35.799863    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-ff2f2"
Jul 28 00:55:41.323466 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.323421    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7684df49c5-5q4bb" podUID="5d905293-5622-4ad4-ba6a-aece516c7ea6" containerName="console" containerID="cri-o://ac9827b995b3b6cb1fb94ac244da5eb8849d414d226fae0b4bd8933223c61112" gracePeriod=15
Jul 28 00:55:41.567945 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.567917    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:55:41.712473 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.712430    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d905293-5622-4ad4-ba6a-aece516c7ea6-trusted-ca-bundle\") pod \"5d905293-5622-4ad4-ba6a-aece516c7ea6\" (UID: \"5d905293-5622-4ad4-ba6a-aece516c7ea6\") "
Jul 28 00:55:41.712648 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.712493    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5d905293-5622-4ad4-ba6a-aece516c7ea6-oauth-serving-cert\") pod \"5d905293-5622-4ad4-ba6a-aece516c7ea6\" (UID: \"5d905293-5622-4ad4-ba6a-aece516c7ea6\") "
Jul 28 00:55:41.712648 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.712518    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5d905293-5622-4ad4-ba6a-aece516c7ea6-console-config\") pod \"5d905293-5622-4ad4-ba6a-aece516c7ea6\" (UID: \"5d905293-5622-4ad4-ba6a-aece516c7ea6\") "
Jul 28 00:55:41.712648 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.712538    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lfvz\" (UniqueName: \"kubernetes.io/projected/5d905293-5622-4ad4-ba6a-aece516c7ea6-kube-api-access-4lfvz\") pod \"5d905293-5622-4ad4-ba6a-aece516c7ea6\" (UID: \"5d905293-5622-4ad4-ba6a-aece516c7ea6\") "
Jul 28 00:55:41.712648 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.712584    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5d905293-5622-4ad4-ba6a-aece516c7ea6-console-oauth-config\") pod \"5d905293-5622-4ad4-ba6a-aece516c7ea6\" (UID: \"5d905293-5622-4ad4-ba6a-aece516c7ea6\") "
Jul 28 00:55:41.712648 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.712610    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5d905293-5622-4ad4-ba6a-aece516c7ea6-service-ca\") pod \"5d905293-5622-4ad4-ba6a-aece516c7ea6\" (UID: \"5d905293-5622-4ad4-ba6a-aece516c7ea6\") "
Jul 28 00:55:41.712648 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.712633    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d905293-5622-4ad4-ba6a-aece516c7ea6-console-serving-cert\") pod \"5d905293-5622-4ad4-ba6a-aece516c7ea6\" (UID: \"5d905293-5622-4ad4-ba6a-aece516c7ea6\") "
Jul 28 00:55:41.713045 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.712912    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d905293-5622-4ad4-ba6a-aece516c7ea6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5d905293-5622-4ad4-ba6a-aece516c7ea6" (UID: "5d905293-5622-4ad4-ba6a-aece516c7ea6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 00:55:41.713184 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.712910    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d905293-5622-4ad4-ba6a-aece516c7ea6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5d905293-5622-4ad4-ba6a-aece516c7ea6" (UID: "5d905293-5622-4ad4-ba6a-aece516c7ea6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 00:55:41.713184 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.713091    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d905293-5622-4ad4-ba6a-aece516c7ea6-service-ca" (OuterVolumeSpecName: "service-ca") pod "5d905293-5622-4ad4-ba6a-aece516c7ea6" (UID: "5d905293-5622-4ad4-ba6a-aece516c7ea6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 00:55:41.713303 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.713187    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d905293-5622-4ad4-ba6a-aece516c7ea6-console-config" (OuterVolumeSpecName: "console-config") pod "5d905293-5622-4ad4-ba6a-aece516c7ea6" (UID: "5d905293-5622-4ad4-ba6a-aece516c7ea6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 00:55:41.714902 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.714872    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d905293-5622-4ad4-ba6a-aece516c7ea6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5d905293-5622-4ad4-ba6a-aece516c7ea6" (UID: "5d905293-5622-4ad4-ba6a-aece516c7ea6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 00:55:41.714985 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.714900    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d905293-5622-4ad4-ba6a-aece516c7ea6-kube-api-access-4lfvz" (OuterVolumeSpecName: "kube-api-access-4lfvz") pod "5d905293-5622-4ad4-ba6a-aece516c7ea6" (UID: "5d905293-5622-4ad4-ba6a-aece516c7ea6"). InnerVolumeSpecName "kube-api-access-4lfvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 00:55:41.714985 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.714914    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d905293-5622-4ad4-ba6a-aece516c7ea6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5d905293-5622-4ad4-ba6a-aece516c7ea6" (UID: "5d905293-5622-4ad4-ba6a-aece516c7ea6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 00:55:41.813490 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.813438    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d905293-5622-4ad4-ba6a-aece516c7ea6-trusted-ca-bundle\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 00:55:41.813490 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.813486    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5d905293-5622-4ad4-ba6a-aece516c7ea6-oauth-serving-cert\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 00:55:41.813490 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.813496    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5d905293-5622-4ad4-ba6a-aece516c7ea6-console-config\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 00:55:41.813490 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.813505    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lfvz\" (UniqueName: \"kubernetes.io/projected/5d905293-5622-4ad4-ba6a-aece516c7ea6-kube-api-access-4lfvz\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 00:55:41.813740 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.813514    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5d905293-5622-4ad4-ba6a-aece516c7ea6-console-oauth-config\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 00:55:41.813740 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.813522    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5d905293-5622-4ad4-ba6a-aece516c7ea6-service-ca\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 00:55:41.813740 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.813531    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d905293-5622-4ad4-ba6a-aece516c7ea6-console-serving-cert\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 00:55:41.948307 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.948268    2578 generic.go:356] "Generic (PLEG): container finished" podID="5d905293-5622-4ad4-ba6a-aece516c7ea6" containerID="ac9827b995b3b6cb1fb94ac244da5eb8849d414d226fae0b4bd8933223c61112" exitCode=0
Jul 28 00:55:41.948461 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.948314    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7684df49c5-5q4bb"
Jul 28 00:55:41.948461 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.948315    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7684df49c5-5q4bb" event={"ID":"5d905293-5622-4ad4-ba6a-aece516c7ea6","Type":"ContainerDied","Data":"ac9827b995b3b6cb1fb94ac244da5eb8849d414d226fae0b4bd8933223c61112"}
Jul 28 00:55:41.948461 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.948440    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7684df49c5-5q4bb" event={"ID":"5d905293-5622-4ad4-ba6a-aece516c7ea6","Type":"ContainerDied","Data":"266749483dba489d55e11ded41664b0374b379e034a64931323c4862cc92169e"}
Jul 28 00:55:41.948569 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.948464    2578 scope.go:117] "RemoveContainer" containerID="ac9827b995b3b6cb1fb94ac244da5eb8849d414d226fae0b4bd8933223c61112"
Jul 28 00:55:41.958776 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.958756    2578 scope.go:117] "RemoveContainer" containerID="ac9827b995b3b6cb1fb94ac244da5eb8849d414d226fae0b4bd8933223c61112"
Jul 28 00:55:41.959053 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:55:41.959033    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac9827b995b3b6cb1fb94ac244da5eb8849d414d226fae0b4bd8933223c61112\": container with ID starting with ac9827b995b3b6cb1fb94ac244da5eb8849d414d226fae0b4bd8933223c61112 not found: ID does not exist" containerID="ac9827b995b3b6cb1fb94ac244da5eb8849d414d226fae0b4bd8933223c61112"
Jul 28 00:55:41.959110 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.959067    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac9827b995b3b6cb1fb94ac244da5eb8849d414d226fae0b4bd8933223c61112"} err="failed to get container status \"ac9827b995b3b6cb1fb94ac244da5eb8849d414d226fae0b4bd8933223c61112\": rpc error: code = NotFound desc = could not find container \"ac9827b995b3b6cb1fb94ac244da5eb8849d414d226fae0b4bd8933223c61112\": container with ID starting with ac9827b995b3b6cb1fb94ac244da5eb8849d414d226fae0b4bd8933223c61112 not found: ID does not exist"
Jul 28 00:55:41.971966 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.971906    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7684df49c5-5q4bb"]
Jul 28 00:55:41.975607 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:41.975583    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7684df49c5-5q4bb"]
Jul 28 00:55:42.400840 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:42.400806    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d905293-5622-4ad4-ba6a-aece516c7ea6" path="/var/lib/kubelet/pods/5d905293-5622-4ad4-ba6a-aece516c7ea6/volumes"
Jul 28 00:55:55.445753 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:55.445714    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:55:55.465307 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:55.465279    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:55:56.027167 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:55:56.027142    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 00:57:24.844539 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:24.844499    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-vdgg2"]
Jul 28 00:57:24.845100 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:24.844834    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e5723ff-4e24-4bcb-ae69-589b5150145f" containerName="console"
Jul 28 00:57:24.845100 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:24.844850    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e5723ff-4e24-4bcb-ae69-589b5150145f" containerName="console"
Jul 28 00:57:24.845100 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:24.844886    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d905293-5622-4ad4-ba6a-aece516c7ea6" containerName="console"
Jul 28 00:57:24.845100 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:24.844892    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d905293-5622-4ad4-ba6a-aece516c7ea6" containerName="console"
Jul 28 00:57:24.845100 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:24.844946    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e5723ff-4e24-4bcb-ae69-589b5150145f" containerName="console"
Jul 28 00:57:24.845100 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:24.844957    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d905293-5622-4ad4-ba6a-aece516c7ea6" containerName="console"
Jul 28 00:57:24.847910 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:24.847889    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-vdgg2"
Jul 28 00:57:24.850495 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:24.850475    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 28 00:57:24.850696 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:24.850683    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-bjjqb\""
Jul 28 00:57:24.851460 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:24.851448    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 28 00:57:24.857941 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:24.857920    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-vdgg2"]
Jul 28 00:57:24.912789 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:24.912754    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddq46\" (UniqueName: \"kubernetes.io/projected/4ebf0964-9de3-47ec-b1fb-c0d272289986-kube-api-access-ddq46\") pod \"cert-manager-webhook-6486ffd997-vdgg2\" (UID: \"4ebf0964-9de3-47ec-b1fb-c0d272289986\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vdgg2"
Jul 28 00:57:24.912958 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:24.912831    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ebf0964-9de3-47ec-b1fb-c0d272289986-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-vdgg2\" (UID: \"4ebf0964-9de3-47ec-b1fb-c0d272289986\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vdgg2"
Jul 28 00:57:25.014128 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:25.014095    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ebf0964-9de3-47ec-b1fb-c0d272289986-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-vdgg2\" (UID: \"4ebf0964-9de3-47ec-b1fb-c0d272289986\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vdgg2"
Jul 28 00:57:25.014310 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:25.014146    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddq46\" (UniqueName: \"kubernetes.io/projected/4ebf0964-9de3-47ec-b1fb-c0d272289986-kube-api-access-ddq46\") pod \"cert-manager-webhook-6486ffd997-vdgg2\" (UID: \"4ebf0964-9de3-47ec-b1fb-c0d272289986\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vdgg2"
Jul 28 00:57:25.022254 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:25.022230    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ebf0964-9de3-47ec-b1fb-c0d272289986-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-vdgg2\" (UID: \"4ebf0964-9de3-47ec-b1fb-c0d272289986\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vdgg2"
Jul 28 00:57:25.022981 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:25.022958    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddq46\" (UniqueName: \"kubernetes.io/projected/4ebf0964-9de3-47ec-b1fb-c0d272289986-kube-api-access-ddq46\") pod \"cert-manager-webhook-6486ffd997-vdgg2\" (UID: \"4ebf0964-9de3-47ec-b1fb-c0d272289986\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vdgg2"
Jul 28 00:57:25.168751 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:25.168717    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-vdgg2"
Jul 28 00:57:25.297752 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:25.297723    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-vdgg2"]
Jul 28 00:57:25.299837 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:57:25.299810    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ebf0964_9de3_47ec_b1fb_c0d272289986.slice/crio-87a6c8a5b9bcba31158837b1f4d9d9ad558b52a5527f9274ebd45f96a22afed8 WatchSource:0}: Error finding container 87a6c8a5b9bcba31158837b1f4d9d9ad558b52a5527f9274ebd45f96a22afed8: Status 404 returned error can't find the container with id 87a6c8a5b9bcba31158837b1f4d9d9ad558b52a5527f9274ebd45f96a22afed8
Jul 28 00:57:25.364287 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:25.364255    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-vdgg2" event={"ID":"4ebf0964-9de3-47ec-b1fb-c0d272289986","Type":"ContainerStarted","Data":"87a6c8a5b9bcba31158837b1f4d9d9ad558b52a5527f9274ebd45f96a22afed8"}
Jul 28 00:57:29.384195 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:29.384162    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-vdgg2" event={"ID":"4ebf0964-9de3-47ec-b1fb-c0d272289986","Type":"ContainerStarted","Data":"cb6cb77c33942de4aa89fcfda3f8d64863b22805ca4a3cf8ebb762b8aabcd6ea"}
Jul 28 00:57:29.384662 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:29.384304    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-vdgg2"
Jul 28 00:57:29.419007 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:29.418867    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-vdgg2" podStartSLOduration=2.241361028 podStartE2EDuration="5.418847922s" totalImagesPullingTime="3.177486894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:57:29.417846258 +0000 UTC m=+219.582654364" watchObservedRunningTime="2026-07-28 00:57:29.418847922 +0000 UTC m=+219.583656030"
Jul 28 00:57:35.207288 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:35.207256    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n59n4"]
Jul 28 00:57:35.210537 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:35.210514    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n59n4"
Jul 28 00:57:35.212804 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:35.212783    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 28 00:57:35.212914 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:35.212877    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 28 00:57:35.213710 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:35.213696    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-zbj2x\""
Jul 28 00:57:35.219319 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:35.219294    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n59n4"]
Jul 28 00:57:35.311254 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:35.311197    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t2v4\" (UniqueName: \"kubernetes.io/projected/c836bcbc-cfd0-4190-8b9e-eb4bf3967557-kube-api-access-5t2v4\") pod \"openshift-lws-operator-fd8ccff4c-n59n4\" (UID: \"c836bcbc-cfd0-4190-8b9e-eb4bf3967557\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n59n4"
Jul 28 00:57:35.311428 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:35.311387    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c836bcbc-cfd0-4190-8b9e-eb4bf3967557-tmp\") pod \"openshift-lws-operator-fd8ccff4c-n59n4\" (UID: \"c836bcbc-cfd0-4190-8b9e-eb4bf3967557\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n59n4"
Jul 28 00:57:35.391779 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:35.391747    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-vdgg2"
Jul 28 00:57:35.412491 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:35.412462    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c836bcbc-cfd0-4190-8b9e-eb4bf3967557-tmp\") pod \"openshift-lws-operator-fd8ccff4c-n59n4\" (UID: \"c836bcbc-cfd0-4190-8b9e-eb4bf3967557\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n59n4"
Jul 28 00:57:35.412669 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:35.412522    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t2v4\" (UniqueName: \"kubernetes.io/projected/c836bcbc-cfd0-4190-8b9e-eb4bf3967557-kube-api-access-5t2v4\") pod \"openshift-lws-operator-fd8ccff4c-n59n4\" (UID: \"c836bcbc-cfd0-4190-8b9e-eb4bf3967557\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n59n4"
Jul 28 00:57:35.412905 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:35.412885    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c836bcbc-cfd0-4190-8b9e-eb4bf3967557-tmp\") pod \"openshift-lws-operator-fd8ccff4c-n59n4\" (UID: \"c836bcbc-cfd0-4190-8b9e-eb4bf3967557\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n59n4"
Jul 28 00:57:35.420264 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:35.420230    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t2v4\" (UniqueName: \"kubernetes.io/projected/c836bcbc-cfd0-4190-8b9e-eb4bf3967557-kube-api-access-5t2v4\") pod \"openshift-lws-operator-fd8ccff4c-n59n4\" (UID: \"c836bcbc-cfd0-4190-8b9e-eb4bf3967557\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n59n4"
Jul 28 00:57:35.519662 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:35.519586    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n59n4"
Jul 28 00:57:35.648079 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:35.648057    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n59n4"]
Jul 28 00:57:35.650338 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:57:35.650311    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc836bcbc_cfd0_4190_8b9e_eb4bf3967557.slice/crio-4da7bdf5ab7d28863f43d8b2c970143be924e268e77d949a9b02cbe9f7f53c45 WatchSource:0}: Error finding container 4da7bdf5ab7d28863f43d8b2c970143be924e268e77d949a9b02cbe9f7f53c45: Status 404 returned error can't find the container with id 4da7bdf5ab7d28863f43d8b2c970143be924e268e77d949a9b02cbe9f7f53c45
Jul 28 00:57:36.416162 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:36.416124    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n59n4" event={"ID":"c836bcbc-cfd0-4190-8b9e-eb4bf3967557","Type":"ContainerStarted","Data":"4da7bdf5ab7d28863f43d8b2c970143be924e268e77d949a9b02cbe9f7f53c45"}
Jul 28 00:57:38.426326 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:38.426291    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n59n4" event={"ID":"c836bcbc-cfd0-4190-8b9e-eb4bf3967557","Type":"ContainerStarted","Data":"9d9c0effa8be476569ebd1d54e3c9cbec886bb04c72d3e9028ea212b79472159"}
Jul 28 00:57:38.443489 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:38.443438    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n59n4" podStartSLOduration=0.892487564 podStartE2EDuration="3.443421285s" totalImagesPullingTime="2.550933721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:57:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:57:38.442265381 +0000 UTC m=+228.607073491" watchObservedRunningTime="2026-07-28 00:57:38.443421285 +0000 UTC m=+228.608229391"
Jul 28 00:57:56.654281 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:56.654241    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6549c9954f-b8gxc"]
Jul 28 00:57:56.695357 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:56.695323    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6549c9954f-b8gxc"]
Jul 28 00:57:56.695505 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:56.695424    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6549c9954f-b8gxc"
Jul 28 00:57:56.698436 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:56.698414    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 00:57:56.698581 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:56.698443    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 00:57:56.698581 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:56.698458    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cm2gp\""
Jul 28 00:57:56.698740 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:56.698724    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 00:57:56.701184 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:56.701157    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 00:57:56.807556 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:56.807516    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ccd038af-eb0f-4256-9b99-f6aed1604b13-tmp\") pod \"opendatahub-operator-controller-manager-6549c9954f-b8gxc\" (UID: \"ccd038af-eb0f-4256-9b99-f6aed1604b13\") " pod="opendatahub/opendatahub-operator-controller-manager-6549c9954f-b8gxc"
Jul 28 00:57:56.807747 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:56.807568    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ccd038af-eb0f-4256-9b99-f6aed1604b13-manifests\") pod \"opendatahub-operator-controller-manager-6549c9954f-b8gxc\" (UID: \"ccd038af-eb0f-4256-9b99-f6aed1604b13\") " pod="opendatahub/opendatahub-operator-controller-manager-6549c9954f-b8gxc"
Jul 28 00:57:56.807747 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:56.807653    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ccd038af-eb0f-4256-9b99-f6aed1604b13-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6549c9954f-b8gxc\" (UID: \"ccd038af-eb0f-4256-9b99-f6aed1604b13\") " pod="opendatahub/opendatahub-operator-controller-manager-6549c9954f-b8gxc"
Jul 28 00:57:56.807747 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:56.807681    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z27dj\" (UniqueName: \"kubernetes.io/projected/ccd038af-eb0f-4256-9b99-f6aed1604b13-kube-api-access-z27dj\") pod \"opendatahub-operator-controller-manager-6549c9954f-b8gxc\" (UID: \"ccd038af-eb0f-4256-9b99-f6aed1604b13\") " pod="opendatahub/opendatahub-operator-controller-manager-6549c9954f-b8gxc"
Jul 28 00:57:56.807747 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:56.807729    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ccd038af-eb0f-4256-9b99-f6aed1604b13-webhook-cert\") pod \"opendatahub-operator-controller-manager-6549c9954f-b8gxc\" (UID: \"ccd038af-eb0f-4256-9b99-f6aed1604b13\") " pod="opendatahub/opendatahub-operator-controller-manager-6549c9954f-b8gxc"
Jul 28 00:57:56.909068 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:56.908978    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ccd038af-eb0f-4256-9b99-f6aed1604b13-tmp\") pod \"opendatahub-operator-controller-manager-6549c9954f-b8gxc\" (UID: \"ccd038af-eb0f-4256-9b99-f6aed1604b13\") " pod="opendatahub/opendatahub-operator-controller-manager-6549c9954f-b8gxc"
Jul 28 00:57:56.909068 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:56.909039    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ccd038af-eb0f-4256-9b99-f6aed1604b13-manifests\") pod \"opendatahub-operator-controller-manager-6549c9954f-b8gxc\" (UID: \"ccd038af-eb0f-4256-9b99-f6aed1604b13\") " pod="opendatahub/opendatahub-operator-controller-manager-6549c9954f-b8gxc"
Jul 28 00:57:56.909341 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:56.909097    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ccd038af-eb0f-4256-9b99-f6aed1604b13-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6549c9954f-b8gxc\" (UID: \"ccd038af-eb0f-4256-9b99-f6aed1604b13\") " pod="opendatahub/opendatahub-operator-controller-manager-6549c9954f-b8gxc"
Jul 28 00:57:56.909341 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:56.909130    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z27dj\" (UniqueName: \"kubernetes.io/projected/ccd038af-eb0f-4256-9b99-f6aed1604b13-kube-api-access-z27dj\") pod \"opendatahub-operator-controller-manager-6549c9954f-b8gxc\" (UID: \"ccd038af-eb0f-4256-9b99-f6aed1604b13\") " pod="opendatahub/opendatahub-operator-controller-manager-6549c9954f-b8gxc"
Jul 28 00:57:56.909341 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:56.909187    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ccd038af-eb0f-4256-9b99-f6aed1604b13-webhook-cert\") pod \"opendatahub-operator-controller-manager-6549c9954f-b8gxc\" (UID: \"ccd038af-eb0f-4256-9b99-f6aed1604b13\") " pod="opendatahub/opendatahub-operator-controller-manager-6549c9954f-b8gxc"
Jul 28 00:57:56.909572 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:56.909541    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ccd038af-eb0f-4256-9b99-f6aed1604b13-manifests\") pod \"opendatahub-operator-controller-manager-6549c9954f-b8gxc\" (UID: \"ccd038af-eb0f-4256-9b99-f6aed1604b13\") " pod="opendatahub/opendatahub-operator-controller-manager-6549c9954f-b8gxc"
Jul 28 00:57:56.911364 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:56.911338    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ccd038af-eb0f-4256-9b99-f6aed1604b13-tmp\") pod \"opendatahub-operator-controller-manager-6549c9954f-b8gxc\" (UID: \"ccd038af-eb0f-4256-9b99-f6aed1604b13\") " pod="opendatahub/opendatahub-operator-controller-manager-6549c9954f-b8gxc"
Jul 28 00:57:56.911572 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:56.911557    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ccd038af-eb0f-4256-9b99-f6aed1604b13-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6549c9954f-b8gxc\" (UID: \"ccd038af-eb0f-4256-9b99-f6aed1604b13\") " pod="opendatahub/opendatahub-operator-controller-manager-6549c9954f-b8gxc"
Jul 28 00:57:56.911704 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:56.911685    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ccd038af-eb0f-4256-9b99-f6aed1604b13-webhook-cert\") pod \"opendatahub-operator-controller-manager-6549c9954f-b8gxc\" (UID: \"ccd038af-eb0f-4256-9b99-f6aed1604b13\") " pod="opendatahub/opendatahub-operator-controller-manager-6549c9954f-b8gxc"
Jul 28 00:57:56.928114 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:56.928092    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z27dj\" (UniqueName: \"kubernetes.io/projected/ccd038af-eb0f-4256-9b99-f6aed1604b13-kube-api-access-z27dj\") pod \"opendatahub-operator-controller-manager-6549c9954f-b8gxc\" (UID: \"ccd038af-eb0f-4256-9b99-f6aed1604b13\") " pod="opendatahub/opendatahub-operator-controller-manager-6549c9954f-b8gxc"
Jul 28 00:57:57.005933 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:57.005900    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6549c9954f-b8gxc"
Jul 28 00:57:57.184066 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:57.184034    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6549c9954f-b8gxc"]
Jul 28 00:57:57.185917 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:57:57.185888    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccd038af_eb0f_4256_9b99_f6aed1604b13.slice/crio-43ff1f7d44c969333c986ff8dca2050c535a5ff9acf0880b462085e1104b3d82 WatchSource:0}: Error finding container 43ff1f7d44c969333c986ff8dca2050c535a5ff9acf0880b462085e1104b3d82: Status 404 returned error can't find the container with id 43ff1f7d44c969333c986ff8dca2050c535a5ff9acf0880b462085e1104b3d82
Jul 28 00:57:57.508846 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:57:57.508758    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6549c9954f-b8gxc" event={"ID":"ccd038af-eb0f-4256-9b99-f6aed1604b13","Type":"ContainerStarted","Data":"43ff1f7d44c969333c986ff8dca2050c535a5ff9acf0880b462085e1104b3d82"}
Jul 28 00:58:00.525426 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:00.525388    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6549c9954f-b8gxc" event={"ID":"ccd038af-eb0f-4256-9b99-f6aed1604b13","Type":"ContainerStarted","Data":"49a4266f5e894cc3d95f490df352d038243ba5c2053c74876e05a00c19a48d1f"}
Jul 28 00:58:01.532410 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:01.532319    2578 generic.go:356] "Generic (PLEG): container finished" podID="ccd038af-eb0f-4256-9b99-f6aed1604b13" containerID="49a4266f5e894cc3d95f490df352d038243ba5c2053c74876e05a00c19a48d1f" exitCode=0
Jul 28 00:58:01.532410 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:01.532375    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6549c9954f-b8gxc" event={"ID":"ccd038af-eb0f-4256-9b99-f6aed1604b13","Type":"ContainerDied","Data":"49a4266f5e894cc3d95f490df352d038243ba5c2053c74876e05a00c19a48d1f"}
Jul 28 00:58:02.539174 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:02.539136    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6549c9954f-b8gxc" event={"ID":"ccd038af-eb0f-4256-9b99-f6aed1604b13","Type":"ContainerStarted","Data":"0294b889c592a29ea128c0236c8686fc0f785a7eff0c7fc71dbb3cab097e92e3"}
Jul 28 00:58:02.539563 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:02.539249    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6549c9954f-b8gxc"
Jul 28 00:58:02.562936 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:02.562881    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6549c9954f-b8gxc" podStartSLOduration=2.310418081 podStartE2EDuration="6.562862912s" totalImagesPullingTime="3.182495967s" totalInitContainerRuntime="1.069948864s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:57:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:58:02.560103378 +0000 UTC m=+252.724911483" watchObservedRunningTime="2026-07-28 00:58:02.562862912 +0000 UTC m=+252.727671018"
Jul 28 00:58:13.545901 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:13.545826    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6549c9954f-b8gxc"
Jul 28 00:58:13.852495 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:13.852418    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6478f6f65f-s844n"]
Jul 28 00:58:13.855916 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:13.855896    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6478f6f65f-s844n"
Jul 28 00:58:13.858356 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:13.858331    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 28 00:58:13.858489 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:13.858340    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-84b42\""
Jul 28 00:58:13.858489 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:13.858423    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 28 00:58:13.858489 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:13.858450    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 28 00:58:13.865080 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:13.865058    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6478f6f65f-s844n"]
Jul 28 00:58:13.959790 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:13.959760    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8b90f209-f469-48bc-8884-de4eb961649f-manager-config\") pod \"lws-controller-manager-6478f6f65f-s844n\" (UID: \"8b90f209-f469-48bc-8884-de4eb961649f\") " pod="openshift-lws-operator/lws-controller-manager-6478f6f65f-s844n"
Jul 28 00:58:13.959945 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:13.959811    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b90f209-f469-48bc-8884-de4eb961649f-cert\") pod \"lws-controller-manager-6478f6f65f-s844n\" (UID: \"8b90f209-f469-48bc-8884-de4eb961649f\") " pod="openshift-lws-operator/lws-controller-manager-6478f6f65f-s844n"
Jul 28 00:58:13.959945 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:13.959843    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t85ql\" (UniqueName: \"kubernetes.io/projected/8b90f209-f469-48bc-8884-de4eb961649f-kube-api-access-t85ql\") pod \"lws-controller-manager-6478f6f65f-s844n\" (UID: \"8b90f209-f469-48bc-8884-de4eb961649f\") " pod="openshift-lws-operator/lws-controller-manager-6478f6f65f-s844n"
Jul 28 00:58:13.959945 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:13.959899    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b90f209-f469-48bc-8884-de4eb961649f-metrics-cert\") pod \"lws-controller-manager-6478f6f65f-s844n\" (UID: \"8b90f209-f469-48bc-8884-de4eb961649f\") " pod="openshift-lws-operator/lws-controller-manager-6478f6f65f-s844n"
Jul 28 00:58:14.061179 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:14.061141    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b90f209-f469-48bc-8884-de4eb961649f-cert\") pod \"lws-controller-manager-6478f6f65f-s844n\" (UID: \"8b90f209-f469-48bc-8884-de4eb961649f\") " pod="openshift-lws-operator/lws-controller-manager-6478f6f65f-s844n"
Jul 28 00:58:14.061403 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:14.061259    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t85ql\" (UniqueName: \"kubernetes.io/projected/8b90f209-f469-48bc-8884-de4eb961649f-kube-api-access-t85ql\") pod \"lws-controller-manager-6478f6f65f-s844n\" (UID: \"8b90f209-f469-48bc-8884-de4eb961649f\") " pod="openshift-lws-operator/lws-controller-manager-6478f6f65f-s844n"
Jul 28 00:58:14.061403 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:14.061316    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b90f209-f469-48bc-8884-de4eb961649f-metrics-cert\") pod \"lws-controller-manager-6478f6f65f-s844n\" (UID: \"8b90f209-f469-48bc-8884-de4eb961649f\") " pod="openshift-lws-operator/lws-controller-manager-6478f6f65f-s844n"
Jul 28 00:58:14.061517 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:14.061432    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8b90f209-f469-48bc-8884-de4eb961649f-manager-config\") pod \"lws-controller-manager-6478f6f65f-s844n\" (UID: \"8b90f209-f469-48bc-8884-de4eb961649f\") " pod="openshift-lws-operator/lws-controller-manager-6478f6f65f-s844n"
Jul 28 00:58:14.062000 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:14.061975    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8b90f209-f469-48bc-8884-de4eb961649f-manager-config\") pod \"lws-controller-manager-6478f6f65f-s844n\" (UID: \"8b90f209-f469-48bc-8884-de4eb961649f\") " pod="openshift-lws-operator/lws-controller-manager-6478f6f65f-s844n"
Jul 28 00:58:14.063808 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:14.063782    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b90f209-f469-48bc-8884-de4eb961649f-cert\") pod \"lws-controller-manager-6478f6f65f-s844n\" (UID: \"8b90f209-f469-48bc-8884-de4eb961649f\") " pod="openshift-lws-operator/lws-controller-manager-6478f6f65f-s844n"
Jul 28 00:58:14.063962 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:14.063797    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b90f209-f469-48bc-8884-de4eb961649f-metrics-cert\") pod \"lws-controller-manager-6478f6f65f-s844n\" (UID: \"8b90f209-f469-48bc-8884-de4eb961649f\") " pod="openshift-lws-operator/lws-controller-manager-6478f6f65f-s844n"
Jul 28 00:58:14.071769 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:14.071743    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t85ql\" (UniqueName: \"kubernetes.io/projected/8b90f209-f469-48bc-8884-de4eb961649f-kube-api-access-t85ql\") pod \"lws-controller-manager-6478f6f65f-s844n\" (UID: \"8b90f209-f469-48bc-8884-de4eb961649f\") " pod="openshift-lws-operator/lws-controller-manager-6478f6f65f-s844n"
Jul 28 00:58:14.165778 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:14.165743    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6478f6f65f-s844n"
Jul 28 00:58:14.294346 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:14.294320    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6478f6f65f-s844n"]
Jul 28 00:58:14.296285 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:58:14.296240    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b90f209_f469_48bc_8884_de4eb961649f.slice/crio-20afd7469073515dcdfa6ebeeb3205c8e8f412796d59614b27f6098d5b645a83 WatchSource:0}: Error finding container 20afd7469073515dcdfa6ebeeb3205c8e8f412796d59614b27f6098d5b645a83: Status 404 returned error can't find the container with id 20afd7469073515dcdfa6ebeeb3205c8e8f412796d59614b27f6098d5b645a83
Jul 28 00:58:14.593707 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:14.593622    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6478f6f65f-s844n" event={"ID":"8b90f209-f469-48bc-8884-de4eb961649f","Type":"ContainerStarted","Data":"20afd7469073515dcdfa6ebeeb3205c8e8f412796d59614b27f6098d5b645a83"}
Jul 28 00:58:16.462054 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:16.462017    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-fbf9bd945-z8bb2"]
Jul 28 00:58:16.466282 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:16.466152    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-fbf9bd945-z8bb2"
Jul 28 00:58:16.469149 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:16.469127    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 28 00:58:16.469347 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:16.469324    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 00:58:16.469523 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:16.469324    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 00:58:16.469624 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:16.469375    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 28 00:58:16.469624 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:16.469388    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-7vwnk\""
Jul 28 00:58:16.479459 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:16.479437    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-fbf9bd945-z8bb2"]
Jul 28 00:58:16.589602 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:16.589571    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgn9k\" (UniqueName: \"kubernetes.io/projected/da1e8311-1c0d-4ff5-b203-74171e7c80e8-kube-api-access-dgn9k\") pod \"kube-auth-proxy-fbf9bd945-z8bb2\" (UID: \"da1e8311-1c0d-4ff5-b203-74171e7c80e8\") " pod="openshift-ingress/kube-auth-proxy-fbf9bd945-z8bb2"
Jul 28 00:58:16.589777 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:16.589611    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da1e8311-1c0d-4ff5-b203-74171e7c80e8-tls-certs\") pod \"kube-auth-proxy-fbf9bd945-z8bb2\" (UID: \"da1e8311-1c0d-4ff5-b203-74171e7c80e8\") " pod="openshift-ingress/kube-auth-proxy-fbf9bd945-z8bb2"
Jul 28 00:58:16.589777 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:16.589646    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da1e8311-1c0d-4ff5-b203-74171e7c80e8-tmp\") pod \"kube-auth-proxy-fbf9bd945-z8bb2\" (UID: \"da1e8311-1c0d-4ff5-b203-74171e7c80e8\") " pod="openshift-ingress/kube-auth-proxy-fbf9bd945-z8bb2"
Jul 28 00:58:16.606046 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:16.606006    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6478f6f65f-s844n" event={"ID":"8b90f209-f469-48bc-8884-de4eb961649f","Type":"ContainerStarted","Data":"c0da8cea4aa07c37fae8570ef53c101a57d7151b0ecaf74ed276b8b66acbd278"}
Jul 28 00:58:16.606267 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:16.606134    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6478f6f65f-s844n"
Jul 28 00:58:16.634500 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:16.634447    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6478f6f65f-s844n" podStartSLOduration=2.056431002 podStartE2EDuration="3.634430684s" totalImagesPullingTime="1.577999682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:58:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:58:16.633322085 +0000 UTC m=+266.798130204" watchObservedRunningTime="2026-07-28 00:58:16.634430684 +0000 UTC m=+266.799238790"
Jul 28 00:58:16.691190 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:16.691152    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgn9k\" (UniqueName: \"kubernetes.io/projected/da1e8311-1c0d-4ff5-b203-74171e7c80e8-kube-api-access-dgn9k\") pod \"kube-auth-proxy-fbf9bd945-z8bb2\" (UID: \"da1e8311-1c0d-4ff5-b203-74171e7c80e8\") " pod="openshift-ingress/kube-auth-proxy-fbf9bd945-z8bb2"
Jul 28 00:58:16.691190 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:16.691196    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da1e8311-1c0d-4ff5-b203-74171e7c80e8-tls-certs\") pod \"kube-auth-proxy-fbf9bd945-z8bb2\" (UID: \"da1e8311-1c0d-4ff5-b203-74171e7c80e8\") " pod="openshift-ingress/kube-auth-proxy-fbf9bd945-z8bb2"
Jul 28 00:58:16.691418 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:16.691256    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da1e8311-1c0d-4ff5-b203-74171e7c80e8-tmp\") pod \"kube-auth-proxy-fbf9bd945-z8bb2\" (UID: \"da1e8311-1c0d-4ff5-b203-74171e7c80e8\") " pod="openshift-ingress/kube-auth-proxy-fbf9bd945-z8bb2"
Jul 28 00:58:16.691668 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:58:16.691544    2578 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 28 00:58:16.691668 ip-10-0-132-238 kubenswrapper[2578]: E0728 00:58:16.691638    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da1e8311-1c0d-4ff5-b203-74171e7c80e8-tls-certs podName:da1e8311-1c0d-4ff5-b203-74171e7c80e8 nodeName:}" failed. No retries permitted until 2026-07-28 00:58:17.191616407 +0000 UTC m=+267.356424512 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/da1e8311-1c0d-4ff5-b203-74171e7c80e8-tls-certs") pod "kube-auth-proxy-fbf9bd945-z8bb2" (UID: "da1e8311-1c0d-4ff5-b203-74171e7c80e8") : secret "kube-auth-proxy-tls" not found
Jul 28 00:58:16.693743 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:16.693718    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da1e8311-1c0d-4ff5-b203-74171e7c80e8-tmp\") pod \"kube-auth-proxy-fbf9bd945-z8bb2\" (UID: \"da1e8311-1c0d-4ff5-b203-74171e7c80e8\") " pod="openshift-ingress/kube-auth-proxy-fbf9bd945-z8bb2"
Jul 28 00:58:16.701693 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:16.701666    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgn9k\" (UniqueName: \"kubernetes.io/projected/da1e8311-1c0d-4ff5-b203-74171e7c80e8-kube-api-access-dgn9k\") pod \"kube-auth-proxy-fbf9bd945-z8bb2\" (UID: \"da1e8311-1c0d-4ff5-b203-74171e7c80e8\") " pod="openshift-ingress/kube-auth-proxy-fbf9bd945-z8bb2"
Jul 28 00:58:17.196462 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:17.196427    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da1e8311-1c0d-4ff5-b203-74171e7c80e8-tls-certs\") pod \"kube-auth-proxy-fbf9bd945-z8bb2\" (UID: \"da1e8311-1c0d-4ff5-b203-74171e7c80e8\") " pod="openshift-ingress/kube-auth-proxy-fbf9bd945-z8bb2"
Jul 28 00:58:17.199069 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:17.199042    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da1e8311-1c0d-4ff5-b203-74171e7c80e8-tls-certs\") pod \"kube-auth-proxy-fbf9bd945-z8bb2\" (UID: \"da1e8311-1c0d-4ff5-b203-74171e7c80e8\") " pod="openshift-ingress/kube-auth-proxy-fbf9bd945-z8bb2"
Jul 28 00:58:17.379200 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:17.379157    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-fbf9bd945-z8bb2"
Jul 28 00:58:17.530335 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:17.530302    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-fbf9bd945-z8bb2"]
Jul 28 00:58:17.533143 ip-10-0-132-238 kubenswrapper[2578]: W0728 00:58:17.533111    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda1e8311_1c0d_4ff5_b203_74171e7c80e8.slice/crio-104708943a3d209f76ede197b6120ae71af6237d0bfe61d571b44b5c75f4d0d5 WatchSource:0}: Error finding container 104708943a3d209f76ede197b6120ae71af6237d0bfe61d571b44b5c75f4d0d5: Status 404 returned error can't find the container with id 104708943a3d209f76ede197b6120ae71af6237d0bfe61d571b44b5c75f4d0d5
Jul 28 00:58:17.613157 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:17.613115    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-fbf9bd945-z8bb2" event={"ID":"da1e8311-1c0d-4ff5-b203-74171e7c80e8","Type":"ContainerStarted","Data":"104708943a3d209f76ede197b6120ae71af6237d0bfe61d571b44b5c75f4d0d5"}
Jul 28 00:58:20.638424 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:20.638388    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-fbf9bd945-z8bb2" event={"ID":"da1e8311-1c0d-4ff5-b203-74171e7c80e8","Type":"ContainerStarted","Data":"8ce39318852c02cde4d6f8aee921823a1edba5e1d764582178e4c85f86156a41"}
Jul 28 00:58:20.654530 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:20.654476    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-fbf9bd945-z8bb2" podStartSLOduration=2.057860164 podStartE2EDuration="4.654460075s" totalImagesPullingTime="2.596599911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:58:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:58:20.653316325 +0000 UTC m=+270.818124433" watchObservedRunningTime="2026-07-28 00:58:20.654460075 +0000 UTC m=+270.819268182"
Jul 28 00:58:27.616045 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:27.616013    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6478f6f65f-s844n"
Jul 28 00:58:50.252048 ip-10-0-132-238 kubenswrapper[2578]: I0728 00:58:50.252019    2578 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 01:00:00.278995 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:00.278959    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-bb8n5"]
Jul 28 01:00:00.281447 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:00.281428    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bb8n5"
Jul 28 01:00:00.284013 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:00.283989    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 28 01:00:00.284131 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:00.284035    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 01:00:00.284131 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:00.284058    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 01:00:00.284957 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:00.284941    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-pmw29\""
Jul 28 01:00:00.292301 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:00.292279    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-bb8n5"]
Jul 28 01:00:00.410560 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:00.410527    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqlpl\" (UniqueName: \"kubernetes.io/projected/96996e3b-d506-43f6-b825-a126c7da9720-kube-api-access-vqlpl\") pod \"dns-operator-controller-manager-864dd78c4c-bb8n5\" (UID: \"96996e3b-d506-43f6-b825-a126c7da9720\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bb8n5"
Jul 28 01:00:00.511103 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:00.511072    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqlpl\" (UniqueName: \"kubernetes.io/projected/96996e3b-d506-43f6-b825-a126c7da9720-kube-api-access-vqlpl\") pod \"dns-operator-controller-manager-864dd78c4c-bb8n5\" (UID: \"96996e3b-d506-43f6-b825-a126c7da9720\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bb8n5"
Jul 28 01:00:00.523036 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:00.523008    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqlpl\" (UniqueName: \"kubernetes.io/projected/96996e3b-d506-43f6-b825-a126c7da9720-kube-api-access-vqlpl\") pod \"dns-operator-controller-manager-864dd78c4c-bb8n5\" (UID: \"96996e3b-d506-43f6-b825-a126c7da9720\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bb8n5"
Jul 28 01:00:00.591842 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:00.591758    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bb8n5"
Jul 28 01:00:00.751785 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:00.751758    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-bb8n5"]
Jul 28 01:00:00.754973 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:00:00.754946    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96996e3b_d506_43f6_b825_a126c7da9720.slice/crio-259442a4b730339b6ab71deeffbe727db52a5059dfea05cda94c79544c8580a5 WatchSource:0}: Error finding container 259442a4b730339b6ab71deeffbe727db52a5059dfea05cda94c79544c8580a5: Status 404 returned error can't find the container with id 259442a4b730339b6ab71deeffbe727db52a5059dfea05cda94c79544c8580a5
Jul 28 01:00:00.757584 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:00.757564    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 01:00:01.117348 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:01.117314    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bb8n5" event={"ID":"96996e3b-d506-43f6-b825-a126c7da9720","Type":"ContainerStarted","Data":"259442a4b730339b6ab71deeffbe727db52a5059dfea05cda94c79544c8580a5"}
Jul 28 01:00:03.176401 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:03.176365    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gmml8"]
Jul 28 01:00:03.179531 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:03.179510    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gmml8"
Jul 28 01:00:03.182486 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:03.182454    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 28 01:00:03.182757 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:03.182736    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 28 01:00:03.182875 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:03.182812    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vq2f5\""
Jul 28 01:00:03.188245 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:03.188181    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gmml8"]
Jul 28 01:00:03.339508 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:03.339468    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/503b066d-ed59-42da-8aa1-e23fedc4157a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gmml8\" (UID: \"503b066d-ed59-42da-8aa1-e23fedc4157a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gmml8"
Jul 28 01:00:03.339687 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:03.339538    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/503b066d-ed59-42da-8aa1-e23fedc4157a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gmml8\" (UID: \"503b066d-ed59-42da-8aa1-e23fedc4157a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gmml8"
Jul 28 01:00:03.339687 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:03.339580    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhz57\" (UniqueName: \"kubernetes.io/projected/503b066d-ed59-42da-8aa1-e23fedc4157a-kube-api-access-rhz57\") pod \"kuadrant-console-plugin-66b5774985-gmml8\" (UID: \"503b066d-ed59-42da-8aa1-e23fedc4157a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gmml8"
Jul 28 01:00:03.441124 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:03.441039    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/503b066d-ed59-42da-8aa1-e23fedc4157a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gmml8\" (UID: \"503b066d-ed59-42da-8aa1-e23fedc4157a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gmml8"
Jul 28 01:00:03.441124 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:03.441111    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/503b066d-ed59-42da-8aa1-e23fedc4157a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gmml8\" (UID: \"503b066d-ed59-42da-8aa1-e23fedc4157a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gmml8"
Jul 28 01:00:03.441384 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:03.441141    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhz57\" (UniqueName: \"kubernetes.io/projected/503b066d-ed59-42da-8aa1-e23fedc4157a-kube-api-access-rhz57\") pod \"kuadrant-console-plugin-66b5774985-gmml8\" (UID: \"503b066d-ed59-42da-8aa1-e23fedc4157a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gmml8"
Jul 28 01:00:03.441384 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:00:03.441205    2578 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 28 01:00:03.441384 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:00:03.441305    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/503b066d-ed59-42da-8aa1-e23fedc4157a-plugin-serving-cert podName:503b066d-ed59-42da-8aa1-e23fedc4157a nodeName:}" failed. No retries permitted until 2026-07-28 01:00:03.941282999 +0000 UTC m=+374.106091091 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/503b066d-ed59-42da-8aa1-e23fedc4157a-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-gmml8" (UID: "503b066d-ed59-42da-8aa1-e23fedc4157a") : secret "plugin-serving-cert" not found
Jul 28 01:00:03.441881 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:03.441858    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/503b066d-ed59-42da-8aa1-e23fedc4157a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gmml8\" (UID: \"503b066d-ed59-42da-8aa1-e23fedc4157a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gmml8"
Jul 28 01:00:03.451890 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:03.451848    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhz57\" (UniqueName: \"kubernetes.io/projected/503b066d-ed59-42da-8aa1-e23fedc4157a-kube-api-access-rhz57\") pod \"kuadrant-console-plugin-66b5774985-gmml8\" (UID: \"503b066d-ed59-42da-8aa1-e23fedc4157a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gmml8"
Jul 28 01:00:03.945283 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:03.945243    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/503b066d-ed59-42da-8aa1-e23fedc4157a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gmml8\" (UID: \"503b066d-ed59-42da-8aa1-e23fedc4157a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gmml8"
Jul 28 01:00:03.947603 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:03.947579    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/503b066d-ed59-42da-8aa1-e23fedc4157a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gmml8\" (UID: \"503b066d-ed59-42da-8aa1-e23fedc4157a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gmml8"
Jul 28 01:00:04.090938 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:04.090899    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gmml8"
Jul 28 01:00:04.138723 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:04.138687    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bb8n5" event={"ID":"96996e3b-d506-43f6-b825-a126c7da9720","Type":"ContainerStarted","Data":"1106f4b4e7180f151204a7f13030537e16e4fbb15476708317293e83757f9ae3"}
Jul 28 01:00:04.138883 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:04.138782    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bb8n5"
Jul 28 01:00:04.155043 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:04.154984    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bb8n5" podStartSLOduration=1.3899844080000001 podStartE2EDuration="4.154963328s" totalImagesPullingTime="2.76497892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:00:04.154739949 +0000 UTC m=+374.319548057" watchObservedRunningTime="2026-07-28 01:00:04.154963328 +0000 UTC m=+374.319771459"
Jul 28 01:00:04.228106 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:04.228061    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gmml8"]
Jul 28 01:00:04.230350 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:00:04.230321    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod503b066d_ed59_42da_8aa1_e23fedc4157a.slice/crio-ad2ada72944129589a633e6e82ddb34716318fdd4aea3283d184af33e977842f WatchSource:0}: Error finding container ad2ada72944129589a633e6e82ddb34716318fdd4aea3283d184af33e977842f: Status 404 returned error can't find the container with id ad2ada72944129589a633e6e82ddb34716318fdd4aea3283d184af33e977842f
Jul 28 01:00:05.144816 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:05.144783    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gmml8" event={"ID":"503b066d-ed59-42da-8aa1-e23fedc4157a","Type":"ContainerStarted","Data":"ad2ada72944129589a633e6e82ddb34716318fdd4aea3283d184af33e977842f"}
Jul 28 01:00:09.415952 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.415915    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c4f87f695-hxhpn"]
Jul 28 01:00:09.427553 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.427138    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c4f87f695-hxhpn"
Jul 28 01:00:09.430244 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.430102    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-krj69\""
Jul 28 01:00:09.430451 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.430427    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 01:00:09.431507 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.431258    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c4f87f695-hxhpn"]
Jul 28 01:00:09.431507 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.431282    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 01:00:09.431507 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.431312    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 01:00:09.431507 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.431401    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 01:00:09.431507 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.431498    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 01:00:09.431872 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.431609    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 01:00:09.431872 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.431680    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 01:00:09.437043 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.437015    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 28 01:00:09.499346 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.499304    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ce759661-df26-4039-8996-0e4579d975e4-console-config\") pod \"console-5c4f87f695-hxhpn\" (UID: \"ce759661-df26-4039-8996-0e4579d975e4\") " pod="openshift-console/console-5c4f87f695-hxhpn"
Jul 28 01:00:09.499523 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.499352    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ce759661-df26-4039-8996-0e4579d975e4-service-ca\") pod \"console-5c4f87f695-hxhpn\" (UID: \"ce759661-df26-4039-8996-0e4579d975e4\") " pod="openshift-console/console-5c4f87f695-hxhpn"
Jul 28 01:00:09.499523 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.499379    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ce759661-df26-4039-8996-0e4579d975e4-oauth-serving-cert\") pod \"console-5c4f87f695-hxhpn\" (UID: \"ce759661-df26-4039-8996-0e4579d975e4\") " pod="openshift-console/console-5c4f87f695-hxhpn"
Jul 28 01:00:09.499523 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.499477    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ce759661-df26-4039-8996-0e4579d975e4-console-serving-cert\") pod \"console-5c4f87f695-hxhpn\" (UID: \"ce759661-df26-4039-8996-0e4579d975e4\") " pod="openshift-console/console-5c4f87f695-hxhpn"
Jul 28 01:00:09.499523 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.499516    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ce759661-df26-4039-8996-0e4579d975e4-console-oauth-config\") pod \"console-5c4f87f695-hxhpn\" (UID: \"ce759661-df26-4039-8996-0e4579d975e4\") " pod="openshift-console/console-5c4f87f695-hxhpn"
Jul 28 01:00:09.499710 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.499542    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce759661-df26-4039-8996-0e4579d975e4-trusted-ca-bundle\") pod \"console-5c4f87f695-hxhpn\" (UID: \"ce759661-df26-4039-8996-0e4579d975e4\") " pod="openshift-console/console-5c4f87f695-hxhpn"
Jul 28 01:00:09.499710 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.499619    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8h6r\" (UniqueName: \"kubernetes.io/projected/ce759661-df26-4039-8996-0e4579d975e4-kube-api-access-p8h6r\") pod \"console-5c4f87f695-hxhpn\" (UID: \"ce759661-df26-4039-8996-0e4579d975e4\") " pod="openshift-console/console-5c4f87f695-hxhpn"
Jul 28 01:00:09.600546 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.600461    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ce759661-df26-4039-8996-0e4579d975e4-console-config\") pod \"console-5c4f87f695-hxhpn\" (UID: \"ce759661-df26-4039-8996-0e4579d975e4\") " pod="openshift-console/console-5c4f87f695-hxhpn"
Jul 28 01:00:09.600546 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.600509    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ce759661-df26-4039-8996-0e4579d975e4-service-ca\") pod \"console-5c4f87f695-hxhpn\" (UID: \"ce759661-df26-4039-8996-0e4579d975e4\") " pod="openshift-console/console-5c4f87f695-hxhpn"
Jul 28 01:00:09.600546 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.600533    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ce759661-df26-4039-8996-0e4579d975e4-oauth-serving-cert\") pod \"console-5c4f87f695-hxhpn\" (UID: \"ce759661-df26-4039-8996-0e4579d975e4\") " pod="openshift-console/console-5c4f87f695-hxhpn"
Jul 28 01:00:09.600834 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.600576    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ce759661-df26-4039-8996-0e4579d975e4-console-serving-cert\") pod \"console-5c4f87f695-hxhpn\" (UID: \"ce759661-df26-4039-8996-0e4579d975e4\") " pod="openshift-console/console-5c4f87f695-hxhpn"
Jul 28 01:00:09.600834 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.600599    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ce759661-df26-4039-8996-0e4579d975e4-console-oauth-config\") pod \"console-5c4f87f695-hxhpn\" (UID: \"ce759661-df26-4039-8996-0e4579d975e4\") " pod="openshift-console/console-5c4f87f695-hxhpn"
Jul 28 01:00:09.600834 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.600623    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce759661-df26-4039-8996-0e4579d975e4-trusted-ca-bundle\") pod \"console-5c4f87f695-hxhpn\" (UID: \"ce759661-df26-4039-8996-0e4579d975e4\") " pod="openshift-console/console-5c4f87f695-hxhpn"
Jul 28 01:00:09.600834 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.600661    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8h6r\" (UniqueName: \"kubernetes.io/projected/ce759661-df26-4039-8996-0e4579d975e4-kube-api-access-p8h6r\") pod \"console-5c4f87f695-hxhpn\" (UID: \"ce759661-df26-4039-8996-0e4579d975e4\") " pod="openshift-console/console-5c4f87f695-hxhpn"
Jul 28 01:00:09.601435 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.601409    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ce759661-df26-4039-8996-0e4579d975e4-console-config\") pod \"console-5c4f87f695-hxhpn\" (UID: \"ce759661-df26-4039-8996-0e4579d975e4\") " pod="openshift-console/console-5c4f87f695-hxhpn"
Jul 28 01:00:09.601560 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.601542    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ce759661-df26-4039-8996-0e4579d975e4-oauth-serving-cert\") pod \"console-5c4f87f695-hxhpn\" (UID: \"ce759661-df26-4039-8996-0e4579d975e4\") " pod="openshift-console/console-5c4f87f695-hxhpn"
Jul 28 01:00:09.601701 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.601621    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ce759661-df26-4039-8996-0e4579d975e4-service-ca\") pod \"console-5c4f87f695-hxhpn\" (UID: \"ce759661-df26-4039-8996-0e4579d975e4\") " pod="openshift-console/console-5c4f87f695-hxhpn"
Jul 28 01:00:09.601883 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.601860    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce759661-df26-4039-8996-0e4579d975e4-trusted-ca-bundle\") pod \"console-5c4f87f695-hxhpn\" (UID: \"ce759661-df26-4039-8996-0e4579d975e4\") " pod="openshift-console/console-5c4f87f695-hxhpn"
Jul 28 01:00:09.603855 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.603832    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ce759661-df26-4039-8996-0e4579d975e4-console-serving-cert\") pod \"console-5c4f87f695-hxhpn\" (UID: \"ce759661-df26-4039-8996-0e4579d975e4\") " pod="openshift-console/console-5c4f87f695-hxhpn"
Jul 28 01:00:09.604559 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.604520    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ce759661-df26-4039-8996-0e4579d975e4-console-oauth-config\") pod \"console-5c4f87f695-hxhpn\" (UID: \"ce759661-df26-4039-8996-0e4579d975e4\") " pod="openshift-console/console-5c4f87f695-hxhpn"
Jul 28 01:00:09.608537 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.608515    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8h6r\" (UniqueName: \"kubernetes.io/projected/ce759661-df26-4039-8996-0e4579d975e4-kube-api-access-p8h6r\") pod \"console-5c4f87f695-hxhpn\" (UID: \"ce759661-df26-4039-8996-0e4579d975e4\") " pod="openshift-console/console-5c4f87f695-hxhpn"
Jul 28 01:00:09.742576 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:09.742494    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c4f87f695-hxhpn"
Jul 28 01:00:10.682960 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:10.682798    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c4f87f695-hxhpn"]
Jul 28 01:00:10.685504 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:00:10.685436    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce759661_df26_4039_8996_0e4579d975e4.slice/crio-ddd51340a5876ff7c51d471485d0d198558cf08c9c325a1972f1b177ba2d40c8 WatchSource:0}: Error finding container ddd51340a5876ff7c51d471485d0d198558cf08c9c325a1972f1b177ba2d40c8: Status 404 returned error can't find the container with id ddd51340a5876ff7c51d471485d0d198558cf08c9c325a1972f1b177ba2d40c8
Jul 28 01:00:11.182903 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:11.182860    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c4f87f695-hxhpn" event={"ID":"ce759661-df26-4039-8996-0e4579d975e4","Type":"ContainerStarted","Data":"31184298d9b8b90e48220a01df4610d1c16a0e8d7d235d45a6f6f8a8fdb8c51c"}
Jul 28 01:00:11.183083 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:11.182923    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c4f87f695-hxhpn" event={"ID":"ce759661-df26-4039-8996-0e4579d975e4","Type":"ContainerStarted","Data":"ddd51340a5876ff7c51d471485d0d198558cf08c9c325a1972f1b177ba2d40c8"}
Jul 28 01:00:11.208641 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:11.208578    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c4f87f695-hxhpn" podStartSLOduration=2.208554807 podStartE2EDuration="2.208554807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:00:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:00:11.205986746 +0000 UTC m=+381.370794870" watchObservedRunningTime="2026-07-28 01:00:11.208554807 +0000 UTC m=+381.373362915"
Jul 28 01:00:15.147653 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:15.147622    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bb8n5"
Jul 28 01:00:19.743335 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:19.743297    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c4f87f695-hxhpn"
Jul 28 01:00:19.743749 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:19.743373    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c4f87f695-hxhpn"
Jul 28 01:00:19.748426 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:19.748401    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c4f87f695-hxhpn"
Jul 28 01:00:20.238161 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:20.238132    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c4f87f695-hxhpn"
Jul 28 01:00:26.047407 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:26.047370    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g2bnz"]
Jul 28 01:00:26.099692 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:26.099651    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g2bnz"]
Jul 28 01:00:26.099864 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:26.099769    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g2bnz"
Jul 28 01:00:26.102760 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:26.102738    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-t94kv\""
Jul 28 01:00:26.167851 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:26.167815    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a0aa94b-8ea5-4ab6-a349-59d2688953bd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-g2bnz\" (UID: \"8a0aa94b-8ea5-4ab6-a349-59d2688953bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g2bnz"
Jul 28 01:00:26.168010 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:26.167873    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8spfw\" (UniqueName: \"kubernetes.io/projected/8a0aa94b-8ea5-4ab6-a349-59d2688953bd-kube-api-access-8spfw\") pod \"kuadrant-operator-controller-manager-86878c769f-g2bnz\" (UID: \"8a0aa94b-8ea5-4ab6-a349-59d2688953bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g2bnz"
Jul 28 01:00:26.269082 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:26.269048    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a0aa94b-8ea5-4ab6-a349-59d2688953bd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-g2bnz\" (UID: \"8a0aa94b-8ea5-4ab6-a349-59d2688953bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g2bnz"
Jul 28 01:00:26.269295 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:26.269107    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8spfw\" (UniqueName: \"kubernetes.io/projected/8a0aa94b-8ea5-4ab6-a349-59d2688953bd-kube-api-access-8spfw\") pod \"kuadrant-operator-controller-manager-86878c769f-g2bnz\" (UID: \"8a0aa94b-8ea5-4ab6-a349-59d2688953bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g2bnz"
Jul 28 01:00:26.269528 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:26.269498    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a0aa94b-8ea5-4ab6-a349-59d2688953bd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-g2bnz\" (UID: \"8a0aa94b-8ea5-4ab6-a349-59d2688953bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g2bnz"
Jul 28 01:00:26.278152 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:26.278125    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8spfw\" (UniqueName: \"kubernetes.io/projected/8a0aa94b-8ea5-4ab6-a349-59d2688953bd-kube-api-access-8spfw\") pod \"kuadrant-operator-controller-manager-86878c769f-g2bnz\" (UID: \"8a0aa94b-8ea5-4ab6-a349-59d2688953bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g2bnz"
Jul 28 01:00:26.412893 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:26.412866    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g2bnz"
Jul 28 01:00:27.539713 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:27.539684    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g2bnz"]
Jul 28 01:00:27.542368 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:00:27.542341    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a0aa94b_8ea5_4ab6_a349_59d2688953bd.slice/crio-fdf81f0aae79d413be91509643b9bcfe528e4ee67a9a0a5bac4b21071a7b5bc2 WatchSource:0}: Error finding container fdf81f0aae79d413be91509643b9bcfe528e4ee67a9a0a5bac4b21071a7b5bc2: Status 404 returned error can't find the container with id fdf81f0aae79d413be91509643b9bcfe528e4ee67a9a0a5bac4b21071a7b5bc2
Jul 28 01:00:28.274677 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:28.274638    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gmml8" event={"ID":"503b066d-ed59-42da-8aa1-e23fedc4157a","Type":"ContainerStarted","Data":"77dbf42a5b484b84e9640424292ee4127cdcbf21cf8ca613c12a37b88a15c94a"}
Jul 28 01:00:28.275917 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:28.275889    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g2bnz" event={"ID":"8a0aa94b-8ea5-4ab6-a349-59d2688953bd","Type":"ContainerStarted","Data":"fdf81f0aae79d413be91509643b9bcfe528e4ee67a9a0a5bac4b21071a7b5bc2"}
Jul 28 01:00:28.293782 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:28.293734    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gmml8" podStartSLOduration=1.73994899 podStartE2EDuration="25.293719289s" totalImagesPullingTime="23.553770299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:00:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:00:28.292152993 +0000 UTC m=+398.456961114" watchObservedRunningTime="2026-07-28 01:00:28.293719289 +0000 UTC m=+398.458527395"
Jul 28 01:00:33.306604 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:33.306520    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g2bnz" event={"ID":"8a0aa94b-8ea5-4ab6-a349-59d2688953bd","Type":"ContainerStarted","Data":"42058a0d73734d325a60316c290526c350b058b40bb7dd7d08d727a62559b1e6"}
Jul 28 01:00:33.306987 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:33.306670    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g2bnz"
Jul 28 01:00:33.325514 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:33.325465    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g2bnz" podStartSLOduration=1.9579905370000001 podStartE2EDuration="7.325450653s" totalImagesPullingTime="5.367460116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:00:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:00:33.323464475 +0000 UTC m=+403.488272583" watchObservedRunningTime="2026-07-28 01:00:33.325450653 +0000 UTC m=+403.490258826"
Jul 28 01:00:44.313685 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:00:44.313654    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g2bnz"
Jul 28 01:03:06.324951 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:06.324911    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj"]
Jul 28 01:03:06.327512 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:06.327488    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj"
Jul 28 01:03:06.331024 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:06.330993    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 28 01:03:06.331024 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:06.331008    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pfrfw\""
Jul 28 01:03:06.331024 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:06.330996    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 01:03:06.331257 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:06.331016    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 01:03:06.339011 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:06.338984    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj"]
Jul 28 01:03:06.453166 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:06.453131    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d3c1751-c420-4949-b199-9b86bf99183c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj\" (UID: \"7d3c1751-c420-4949-b199-9b86bf99183c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj"
Jul 28 01:03:06.453166 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:06.453166    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d3c1751-c420-4949-b199-9b86bf99183c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj\" (UID: \"7d3c1751-c420-4949-b199-9b86bf99183c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj"
Jul 28 01:03:06.453404 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:06.453272    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d3c1751-c420-4949-b199-9b86bf99183c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj\" (UID: \"7d3c1751-c420-4949-b199-9b86bf99183c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj"
Jul 28 01:03:06.453404 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:06.453327    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d3c1751-c420-4949-b199-9b86bf99183c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj\" (UID: \"7d3c1751-c420-4949-b199-9b86bf99183c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj"
Jul 28 01:03:06.453404 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:06.453362    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx7nv\" (UniqueName: \"kubernetes.io/projected/7d3c1751-c420-4949-b199-9b86bf99183c-kube-api-access-hx7nv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj\" (UID: \"7d3c1751-c420-4949-b199-9b86bf99183c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj"
Jul 28 01:03:06.453404 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:06.453393    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d3c1751-c420-4949-b199-9b86bf99183c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj\" (UID: \"7d3c1751-c420-4949-b199-9b86bf99183c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj"
Jul 28 01:03:06.554399 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:06.554361    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d3c1751-c420-4949-b199-9b86bf99183c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj\" (UID: \"7d3c1751-c420-4949-b199-9b86bf99183c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj"
Jul 28 01:03:06.554576 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:06.554527    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d3c1751-c420-4949-b199-9b86bf99183c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj\" (UID: \"7d3c1751-c420-4949-b199-9b86bf99183c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj"
Jul 28 01:03:06.554576 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:06.554564    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d3c1751-c420-4949-b199-9b86bf99183c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj\" (UID: \"7d3c1751-c420-4949-b199-9b86bf99183c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj"
Jul 28 01:03:06.554675 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:06.554626    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d3c1751-c420-4949-b199-9b86bf99183c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj\" (UID: \"7d3c1751-c420-4949-b199-9b86bf99183c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj"
Jul 28 01:03:06.554731 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:06.554692    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d3c1751-c420-4949-b199-9b86bf99183c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj\" (UID: \"7d3c1751-c420-4949-b199-9b86bf99183c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj"
Jul 28 01:03:06.554786 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:06.554736    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx7nv\" (UniqueName: \"kubernetes.io/projected/7d3c1751-c420-4949-b199-9b86bf99183c-kube-api-access-hx7nv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj\" (UID: \"7d3c1751-c420-4949-b199-9b86bf99183c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj"
Jul 28 01:03:06.554997 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:06.554969    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d3c1751-c420-4949-b199-9b86bf99183c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj\" (UID: \"7d3c1751-c420-4949-b199-9b86bf99183c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj"
Jul 28 01:03:06.555143 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:06.555047    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d3c1751-c420-4949-b199-9b86bf99183c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj\" (UID: \"7d3c1751-c420-4949-b199-9b86bf99183c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj"
Jul 28 01:03:06.555232 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:06.555144    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d3c1751-c420-4949-b199-9b86bf99183c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj\" (UID: \"7d3c1751-c420-4949-b199-9b86bf99183c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj"
Jul 28 01:03:06.556891 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:06.556874    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d3c1751-c420-4949-b199-9b86bf99183c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj\" (UID: \"7d3c1751-c420-4949-b199-9b86bf99183c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj"
Jul 28 01:03:06.557245 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:06.557206    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d3c1751-c420-4949-b199-9b86bf99183c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj\" (UID: \"7d3c1751-c420-4949-b199-9b86bf99183c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj"
Jul 28 01:03:06.566442 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:06.566421    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx7nv\" (UniqueName: \"kubernetes.io/projected/7d3c1751-c420-4949-b199-9b86bf99183c-kube-api-access-hx7nv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj\" (UID: \"7d3c1751-c420-4949-b199-9b86bf99183c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj"
Jul 28 01:03:06.639471 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:06.639415    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj"
Jul 28 01:03:06.777127 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:06.777093    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj"]
Jul 28 01:03:06.779023 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:03:06.778992    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d3c1751_c420_4949_b199_9b86bf99183c.slice/crio-8e095e5571b27af3f935f2d59a6759843e76e8c79b6a30a7970ac78ed9d1e3c6 WatchSource:0}: Error finding container 8e095e5571b27af3f935f2d59a6759843e76e8c79b6a30a7970ac78ed9d1e3c6: Status 404 returned error can't find the container with id 8e095e5571b27af3f935f2d59a6759843e76e8c79b6a30a7970ac78ed9d1e3c6
Jul 28 01:03:07.054170 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:07.053864    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj" event={"ID":"7d3c1751-c420-4949-b199-9b86bf99183c","Type":"ContainerStarted","Data":"8e095e5571b27af3f935f2d59a6759843e76e8c79b6a30a7970ac78ed9d1e3c6"}
Jul 28 01:03:08.060404 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:08.060367    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj" event={"ID":"7d3c1751-c420-4949-b199-9b86bf99183c","Type":"ContainerStarted","Data":"5242258078fac1847a87def8557484e42f14d3d3d29775ee08a453bef52f53f8"}
Jul 28 01:03:08.078414 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:08.078361    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj" podStartSLOduration=0.925562671 podStartE2EDuration="2.078344184s" totalImagesPullingTime="1.152781513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:03:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:03:08.076389566 +0000 UTC m=+558.241197688" watchObservedRunningTime="2026-07-28 01:03:08.078344184 +0000 UTC m=+558.243152290"
Jul 28 01:03:16.640085 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:16.640047    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj"
Jul 28 01:03:16.640581 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:16.640111    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj"
Jul 28 01:03:16.652531 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:16.652505    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj"
Jul 28 01:03:17.116198 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:17.116165    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj"
Jul 28 01:03:59.629598 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:59.629562    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx"]
Jul 28 01:03:59.632670 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:59.632653    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx"
Jul 28 01:03:59.635065 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:59.635038    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 28 01:03:59.640207 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:59.640183    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx"]
Jul 28 01:03:59.662118 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:59.662077    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b8e9d3d-446f-4241-b181-e78333d4f5a9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx\" (UID: \"9b8e9d3d-446f-4241-b181-e78333d4f5a9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx"
Jul 28 01:03:59.662321 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:59.662125    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b8e9d3d-446f-4241-b181-e78333d4f5a9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx\" (UID: \"9b8e9d3d-446f-4241-b181-e78333d4f5a9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx"
Jul 28 01:03:59.662321 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:59.662189    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b8e9d3d-446f-4241-b181-e78333d4f5a9-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx\" (UID: \"9b8e9d3d-446f-4241-b181-e78333d4f5a9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx"
Jul 28 01:03:59.662427 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:59.662331    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b8e9d3d-446f-4241-b181-e78333d4f5a9-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx\" (UID: \"9b8e9d3d-446f-4241-b181-e78333d4f5a9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx"
Jul 28 01:03:59.662427 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:59.662404    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b8e9d3d-446f-4241-b181-e78333d4f5a9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx\" (UID: \"9b8e9d3d-446f-4241-b181-e78333d4f5a9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx"
Jul 28 01:03:59.662491 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:59.662431    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwvzx\" (UniqueName: \"kubernetes.io/projected/9b8e9d3d-446f-4241-b181-e78333d4f5a9-kube-api-access-mwvzx\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx\" (UID: \"9b8e9d3d-446f-4241-b181-e78333d4f5a9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx"
Jul 28 01:03:59.763959 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:59.763909    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b8e9d3d-446f-4241-b181-e78333d4f5a9-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx\" (UID: \"9b8e9d3d-446f-4241-b181-e78333d4f5a9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx"
Jul 28 01:03:59.764158 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:59.764006    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b8e9d3d-446f-4241-b181-e78333d4f5a9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx\" (UID: \"9b8e9d3d-446f-4241-b181-e78333d4f5a9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx"
Jul 28 01:03:59.764158 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:59.764032    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwvzx\" (UniqueName: \"kubernetes.io/projected/9b8e9d3d-446f-4241-b181-e78333d4f5a9-kube-api-access-mwvzx\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx\" (UID: \"9b8e9d3d-446f-4241-b181-e78333d4f5a9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx"
Jul 28 01:03:59.764158 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:59.764068    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b8e9d3d-446f-4241-b181-e78333d4f5a9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx\" (UID: \"9b8e9d3d-446f-4241-b181-e78333d4f5a9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx"
Jul 28 01:03:59.764158 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:59.764105    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b8e9d3d-446f-4241-b181-e78333d4f5a9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx\" (UID: \"9b8e9d3d-446f-4241-b181-e78333d4f5a9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx"
Jul 28 01:03:59.764158 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:59.764153    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b8e9d3d-446f-4241-b181-e78333d4f5a9-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx\" (UID: \"9b8e9d3d-446f-4241-b181-e78333d4f5a9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx"
Jul 28 01:03:59.764522 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:59.764492    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b8e9d3d-446f-4241-b181-e78333d4f5a9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx\" (UID: \"9b8e9d3d-446f-4241-b181-e78333d4f5a9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx"
Jul 28 01:03:59.764638 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:59.764541    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b8e9d3d-446f-4241-b181-e78333d4f5a9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx\" (UID: \"9b8e9d3d-446f-4241-b181-e78333d4f5a9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx"
Jul 28 01:03:59.764638 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:59.764574    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b8e9d3d-446f-4241-b181-e78333d4f5a9-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx\" (UID: \"9b8e9d3d-446f-4241-b181-e78333d4f5a9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx"
Jul 28 01:03:59.766284 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:59.766261    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b8e9d3d-446f-4241-b181-e78333d4f5a9-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx\" (UID: \"9b8e9d3d-446f-4241-b181-e78333d4f5a9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx"
Jul 28 01:03:59.766749 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:59.766729    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b8e9d3d-446f-4241-b181-e78333d4f5a9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx\" (UID: \"9b8e9d3d-446f-4241-b181-e78333d4f5a9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx"
Jul 28 01:03:59.771270 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:59.771247    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwvzx\" (UniqueName: \"kubernetes.io/projected/9b8e9d3d-446f-4241-b181-e78333d4f5a9-kube-api-access-mwvzx\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx\" (UID: \"9b8e9d3d-446f-4241-b181-e78333d4f5a9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx"
Jul 28 01:03:59.944354 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:03:59.944256    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx"
Jul 28 01:04:00.079425 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:04:00.079392    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx"]
Jul 28 01:04:00.082692 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:04:00.082661    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b8e9d3d_446f_4241_b181_e78333d4f5a9.slice/crio-503bc2174d620a072baac1b224f508608ebcce69c9ea83909a4f5a0df1865625 WatchSource:0}: Error finding container 503bc2174d620a072baac1b224f508608ebcce69c9ea83909a4f5a0df1865625: Status 404 returned error can't find the container with id 503bc2174d620a072baac1b224f508608ebcce69c9ea83909a4f5a0df1865625
Jul 28 01:04:00.317969 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:04:00.317944    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx" event={"ID":"9b8e9d3d-446f-4241-b181-e78333d4f5a9","Type":"ContainerStarted","Data":"503bc2174d620a072baac1b224f508608ebcce69c9ea83909a4f5a0df1865625"}
Jul 28 01:04:00.335557 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:04:00.335499    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx" podStartSLOduration=1.169883021 podStartE2EDuration="1.335484296s" totalImagesPullingTime="165.601275ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:03:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:04:00.333689249 +0000 UTC m=+610.498497380" watchObservedRunningTime="2026-07-28 01:04:00.335484296 +0000 UTC m=+610.500292402"
Jul 28 01:04:01.323944 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:04:01.323910    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx" event={"ID":"9b8e9d3d-446f-4241-b181-e78333d4f5a9","Type":"ContainerStarted","Data":"2312039379d3ebc959ee84a4d27a47a80978a758ffe68b299f22fe758d996024"}
Jul 28 01:04:09.945377 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:04:09.945336    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx"
Jul 28 01:04:09.945859 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:04:09.945402    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx"
Jul 28 01:04:09.958189 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:04:09.958166    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx"
Jul 28 01:04:10.381467 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:04:10.381440    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx"
Jul 28 01:12:02.305323 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:12:02.305285    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g2bnz"]
Jul 28 01:12:02.305794 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:12:02.305499    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g2bnz" podUID="8a0aa94b-8ea5-4ab6-a349-59d2688953bd" containerName="manager" containerID="cri-o://42058a0d73734d325a60316c290526c350b058b40bb7dd7d08d727a62559b1e6" gracePeriod=10
Jul 28 01:12:05.453813 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:12:05.453783    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g2bnz"
Jul 28 01:12:05.564092 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:12:05.564062    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a0aa94b-8ea5-4ab6-a349-59d2688953bd-extensions-socket-volume\") pod \"8a0aa94b-8ea5-4ab6-a349-59d2688953bd\" (UID: \"8a0aa94b-8ea5-4ab6-a349-59d2688953bd\") "
Jul 28 01:12:05.564301 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:12:05.564121    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8spfw\" (UniqueName: \"kubernetes.io/projected/8a0aa94b-8ea5-4ab6-a349-59d2688953bd-kube-api-access-8spfw\") pod \"8a0aa94b-8ea5-4ab6-a349-59d2688953bd\" (UID: \"8a0aa94b-8ea5-4ab6-a349-59d2688953bd\") "
Jul 28 01:12:05.564528 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:12:05.564504    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a0aa94b-8ea5-4ab6-a349-59d2688953bd-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8a0aa94b-8ea5-4ab6-a349-59d2688953bd" (UID: "8a0aa94b-8ea5-4ab6-a349-59d2688953bd"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:12:05.566282 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:12:05.566259    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a0aa94b-8ea5-4ab6-a349-59d2688953bd-kube-api-access-8spfw" (OuterVolumeSpecName: "kube-api-access-8spfw") pod "8a0aa94b-8ea5-4ab6-a349-59d2688953bd" (UID: "8a0aa94b-8ea5-4ab6-a349-59d2688953bd"). InnerVolumeSpecName "kube-api-access-8spfw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:12:05.665276 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:12:05.665208    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8spfw\" (UniqueName: \"kubernetes.io/projected/8a0aa94b-8ea5-4ab6-a349-59d2688953bd-kube-api-access-8spfw\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:12:05.665276 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:12:05.665269    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a0aa94b-8ea5-4ab6-a349-59d2688953bd-extensions-socket-volume\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:12:05.685481 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:12:05.685446    2578 generic.go:356] "Generic (PLEG): container finished" podID="8a0aa94b-8ea5-4ab6-a349-59d2688953bd" containerID="42058a0d73734d325a60316c290526c350b058b40bb7dd7d08d727a62559b1e6" exitCode=0
Jul 28 01:12:05.685606 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:12:05.685497    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g2bnz"
Jul 28 01:12:05.685606 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:12:05.685503    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g2bnz" event={"ID":"8a0aa94b-8ea5-4ab6-a349-59d2688953bd","Type":"ContainerDied","Data":"42058a0d73734d325a60316c290526c350b058b40bb7dd7d08d727a62559b1e6"}
Jul 28 01:12:05.685606 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:12:05.685556    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g2bnz" event={"ID":"8a0aa94b-8ea5-4ab6-a349-59d2688953bd","Type":"ContainerDied","Data":"fdf81f0aae79d413be91509643b9bcfe528e4ee67a9a0a5bac4b21071a7b5bc2"}
Jul 28 01:12:05.685606 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:12:05.685585    2578 scope.go:117] "RemoveContainer" containerID="42058a0d73734d325a60316c290526c350b058b40bb7dd7d08d727a62559b1e6"
Jul 28 01:12:05.695271 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:12:05.695252    2578 scope.go:117] "RemoveContainer" containerID="42058a0d73734d325a60316c290526c350b058b40bb7dd7d08d727a62559b1e6"
Jul 28 01:12:05.695520 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:12:05.695501    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42058a0d73734d325a60316c290526c350b058b40bb7dd7d08d727a62559b1e6\": container with ID starting with 42058a0d73734d325a60316c290526c350b058b40bb7dd7d08d727a62559b1e6 not found: ID does not exist" containerID="42058a0d73734d325a60316c290526c350b058b40bb7dd7d08d727a62559b1e6"
Jul 28 01:12:05.695583 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:12:05.695533    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42058a0d73734d325a60316c290526c350b058b40bb7dd7d08d727a62559b1e6"} err="failed to get container status \"42058a0d73734d325a60316c290526c350b058b40bb7dd7d08d727a62559b1e6\": rpc error: code = NotFound desc = could not find container \"42058a0d73734d325a60316c290526c350b058b40bb7dd7d08d727a62559b1e6\": container with ID starting with 42058a0d73734d325a60316c290526c350b058b40bb7dd7d08d727a62559b1e6 not found: ID does not exist"
Jul 28 01:12:05.708316 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:12:05.708291    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g2bnz"]
Jul 28 01:12:05.711456 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:12:05.711432    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g2bnz"]
Jul 28 01:12:06.402677 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:12:06.402647    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a0aa94b-8ea5-4ab6-a349-59d2688953bd" path="/var/lib/kubelet/pods/8a0aa94b-8ea5-4ab6-a349-59d2688953bd/volumes"
Jul 28 01:12:32.200704 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:12:32.200655    2578 controller.go:195] "Failed to update lease" err="Put \"https://afec2574f58554198b2bdecc84cc978c-75182989f48b2d15.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-132-238.ec2.internal?timeout=10s\": context deadline exceeded"
Jul 28 01:12:42.200864 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:12:42.200821    2578 controller.go:195] "Failed to update lease" err="Put \"https://afec2574f58554198b2bdecc84cc978c-75182989f48b2d15.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-132-238.ec2.internal?timeout=10s\": context deadline exceeded"
Jul 28 01:12:42.969262 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:12:42.969195    2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-28T01:12:32Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-28T01:12:32Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-28T01:12:32Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-28T01:12:32Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-132-238.ec2.internal\": Patch \"https://afec2574f58554198b2bdecc84cc978c-75182989f48b2d15.elb.us-east-1.amazonaws.com:6443/api/v1/nodes/ip-10-0-132-238.ec2.internal/status?timeout=10s\": context deadline exceeded"
Jul 28 01:12:52.201717 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:12:52.201670    2578 controller.go:195] "Failed to update lease" err="Put \"https://afec2574f58554198b2bdecc84cc978c-75182989f48b2d15.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-132-238.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Jul 28 01:12:52.970428 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:12:52.970389    2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-238.ec2.internal\": Get \"https://afec2574f58554198b2bdecc84cc978c-75182989f48b2d15.elb.us-east-1.amazonaws.com:6443/api/v1/nodes/ip-10-0-132-238.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Jul 28 01:13:02.202665 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:13:02.202620    2578 controller.go:195] "Failed to update lease" err="Put \"https://afec2574f58554198b2bdecc84cc978c-75182989f48b2d15.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-132-238.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Jul 28 01:13:02.971303 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:13:02.971267    2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-238.ec2.internal\": Get \"https://afec2574f58554198b2bdecc84cc978c-75182989f48b2d15.elb.us-east-1.amazonaws.com:6443/api/v1/nodes/ip-10-0-132-238.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Jul 28 01:13:06.143228 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143171    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.143228 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143184    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.143725 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143237    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-84b42\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.143725 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143252    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.143725 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143257    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-trkfd\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.143725 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143288    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.143725 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143293    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.143725 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143307    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.143725 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143353    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.143725 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143371    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.143725 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143423    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.143725 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143445    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.143725 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143489    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-s4cm9\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.143725 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143505    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-9z9cw\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.143725 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143519    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.143725 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143521    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8gjuvrvjnal77\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.143725 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143555    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.143725 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143568    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.143725 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143589    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.143725 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143619    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wktlp\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.144407 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143623    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.144407 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143627    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-zbj2x\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.144407 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143648    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.144407 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143664    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.144407 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143683    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.144407 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143689    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"llm\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.144407 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143718    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.144407 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143735    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-tgr4v\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.144407 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143756    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.144407 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143780    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-2tq2g\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.144407 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143798    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.144407 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143816    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.144407 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143846    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-6zkgf\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.144407 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143865    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"default-dockercfg-q5z4q\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.144407 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143876    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.144407 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143874    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"default-dockercfg-n77xz\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.144407 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143892    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gjx9x\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.144407 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143902    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145229 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143905    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-t9w4q\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145229 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143916    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145229 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143918    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145229 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143936    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145229 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143970    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"console-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145229 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143969    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145229 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143981    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145229 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.143987    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145229 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144003    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-z9jfm\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145229 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144004    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145229 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144027    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145229 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144045    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145229 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144045    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145229 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144050    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145229 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144058    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145229 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144070    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145229 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144089    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145229 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:13:06.144128    2578 controller.go:195] "Failed to update lease" err="Put \"https://afec2574f58554198b2bdecc84cc978c-75182989f48b2d15.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-132-238.ec2.internal?timeout=10s\": http2: client connection lost"
Jul 28 01:13:06.145229 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144149    2578 controller.go:115] "failed to update lease using latest lease, fallback to ensure lease" err="failed 5 attempts to update lease"
Jul 28 01:13:06.145796 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144234    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145796 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144236    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-bjjqb\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145796 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144244    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145796 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144247    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145796 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144263    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cm2gp\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145796 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144264    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-bsvd5\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145796 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144271    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145796 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144288    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145796 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144294    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145796 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144299    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145796 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144298    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145796 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144290    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145796 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144317    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145796 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144326    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145796 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144344    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"console-serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145796 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144344    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-75k5f\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145796 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144352    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"dns-dockercfg-w6lql\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.145796 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144343    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146360 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144345    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146360 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144356    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-5d2z2\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146360 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144372    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146360 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144380    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"service-ca\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146360 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144353    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9zh8t\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146360 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144389    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146360 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144405    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146360 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144410    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146360 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144405    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146360 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144406    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146360 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144426    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146360 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144425    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146360 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144412    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146360 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144429    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146360 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144455    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146360 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144459    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146360 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144454    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146360 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144465    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146910 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144471    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-txw5m\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146910 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144484    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146910 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144490    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-24fn0i4iqquvl\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146910 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144493    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146910 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144499    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146910 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144501    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146910 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144514    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146910 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144528    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146910 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144533    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146910 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144537    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146910 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144530    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146910 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144544    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"metrics-server-ddo0llh01t7f6\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146910 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144565    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146910 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144567    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146910 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144591    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146910 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144594    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"console-dockercfg-krj69\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146910 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144603    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-49nz8\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.146910 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144608    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147482 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144616    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147482 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144631    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-w8g2g\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147482 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144639    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147482 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144646    2578 reflector.go:571] "Warning: watch ended with error" reflector="pkg/kubelet/config/apiserver.go:66" type="*v1.Pod" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147482 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144642    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147482 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144663    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147482 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144663    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147482 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144678    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147482 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144687    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"console-oauth-config\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147482 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144697    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147482 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144702    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147482 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144714    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-pmw29\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147482 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144718    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147482 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144721    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147482 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144729    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147482 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144734    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cknlx\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147482 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144702    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147482 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144745    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-m9szw\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147981 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144752    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147981 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144768    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-spd2s\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147981 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144772    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147981 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144781    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147981 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144792    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"llm\"/\"default-dockercfg-pfrfw\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147981 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144770    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147981 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144805    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147981 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144808    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147981 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144817    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147981 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144821    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147981 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144821    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147981 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144827    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147981 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144836    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"original-pull-secret\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147981 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144849    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-7vwnk\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147981 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144856    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-h7rjf\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147981 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144869    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147981 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144875    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.147981 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144770    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.148496 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144886    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.148496 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144892    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.148496 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144896    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-r566t\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.148496 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144748    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.148496 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144912    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.148496 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144928    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"konnectivity-agent\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.148496 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144935    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.148496 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144858    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.148496 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144947    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.148496 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144876    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.148496 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144952    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.148496 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144956    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.148496 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144900    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"dns-default\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.148496 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144899    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.148496 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144928    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.148496 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144958    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.148496 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.144990    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.148496 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.145018    2578 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vq2f5\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 28 01:13:06.959583 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.959552    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wktlp\""
Jul 28 01:13:06.959763 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.959661    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 28 01:13:06.965800 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.965780    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 01:13:06.983062 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.983036    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 28 01:13:06.983169 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.983036    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 01:13:06.988185 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.988167    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-7vwnk\""
Jul 28 01:13:06.994481 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:06.994462    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 28 01:13:07.002531 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.002512    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-pmw29\""
Jul 28 01:13:07.013949 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.013922    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 01:13:07.013949 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.013933    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 28 01:13:07.018821 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.018803    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 28 01:13:07.021702 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.021687    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 01:13:07.039997 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.039974    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 01:13:07.042390 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.042374    2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 01:13:07.045159 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.045125    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 01:13:07.052484 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.052466    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 01:13:07.060763 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.060740    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 01:13:07.063910 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.063875    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 28 01:13:07.064049 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.063943    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-w6lql\""
Jul 28 01:13:07.064422 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.064390    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 01:13:07.065561 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.065545    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-tgr4v\""
Jul 28 01:13:07.068020 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.067999    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 28 01:13:07.074930 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.074907    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 28 01:13:07.075252 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.075236    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 01:13:07.080259 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.080241    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 28 01:13:07.080517 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.080501    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-t9w4q\""
Jul 28 01:13:07.093383 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.093354    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 01:13:07.106656 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.106634    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 28 01:13:07.107818 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.107802    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 28 01:13:07.108025 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.108007    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 01:13:07.113907 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.113885    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 28 01:13:07.120051 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.120032    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 01:13:07.120850 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.120836    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cm2gp\""
Jul 28 01:13:07.128035 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.128019    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gjx9x\""
Jul 28 01:13:07.142368 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.142338    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 28 01:13:07.142882 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.142867    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 01:13:07.159037 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.159007    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 01:13:07.169820 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.169799    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-z9jfm\""
Jul 28 01:13:07.172781 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.172766    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 01:13:07.173710 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.173693    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-r566t\""
Jul 28 01:13:07.178629 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.178614    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 01:13:07.181590 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.181573    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-spd2s\""
Jul 28 01:13:07.189331 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.189313    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 01:13:07.192933 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.192917    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-m9szw\""
Jul 28 01:13:07.194890 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.194874    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 01:13:07.195813 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.195798    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 01:13:07.197002 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.196987    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 28 01:13:07.200918 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.200899    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 01:13:07.214044 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.213988    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 28 01:13:07.217056 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.217036    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 01:13:07.221012 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.220995    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 28 01:13:07.224135 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.224120    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-zbj2x\""
Jul 28 01:13:07.226198 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.226184    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-krj69\""
Jul 28 01:13:07.229086 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.229070    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 28 01:13:07.229318 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.229302    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 28 01:13:07.242654 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.242635    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 28 01:13:07.246554 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.246534    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 28 01:13:07.250566 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.250550    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cknlx\""
Jul 28 01:13:07.250625 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.250611    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-5d2z2\""
Jul 28 01:13:07.256935 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.256917    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 01:13:07.257328 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.257313    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-bsvd5\""
Jul 28 01:13:07.257912 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.257899    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 01:13:07.261493 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.261479    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-49nz8\""
Jul 28 01:13:07.263577 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.263562    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 01:13:07.267101 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.267086    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pfrfw\""
Jul 28 01:13:07.272269 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.272251    2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 01:13:07.280164 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.280146    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 01:13:07.287080 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.287063    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 28 01:13:07.288114 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.288100    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 01:13:07.292853 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.292838    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 01:13:07.305052 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.305029    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 28 01:13:07.308828 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.308809    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 01:13:07.309833 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.309816    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8gjuvrvjnal77\""
Jul 28 01:13:07.310774 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.310758    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 28 01:13:07.316619 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.316604    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 01:13:07.319459 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.319445    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 28 01:13:07.321413 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.321399    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-txw5m\""
Jul 28 01:13:07.324614 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.324600    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-n77xz\""
Jul 28 01:13:07.329706 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.329690    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 01:13:07.333311 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.333280    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-6zkgf\""
Jul 28 01:13:07.343312 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.343290    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9zh8t\""
Jul 28 01:13:07.369420 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.369396    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 28 01:13:07.372415 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.372394    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 28 01:13:07.372671 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.372659    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 01:13:07.375641 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.375622    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 28 01:13:07.378557 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.378543    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 01:13:07.393785 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.393763    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 01:13:07.403997 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.403979    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 28 01:13:07.408981 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.408965    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 28 01:13:07.409205 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.409190    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 28 01:13:07.415447 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.415428    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 01:13:07.418280 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.418264    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 01:13:07.424061 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.424035    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 01:13:07.424150 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.424132    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-trkfd\""
Jul 28 01:13:07.428251 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.428236    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 28 01:13:07.428947 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.428933    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-s4cm9\""
Jul 28 01:13:07.432861 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.432844    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 28 01:13:07.441595 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.441572    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-h7rjf\""
Jul 28 01:13:07.448429 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.448414    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 28 01:13:07.453143 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.453129    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 28 01:13:07.468791 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.468739    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-w8g2g\""
Jul 28 01:13:07.469034 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.469018    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 28 01:13:07.473342 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.473317    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-84b42\""
Jul 28 01:13:07.479106 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.479082    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 28 01:13:07.480057 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.480039    2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 01:13:07.482715 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.482696    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhx46"]
Jul 28 01:13:07.483097 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.483086    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a0aa94b-8ea5-4ab6-a349-59d2688953bd" containerName="manager"
Jul 28 01:13:07.483139 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.483099    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a0aa94b-8ea5-4ab6-a349-59d2688953bd" containerName="manager"
Jul 28 01:13:07.483168 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.483164    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a0aa94b-8ea5-4ab6-a349-59d2688953bd" containerName="manager"
Jul 28 01:13:07.487374 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.487359    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhx46"
Jul 28 01:13:07.487917 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.487902    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 28 01:13:07.489673 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.489654    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-t94kv\""
Jul 28 01:13:07.492427 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.492410    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ddo0llh01t7f6\""
Jul 28 01:13:07.492877 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.492860    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 01:13:07.495961 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.495940    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 28 01:13:07.496801 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.496783    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 28 01:13:07.497914 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.497900    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 28 01:13:07.506753 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.506735    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 28 01:13:07.519123 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.519103    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 01:13:07.519123 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.519114    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 01:13:07.520118 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.520097    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 28 01:13:07.532515 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.532495    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 01:13:07.535538 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.535520    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 01:13:07.547581 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.547560    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-75k5f\""
Jul 28 01:13:07.551328 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.551312    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 28 01:13:07.562732 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.562713    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 01:13:07.562855 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.562767    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 28 01:13:07.562938 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.562923    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 28 01:13:07.573252 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.573232    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 01:13:07.575152 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.575136    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 01:13:07.581281 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.581261    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 28 01:13:07.582011 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.581997    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 01:13:07.590946 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.590924    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 28 01:13:07.592574 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.592559    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 01:13:07.593229 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.593198    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 28 01:13:07.613268 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.613246    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-24fn0i4iqquvl\""
Jul 28 01:13:07.633722 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.633696    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 01:13:07.653422 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.653398    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 28 01:13:07.657785 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.657765    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7jj6\" (UniqueName: \"kubernetes.io/projected/ed3db210-4391-45ce-a306-6f9a4a948496-kube-api-access-w7jj6\") pod \"kuadrant-operator-controller-manager-86878c769f-qhx46\" (UID: \"ed3db210-4391-45ce-a306-6f9a4a948496\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhx46"
Jul 28 01:13:07.657843 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.657821    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed3db210-4391-45ce-a306-6f9a4a948496-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qhx46\" (UID: \"ed3db210-4391-45ce-a306-6f9a4a948496\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhx46"
Jul 28 01:13:07.673385 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.673364    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 28 01:13:07.693710 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.693684    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 28 01:13:07.713925 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.713897    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 01:13:07.732874 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.732808    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 28 01:13:07.753390 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.753365    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 28 01:13:07.758692 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.758668    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7jj6\" (UniqueName: \"kubernetes.io/projected/ed3db210-4391-45ce-a306-6f9a4a948496-kube-api-access-w7jj6\") pod \"kuadrant-operator-controller-manager-86878c769f-qhx46\" (UID: \"ed3db210-4391-45ce-a306-6f9a4a948496\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhx46"
Jul 28 01:13:07.758777 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.758726    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed3db210-4391-45ce-a306-6f9a4a948496-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qhx46\" (UID: \"ed3db210-4391-45ce-a306-6f9a4a948496\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhx46"
Jul 28 01:13:07.759128 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.759108    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed3db210-4391-45ce-a306-6f9a4a948496-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qhx46\" (UID: \"ed3db210-4391-45ce-a306-6f9a4a948496\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhx46"
Jul 28 01:13:07.773121 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.773094    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 01:13:07.793718 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.793696    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-q5z4q\""
Jul 28 01:13:07.813358 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.813334    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vq2f5\""
Jul 28 01:13:07.833031 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.833007    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 28 01:13:07.853231 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.853191    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 01:13:07.873529 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.873505    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 01:13:07.893753 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.893730    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 28 01:13:07.913829 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.913806    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 01:13:07.933206 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.933183    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 28 01:13:07.953761 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.953738    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 28 01:13:07.972970 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.972947    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-9z9cw\""
Jul 28 01:13:07.993720 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:07.993656    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 01:13:08.013754 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:08.013727    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 28 01:13:08.033337 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:08.033313    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 01:13:08.053422 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:08.053397    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 28 01:13:08.073686 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:08.073661    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 01:13:08.094514 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:08.094490    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 01:13:08.113842 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:08.113815    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 28 01:13:08.133529 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:08.133507    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 01:13:08.154124 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:08.154102    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 01:13:08.173651 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:08.173631    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-bjjqb\""
Jul 28 01:13:08.193175 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:08.193152    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 01:13:08.213589 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:08.213562    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-2tq2g\""
Jul 28 01:13:08.233042 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:08.233020    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 01:13:08.253860 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:08.253803    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 01:13:08.273202 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:08.273180    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 01:13:08.298805 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:08.298781    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7jj6\" (UniqueName: \"kubernetes.io/projected/ed3db210-4391-45ce-a306-6f9a4a948496-kube-api-access-w7jj6\") pod \"kuadrant-operator-controller-manager-86878c769f-qhx46\" (UID: \"ed3db210-4391-45ce-a306-6f9a4a948496\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhx46"
Jul 28 01:13:08.398263 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:08.398231    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhx46"
Jul 28 01:13:08.533726 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:08.533701    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhx46"]
Jul 28 01:13:08.534798 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:13:08.534773    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded3db210_4391_45ce_a306_6f9a4a948496.slice/crio-6676b4a35ef335a6ab87347a4b78f376bb0fc454d8f5ec78378a44a7020bf5be WatchSource:0}: Error finding container 6676b4a35ef335a6ab87347a4b78f376bb0fc454d8f5ec78378a44a7020bf5be: Status 404 returned error can't find the container with id 6676b4a35ef335a6ab87347a4b78f376bb0fc454d8f5ec78378a44a7020bf5be
Jul 28 01:13:09.005277 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:09.005242    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhx46" event={"ID":"ed3db210-4391-45ce-a306-6f9a4a948496","Type":"ContainerStarted","Data":"10303b07a75384432c99c243a3b207fe275c0c34cda99ab81a7efd5824f14e38"}
Jul 28 01:13:09.005444 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:09.005296    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhx46" event={"ID":"ed3db210-4391-45ce-a306-6f9a4a948496","Type":"ContainerStarted","Data":"6676b4a35ef335a6ab87347a4b78f376bb0fc454d8f5ec78378a44a7020bf5be"}
Jul 28 01:13:09.005444 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:09.005335    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhx46"
Jul 28 01:13:09.025906 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:09.025627    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhx46" podStartSLOduration=6.025594054 podStartE2EDuration="6.025594054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:13:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:13:09.023200866 +0000 UTC m=+1159.188008973" watchObservedRunningTime="2026-07-28 01:13:09.025594054 +0000 UTC m=+1159.190402165"
Jul 28 01:13:20.012654 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:13:20.012578    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhx46"
Jul 28 01:15:00.144319 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:15:00.144277    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753355-qqhg7"]
Jul 28 01:15:00.147679 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:15:00.147661    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753355-qqhg7"
Jul 28 01:15:00.150358 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:15:00.150336    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8ph8w\""
Jul 28 01:15:00.157036 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:15:00.157018    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753355-qqhg7"
Jul 28 01:15:00.167166 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:15:00.167136    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753355-qqhg7"]
Jul 28 01:15:00.302378 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:15:00.302343    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753355-qqhg7"]
Jul 28 01:15:00.303820 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:15:00.303796    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39490d82_139f_4115_82fa_9f51f159433f.slice/crio-3177977b42015eed9411f95e2eb7850d6f3913f173b6ce8f22e55286c9e3484d WatchSource:0}: Error finding container 3177977b42015eed9411f95e2eb7850d6f3913f173b6ce8f22e55286c9e3484d: Status 404 returned error can't find the container with id 3177977b42015eed9411f95e2eb7850d6f3913f173b6ce8f22e55286c9e3484d
Jul 28 01:15:00.305683 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:15:00.305668    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 01:15:00.556631 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:15:00.556539    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753355-qqhg7" event={"ID":"39490d82-139f-4115-82fa-9f51f159433f","Type":"ContainerStarted","Data":"3177977b42015eed9411f95e2eb7850d6f3913f173b6ce8f22e55286c9e3484d"}
Jul 28 01:15:03.576357 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:15:03.576320    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753355-qqhg7" event={"ID":"39490d82-139f-4115-82fa-9f51f159433f","Type":"ContainerStarted","Data":"a5dca1bff003adf4e4941e8f8b5433073d9756c6caf91784e0fc3749f03b1568"}
Jul 28 01:15:04.584887 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:15:04.584850    2578 generic.go:356] "Generic (PLEG): container finished" podID="39490d82-139f-4115-82fa-9f51f159433f" containerID="a5dca1bff003adf4e4941e8f8b5433073d9756c6caf91784e0fc3749f03b1568" exitCode=0
Jul 28 01:15:04.585319 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:15:04.584903    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753355-qqhg7" event={"ID":"39490d82-139f-4115-82fa-9f51f159433f","Type":"ContainerDied","Data":"a5dca1bff003adf4e4941e8f8b5433073d9756c6caf91784e0fc3749f03b1568"}
Jul 28 01:15:04.731244 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:15:04.731198    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753355-qqhg7"
Jul 28 01:15:05.598287 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:15:05.598252    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753355-qqhg7" event={"ID":"39490d82-139f-4115-82fa-9f51f159433f","Type":"ContainerDied","Data":"3177977b42015eed9411f95e2eb7850d6f3913f173b6ce8f22e55286c9e3484d"}
Jul 28 01:15:05.598287 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:15:05.598272    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753355-qqhg7"
Jul 28 01:15:05.598790 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:15:05.598327    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3177977b42015eed9411f95e2eb7850d6f3913f173b6ce8f22e55286c9e3484d"
Jul 28 01:16:17.244338 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.244261    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"]
Jul 28 01:16:17.244787 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.244662    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39490d82-139f-4115-82fa-9f51f159433f" containerName="cleanup"
Jul 28 01:16:17.244787 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.244673    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="39490d82-139f-4115-82fa-9f51f159433f" containerName="cleanup"
Jul 28 01:16:17.244787 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.244733    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="39490d82-139f-4115-82fa-9f51f159433f" containerName="cleanup"
Jul 28 01:16:17.247934 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.247916    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"
Jul 28 01:16:17.250581 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.250557    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-502777a0-kserve-self-signed-certs\""
Jul 28 01:16:17.257345 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.257323    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"]
Jul 28 01:16:17.339188 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.339139    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjczz\" (UniqueName: \"kubernetes.io/projected/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-kube-api-access-vjczz\") pod \"e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt\" (UID: \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\") " pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"
Jul 28 01:16:17.339423 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.339197    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-tls-certs\") pod \"e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt\" (UID: \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\") " pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"
Jul 28 01:16:17.339423 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.339259    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-dshm\") pod \"e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt\" (UID: \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\") " pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"
Jul 28 01:16:17.339423 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.339373    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-model-cache\") pod \"e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt\" (UID: \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\") " pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"
Jul 28 01:16:17.339423 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.339421    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-tmp-dir\") pod \"e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt\" (UID: \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\") " pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"
Jul 28 01:16:17.339646 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.339521    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-home\") pod \"e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt\" (UID: \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\") " pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"
Jul 28 01:16:17.440870 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.440830    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-tls-certs\") pod \"e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt\" (UID: \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\") " pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"
Jul 28 01:16:17.440870 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.440869    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-dshm\") pod \"e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt\" (UID: \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\") " pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"
Jul 28 01:16:17.441122 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.440891    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-model-cache\") pod \"e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt\" (UID: \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\") " pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"
Jul 28 01:16:17.441122 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.440911    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-tmp-dir\") pod \"e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt\" (UID: \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\") " pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"
Jul 28 01:16:17.441122 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.440954    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-home\") pod \"e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt\" (UID: \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\") " pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"
Jul 28 01:16:17.441122 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.441017    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjczz\" (UniqueName: \"kubernetes.io/projected/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-kube-api-access-vjczz\") pod \"e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt\" (UID: \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\") " pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"
Jul 28 01:16:17.441361 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.441324    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-model-cache\") pod \"e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt\" (UID: \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\") " pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"
Jul 28 01:16:17.441447 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.441399    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-tmp-dir\") pod \"e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt\" (UID: \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\") " pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"
Jul 28 01:16:17.441447 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.441433    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-home\") pod \"e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt\" (UID: \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\") " pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"
Jul 28 01:16:17.443363 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.443339    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-dshm\") pod \"e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt\" (UID: \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\") " pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"
Jul 28 01:16:17.443583 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.443565    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-tls-certs\") pod \"e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt\" (UID: \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\") " pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"
Jul 28 01:16:17.451136 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.451111    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjczz\" (UniqueName: \"kubernetes.io/projected/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-kube-api-access-vjczz\") pod \"e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt\" (UID: \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\") " pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"
Jul 28 01:16:17.559653 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.559550    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"
Jul 28 01:16:17.716207 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.716177    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"]
Jul 28 01:16:17.718058 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:16:17.718029    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd7bbe5c4_b05f_4a4c_8c73_c119b37cbc36.slice/crio-80a6b5ea9335ae3e9c2f1c9831b0c1ea3c111eb33d334b3ad46c8a263c2cbe60 WatchSource:0}: Error finding container 80a6b5ea9335ae3e9c2f1c9831b0c1ea3c111eb33d334b3ad46c8a263c2cbe60: Status 404 returned error can't find the container with id 80a6b5ea9335ae3e9c2f1c9831b0c1ea3c111eb33d334b3ad46c8a263c2cbe60
Jul 28 01:16:17.944753 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.944719    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt" event={"ID":"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36","Type":"ContainerStarted","Data":"4bfd5af8e599eb2de3c36587a707ec39bb2ad1c29d76163c2e4cb37d33986020"}
Jul 28 01:16:17.944753 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.944761    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt" event={"ID":"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36","Type":"ContainerStarted","Data":"80a6b5ea9335ae3e9c2f1c9831b0c1ea3c111eb33d334b3ad46c8a263c2cbe60"}
Jul 28 01:16:17.963398 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:17.963348    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt" podStartSLOduration=0.96333065 podStartE2EDuration="963.33065ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:16:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:16:17.960976452 +0000 UTC m=+1348.125784558" watchObservedRunningTime="2026-07-28 01:16:17.96333065 +0000 UTC m=+1348.128138757"
Jul 28 01:16:27.560546 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:27.560431    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"
Jul 28 01:16:27.560546 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:27.560521    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"
Jul 28 01:16:27.573438 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:27.573408    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"
Jul 28 01:16:28.020258 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:16:28.020200    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"
Jul 28 01:17:00.996649 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:00.996618    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"]
Jul 28 01:17:16.016525 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:16.016486    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt" podUID="d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36" containerName="main" containerID="cri-o://4bfd5af8e599eb2de3c36587a707ec39bb2ad1c29d76163c2e4cb37d33986020" gracePeriod=45
Jul 28 01:17:16.243042 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:16.243009    2578 generic.go:356] "Generic (PLEG): container finished" podID="d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36" containerID="4bfd5af8e599eb2de3c36587a707ec39bb2ad1c29d76163c2e4cb37d33986020" exitCode=0
Jul 28 01:17:16.243254 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:16.243083    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt" event={"ID":"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36","Type":"ContainerDied","Data":"4bfd5af8e599eb2de3c36587a707ec39bb2ad1c29d76163c2e4cb37d33986020"}
Jul 28 01:17:16.276422 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:16.276349    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"
Jul 28 01:17:16.311869 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:16.311841    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjczz\" (UniqueName: \"kubernetes.io/projected/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-kube-api-access-vjczz\") pod \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\" (UID: \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\") "
Jul 28 01:17:16.312043 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:16.311885    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-dshm\") pod \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\" (UID: \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\") "
Jul 28 01:17:16.312043 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:16.311908    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-model-cache\") pod \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\" (UID: \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\") "
Jul 28 01:17:16.312043 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:16.311963    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-tls-certs\") pod \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\" (UID: \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\") "
Jul 28 01:17:16.312043 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:16.312005    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-home\") pod \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\" (UID: \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\") "
Jul 28 01:17:16.312043 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:16.312041    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-tmp-dir\") pod \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\" (UID: \"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36\") "
Jul 28 01:17:16.312325 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:16.312302    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-model-cache" (OuterVolumeSpecName: "model-cache") pod "d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36" (UID: "d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:17:16.312371 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:16.312354    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-home" (OuterVolumeSpecName: "home") pod "d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36" (UID: "d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:17:16.312534 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:16.312504    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36" (UID: "d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:17:16.314178 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:16.314148    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-dshm" (OuterVolumeSpecName: "dshm") pod "d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36" (UID: "d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:17:16.314299 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:16.314235    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36" (UID: "d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:17:16.314499 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:16.314482    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-kube-api-access-vjczz" (OuterVolumeSpecName: "kube-api-access-vjczz") pod "d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36" (UID: "d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36"). InnerVolumeSpecName "kube-api-access-vjczz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:17:16.413424 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:16.413389    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-dshm\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:17:16.413424 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:16.413423    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-model-cache\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:17:16.413424 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:16.413435    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-tls-certs\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:17:16.413623 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:16.413442    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-home\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:17:16.413623 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:16.413450    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-tmp-dir\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:17:16.413623 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:16.413458    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjczz\" (UniqueName: \"kubernetes.io/projected/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36-kube-api-access-vjczz\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:17:17.249801 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:17.249770    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"
Jul 28 01:17:17.250293 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:17.249773    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt" event={"ID":"d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36","Type":"ContainerDied","Data":"80a6b5ea9335ae3e9c2f1c9831b0c1ea3c111eb33d334b3ad46c8a263c2cbe60"}
Jul 28 01:17:17.250293 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:17.249893    2578 scope.go:117] "RemoveContainer" containerID="4bfd5af8e599eb2de3c36587a707ec39bb2ad1c29d76163c2e4cb37d33986020"
Jul 28 01:17:17.267206 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:17.267179    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"]
Jul 28 01:17:17.272706 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:17.272680    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-502777a0-kserve-55cb96c5dc-xnblt"]
Jul 28 01:17:18.402744 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:17:18.402712    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36" path="/var/lib/kubelet/pods/d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36/volumes"
Jul 28 01:20:57.570534 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:57.570459    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx"]
Jul 28 01:20:57.570975 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:57.570887    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36" containerName="main"
Jul 28 01:20:57.570975 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:57.570900    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36" containerName="main"
Jul 28 01:20:57.571056 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:57.570980    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7bbe5c4-b05f-4a4c-8c73-c119b37cbc36" containerName="main"
Jul 28 01:20:57.574294 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:57.574274    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx"
Jul 28 01:20:57.577013 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:57.576992    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lrljm\""
Jul 28 01:20:57.577013 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:57.577003    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 01:20:57.577942 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:57.577921    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 01:20:57.578081 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:57.577939    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-0dbe3ebb-serving-cert\""
Jul 28 01:20:57.586139 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:57.586118    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx"]
Jul 28 01:20:57.729461 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:57.729429    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5vvj\" (UniqueName: \"kubernetes.io/projected/7c07e842-ad0b-4709-9c0a-ab72befafa33-kube-api-access-q5vvj\") pod \"maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx\" (UID: \"7c07e842-ad0b-4709-9c0a-ab72befafa33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx"
Jul 28 01:20:57.729461 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:57.729466    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c07e842-ad0b-4709-9c0a-ab72befafa33-openshift-service-ca\") pod \"maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx\" (UID: \"7c07e842-ad0b-4709-9c0a-ab72befafa33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx"
Jul 28 01:20:57.729665 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:57.729604    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c07e842-ad0b-4709-9c0a-ab72befafa33-maas-api-tls\") pod \"maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx\" (UID: \"7c07e842-ad0b-4709-9c0a-ab72befafa33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx"
Jul 28 01:20:57.830447 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:57.830351    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c07e842-ad0b-4709-9c0a-ab72befafa33-maas-api-tls\") pod \"maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx\" (UID: \"7c07e842-ad0b-4709-9c0a-ab72befafa33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx"
Jul 28 01:20:57.830447 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:57.830410    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5vvj\" (UniqueName: \"kubernetes.io/projected/7c07e842-ad0b-4709-9c0a-ab72befafa33-kube-api-access-q5vvj\") pod \"maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx\" (UID: \"7c07e842-ad0b-4709-9c0a-ab72befafa33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx"
Jul 28 01:20:57.830447 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:57.830439    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c07e842-ad0b-4709-9c0a-ab72befafa33-openshift-service-ca\") pod \"maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx\" (UID: \"7c07e842-ad0b-4709-9c0a-ab72befafa33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx"
Jul 28 01:20:57.831201 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:57.831181    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c07e842-ad0b-4709-9c0a-ab72befafa33-openshift-service-ca\") pod \"maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx\" (UID: \"7c07e842-ad0b-4709-9c0a-ab72befafa33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx"
Jul 28 01:20:57.832728 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:57.832708    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c07e842-ad0b-4709-9c0a-ab72befafa33-maas-api-tls\") pod \"maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx\" (UID: \"7c07e842-ad0b-4709-9c0a-ab72befafa33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx"
Jul 28 01:20:57.839575 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:57.839543    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5vvj\" (UniqueName: \"kubernetes.io/projected/7c07e842-ad0b-4709-9c0a-ab72befafa33-kube-api-access-q5vvj\") pod \"maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx\" (UID: \"7c07e842-ad0b-4709-9c0a-ab72befafa33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx"
Jul 28 01:20:57.885559 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:57.885526    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx"
Jul 28 01:20:58.040863 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:58.040820    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx"]
Jul 28 01:20:58.052798 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:20:58.052770    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c07e842_ad0b_4709_9c0a_ab72befafa33.slice/crio-c1d05ca51a270f9ee4831dc2eb89f14faa5a1d21a4f516ab49a97f9d7d25c56e WatchSource:0}: Error finding container c1d05ca51a270f9ee4831dc2eb89f14faa5a1d21a4f516ab49a97f9d7d25c56e: Status 404 returned error can't find the container with id c1d05ca51a270f9ee4831dc2eb89f14faa5a1d21a4f516ab49a97f9d7d25c56e
Jul 28 01:20:58.054135 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:58.054118    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 01:20:58.359404 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:58.359374    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx" event={"ID":"7c07e842-ad0b-4709-9c0a-ab72befafa33","Type":"ContainerStarted","Data":"c1d05ca51a270f9ee4831dc2eb89f14faa5a1d21a4f516ab49a97f9d7d25c56e"}
Jul 28 01:20:58.446203 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:58.446170    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j"]
Jul 28 01:20:58.451291 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:58.451273    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j"
Jul 28 01:20:58.456849 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:58.456830    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-0dbe3ebb\""
Jul 28 01:20:58.466913 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:58.466887    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j"]
Jul 28 01:20:58.471609 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:58.471585    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw"]
Jul 28 01:20:58.475378 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:58.475357    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw"
Jul 28 01:20:58.478441 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:58.478423    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-0dbe3ebb-dockercfg-m9flk\""
Jul 28 01:20:58.490177 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:58.490153    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw"]
Jul 28 01:20:58.537777 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:58.537743    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb1914fe-6176-45e9-b9d1-e886271ac3fb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j\" (UID: \"cb1914fe-6176-45e9-b9d1-e886271ac3fb\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j"
Jul 28 01:20:58.537938 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:58.537793    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s59q\" (UniqueName: \"kubernetes.io/projected/cb1914fe-6176-45e9-b9d1-e886271ac3fb-kube-api-access-2s59q\") pod \"payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j\" (UID: \"cb1914fe-6176-45e9-b9d1-e886271ac3fb\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j"
Jul 28 01:20:58.638836 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:58.638751    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb1914fe-6176-45e9-b9d1-e886271ac3fb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j\" (UID: \"cb1914fe-6176-45e9-b9d1-e886271ac3fb\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j"
Jul 28 01:20:58.638836 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:58.638798    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e15223c3-401a-4621-9dd9-cf5fddfaa996-plugins-config-volume\") pod \"payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw\" (UID: \"e15223c3-401a-4621-9dd9-cf5fddfaa996\") " pod="openshift-ingress/payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw"
Jul 28 01:20:58.638836 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:58.638828    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwc6r\" (UniqueName: \"kubernetes.io/projected/e15223c3-401a-4621-9dd9-cf5fddfaa996-kube-api-access-fwc6r\") pod \"payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw\" (UID: \"e15223c3-401a-4621-9dd9-cf5fddfaa996\") " pod="openshift-ingress/payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw"
Jul 28 01:20:58.639335 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:58.638848    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s59q\" (UniqueName: \"kubernetes.io/projected/cb1914fe-6176-45e9-b9d1-e886271ac3fb-kube-api-access-2s59q\") pod \"payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j\" (UID: \"cb1914fe-6176-45e9-b9d1-e886271ac3fb\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j"
Jul 28 01:20:58.639647 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:58.639623    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb1914fe-6176-45e9-b9d1-e886271ac3fb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j\" (UID: \"cb1914fe-6176-45e9-b9d1-e886271ac3fb\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j"
Jul 28 01:20:58.649973 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:58.649948    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s59q\" (UniqueName: \"kubernetes.io/projected/cb1914fe-6176-45e9-b9d1-e886271ac3fb-kube-api-access-2s59q\") pod \"payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j\" (UID: \"cb1914fe-6176-45e9-b9d1-e886271ac3fb\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j"
Jul 28 01:20:58.740069 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:58.740026    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e15223c3-401a-4621-9dd9-cf5fddfaa996-plugins-config-volume\") pod \"payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw\" (UID: \"e15223c3-401a-4621-9dd9-cf5fddfaa996\") " pod="openshift-ingress/payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw"
Jul 28 01:20:58.740270 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:58.740079    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwc6r\" (UniqueName: \"kubernetes.io/projected/e15223c3-401a-4621-9dd9-cf5fddfaa996-kube-api-access-fwc6r\") pod \"payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw\" (UID: \"e15223c3-401a-4621-9dd9-cf5fddfaa996\") " pod="openshift-ingress/payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw"
Jul 28 01:20:58.740944 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:58.740921    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e15223c3-401a-4621-9dd9-cf5fddfaa996-plugins-config-volume\") pod \"payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw\" (UID: \"e15223c3-401a-4621-9dd9-cf5fddfaa996\") " pod="openshift-ingress/payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw"
Jul 28 01:20:58.749808 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:58.749780    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwc6r\" (UniqueName: \"kubernetes.io/projected/e15223c3-401a-4621-9dd9-cf5fddfaa996-kube-api-access-fwc6r\") pod \"payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw\" (UID: \"e15223c3-401a-4621-9dd9-cf5fddfaa996\") " pod="openshift-ingress/payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw"
Jul 28 01:20:58.763066 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:58.763028    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j"
Jul 28 01:20:58.786621 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:58.786588    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw"
Jul 28 01:20:58.969522 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:58.968616    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j"]
Jul 28 01:20:59.007666 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:59.007629    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw"]
Jul 28 01:20:59.008427 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:20:59.008317    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode15223c3_401a_4621_9dd9_cf5fddfaa996.slice/crio-3f3076d680ec96ec24162d6a99c691d23f36a79e82c3bfb015bcf0a4bf6945d2 WatchSource:0}: Error finding container 3f3076d680ec96ec24162d6a99c691d23f36a79e82c3bfb015bcf0a4bf6945d2: Status 404 returned error can't find the container with id 3f3076d680ec96ec24162d6a99c691d23f36a79e82c3bfb015bcf0a4bf6945d2
Jul 28 01:20:59.368366 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:59.368284    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw" event={"ID":"e15223c3-401a-4621-9dd9-cf5fddfaa996","Type":"ContainerStarted","Data":"3f3076d680ec96ec24162d6a99c691d23f36a79e82c3bfb015bcf0a4bf6945d2"}
Jul 28 01:20:59.370169 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:20:59.370003    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j" event={"ID":"cb1914fe-6176-45e9-b9d1-e886271ac3fb","Type":"ContainerStarted","Data":"5ea6c38b3f502410ef7405554beda79f1976fcc77be859e26e749dda9cd31a05"}
Jul 28 01:21:02.390974 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:21:02.390935    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw" event={"ID":"e15223c3-401a-4621-9dd9-cf5fddfaa996","Type":"ContainerStarted","Data":"21ef2392470a57fa0b46057135757fec0bf9fb12ebfe6d2cfd39724c84bf43d0"}
Jul 28 01:21:02.391474 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:21:02.391106    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw"
Jul 28 01:21:02.392573 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:21:02.392555    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx" event={"ID":"7c07e842-ad0b-4709-9c0a-ab72befafa33","Type":"ContainerStarted","Data":"32d4281118ec40443f17ce489c3be9723680e61b7bec06d7c9b2a7cfa1731220"}
Jul 28 01:21:02.392636 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:21:02.392604    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx"
Jul 28 01:21:02.394185 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:21:02.394159    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j" event={"ID":"cb1914fe-6176-45e9-b9d1-e886271ac3fb","Type":"ContainerStarted","Data":"37a90c0d1e79648187bb9b41b177f34a0b52451f99ddc077100fde3527a20462"}
Jul 28 01:21:02.394288 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:21:02.394222    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j"
Jul 28 01:21:02.411205 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:21:02.411160    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw" podStartSLOduration=2.089655159 podStartE2EDuration="4.411138257s" totalImagesPullingTime="2.321483098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:21:02.408691347 +0000 UTC m=+1632.573499447" watchObservedRunningTime="2026-07-28 01:21:02.411138257 +0000 UTC m=+1632.575946364"
Jul 28 01:21:02.448067 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:21:02.448008    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j" podStartSLOduration=2.097244055 podStartE2EDuration="4.447992522s" totalImagesPullingTime="2.350748467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:21:02.428005337 +0000 UTC m=+1632.592813456" watchObservedRunningTime="2026-07-28 01:21:02.447992522 +0000 UTC m=+1632.612800628"
Jul 28 01:21:02.448560 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:21:02.448526    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx" podStartSLOduration=2.176022955 podStartE2EDuration="5.448514795s" totalImagesPullingTime="3.27249184s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:20:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:21:02.446780577 +0000 UTC m=+1632.611588702" watchObservedRunningTime="2026-07-28 01:21:02.448514795 +0000 UTC m=+1632.613322902"
Jul 28 01:21:08.405543 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:21:08.405511    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx"
Jul 28 01:21:13.401821 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:21:13.401775    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw"
Jul 28 01:21:13.402275 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:21:13.401865    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j"
Jul 28 01:26:04.529838 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:04.529799    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0dbe3ebb-qgzwr"]
Jul 28 01:26:04.533400 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:04.533383    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0dbe3ebb-qgzwr"
Jul 28 01:26:04.536240 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:04.536195    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8ph8w\""
Jul 28 01:26:04.555427 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:04.555396    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0dbe3ebb-qgzwr"]
Jul 28 01:26:04.651689 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:04.651650    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26ac1248-ce1f-47eb-be35-23015356a348-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0dbe3ebb-qgzwr\" (UID: \"26ac1248-ce1f-47eb-be35-23015356a348\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0dbe3ebb-qgzwr"
Jul 28 01:26:04.753033 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:04.752990    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26ac1248-ce1f-47eb-be35-23015356a348-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0dbe3ebb-qgzwr\" (UID: \"26ac1248-ce1f-47eb-be35-23015356a348\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0dbe3ebb-qgzwr"
Jul 28 01:26:04.753680 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:04.753660    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26ac1248-ce1f-47eb-be35-23015356a348-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0dbe3ebb-qgzwr\" (UID: \"26ac1248-ce1f-47eb-be35-23015356a348\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0dbe3ebb-qgzwr"
Jul 28 01:26:04.843957 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:04.843867    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0dbe3ebb-qgzwr"
Jul 28 01:26:04.980111 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:04.980080    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0dbe3ebb-qgzwr"]
Jul 28 01:26:04.981462 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:26:04.981431    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26ac1248_ce1f_47eb_be35_23015356a348.slice/crio-1ad0fdd890624516e310e0da24ec8f8d3fa2b5408b6948c69996c386d7b2b143 WatchSource:0}: Error finding container 1ad0fdd890624516e310e0da24ec8f8d3fa2b5408b6948c69996c386d7b2b143: Status 404 returned error can't find the container with id 1ad0fdd890624516e310e0da24ec8f8d3fa2b5408b6948c69996c386d7b2b143
Jul 28 01:26:05.949759 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:05.949725    2578 generic.go:356] "Generic (PLEG): container finished" podID="26ac1248-ce1f-47eb-be35-23015356a348" containerID="bf55e35fb6d33b96ef631e464b65ee1c9e976861d3c91a75a0198e4d1d214ed9" exitCode=0
Jul 28 01:26:05.950133 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:05.949806    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0dbe3ebb-qgzwr" event={"ID":"26ac1248-ce1f-47eb-be35-23015356a348","Type":"ContainerDied","Data":"bf55e35fb6d33b96ef631e464b65ee1c9e976861d3c91a75a0198e4d1d214ed9"}
Jul 28 01:26:05.950133 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:05.949858    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0dbe3ebb-qgzwr" event={"ID":"26ac1248-ce1f-47eb-be35-23015356a348","Type":"ContainerStarted","Data":"1ad0fdd890624516e310e0da24ec8f8d3fa2b5408b6948c69996c386d7b2b143"}
Jul 28 01:26:07.094509 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:07.094482    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0dbe3ebb-qgzwr"
Jul 28 01:26:07.177836 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:07.177789    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26ac1248-ce1f-47eb-be35-23015356a348-maas-api-service-ca\") pod \"26ac1248-ce1f-47eb-be35-23015356a348\" (UID: \"26ac1248-ce1f-47eb-be35-23015356a348\") "
Jul 28 01:26:07.178150 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:07.178126    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26ac1248-ce1f-47eb-be35-23015356a348-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "26ac1248-ce1f-47eb-be35-23015356a348" (UID: "26ac1248-ce1f-47eb-be35-23015356a348"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:26:07.279104 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:07.279012    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26ac1248-ce1f-47eb-be35-23015356a348-maas-api-service-ca\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:26:07.961526 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:07.961492    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0dbe3ebb-qgzwr" event={"ID":"26ac1248-ce1f-47eb-be35-23015356a348","Type":"ContainerDied","Data":"1ad0fdd890624516e310e0da24ec8f8d3fa2b5408b6948c69996c386d7b2b143"}
Jul 28 01:26:07.961697 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:07.961542    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1ad0fdd890624516e310e0da24ec8f8d3fa2b5408b6948c69996c386d7b2b143"
Jul 28 01:26:07.961697 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:07.961508    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0dbe3ebb-qgzwr"
Jul 28 01:26:15.624069 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:15.624031    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx"]
Jul 28 01:26:15.624508 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:15.624290    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx" podUID="7c07e842-ad0b-4709-9c0a-ab72befafa33" containerName="maas-api" containerID="cri-o://32d4281118ec40443f17ce489c3be9723680e61b7bec06d7c9b2a7cfa1731220" gracePeriod=30
Jul 28 01:26:15.677461 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:15.677421    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw"]
Jul 28 01:26:15.677668 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:15.677641    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw" podUID="e15223c3-401a-4621-9dd9-cf5fddfaa996" containerName="payload-processing" containerID="cri-o://21ef2392470a57fa0b46057135757fec0bf9fb12ebfe6d2cfd39724c84bf43d0" gracePeriod=30
Jul 28 01:26:15.685702 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:15.685660    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j"]
Jul 28 01:26:15.685910 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:15.685885    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j" podUID="cb1914fe-6176-45e9-b9d1-e886271ac3fb" containerName="payload-pre-processing" containerID="cri-o://37a90c0d1e79648187bb9b41b177f34a0b52451f99ddc077100fde3527a20462" gracePeriod=30
Jul 28 01:26:15.941785 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:15.941764    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx"
Jul 28 01:26:15.966812 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:15.966776    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw"
Jul 28 01:26:16.004730 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.004700    2578 generic.go:356] "Generic (PLEG): container finished" podID="e15223c3-401a-4621-9dd9-cf5fddfaa996" containerID="21ef2392470a57fa0b46057135757fec0bf9fb12ebfe6d2cfd39724c84bf43d0" exitCode=0
Jul 28 01:26:16.004892 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.004752    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw"
Jul 28 01:26:16.004892 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.004784    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw" event={"ID":"e15223c3-401a-4621-9dd9-cf5fddfaa996","Type":"ContainerDied","Data":"21ef2392470a57fa0b46057135757fec0bf9fb12ebfe6d2cfd39724c84bf43d0"}
Jul 28 01:26:16.004892 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.004838    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw" event={"ID":"e15223c3-401a-4621-9dd9-cf5fddfaa996","Type":"ContainerDied","Data":"3f3076d680ec96ec24162d6a99c691d23f36a79e82c3bfb015bcf0a4bf6945d2"}
Jul 28 01:26:16.004892 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.004870    2578 scope.go:117] "RemoveContainer" containerID="21ef2392470a57fa0b46057135757fec0bf9fb12ebfe6d2cfd39724c84bf43d0"
Jul 28 01:26:16.006277 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.006251    2578 generic.go:356] "Generic (PLEG): container finished" podID="7c07e842-ad0b-4709-9c0a-ab72befafa33" containerID="32d4281118ec40443f17ce489c3be9723680e61b7bec06d7c9b2a7cfa1731220" exitCode=0
Jul 28 01:26:16.006401 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.006380    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx" event={"ID":"7c07e842-ad0b-4709-9c0a-ab72befafa33","Type":"ContainerDied","Data":"32d4281118ec40443f17ce489c3be9723680e61b7bec06d7c9b2a7cfa1731220"}
Jul 28 01:26:16.006463 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.006429    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx" event={"ID":"7c07e842-ad0b-4709-9c0a-ab72befafa33","Type":"ContainerDied","Data":"c1d05ca51a270f9ee4831dc2eb89f14faa5a1d21a4f516ab49a97f9d7d25c56e"}
Jul 28 01:26:16.006463 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.006391    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx"
Jul 28 01:26:16.007910 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.007886    2578 generic.go:356] "Generic (PLEG): container finished" podID="cb1914fe-6176-45e9-b9d1-e886271ac3fb" containerID="37a90c0d1e79648187bb9b41b177f34a0b52451f99ddc077100fde3527a20462" exitCode=0
Jul 28 01:26:16.008016 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.007950    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j" event={"ID":"cb1914fe-6176-45e9-b9d1-e886271ac3fb","Type":"ContainerDied","Data":"37a90c0d1e79648187bb9b41b177f34a0b52451f99ddc077100fde3527a20462"}
Jul 28 01:26:16.010228 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.010189    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j"
Jul 28 01:26:16.014043 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.014025    2578 scope.go:117] "RemoveContainer" containerID="21ef2392470a57fa0b46057135757fec0bf9fb12ebfe6d2cfd39724c84bf43d0"
Jul 28 01:26:16.014342 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:26:16.014313    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21ef2392470a57fa0b46057135757fec0bf9fb12ebfe6d2cfd39724c84bf43d0\": container with ID starting with 21ef2392470a57fa0b46057135757fec0bf9fb12ebfe6d2cfd39724c84bf43d0 not found: ID does not exist" containerID="21ef2392470a57fa0b46057135757fec0bf9fb12ebfe6d2cfd39724c84bf43d0"
Jul 28 01:26:16.014423 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.014349    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21ef2392470a57fa0b46057135757fec0bf9fb12ebfe6d2cfd39724c84bf43d0"} err="failed to get container status \"21ef2392470a57fa0b46057135757fec0bf9fb12ebfe6d2cfd39724c84bf43d0\": rpc error: code = NotFound desc = could not find container \"21ef2392470a57fa0b46057135757fec0bf9fb12ebfe6d2cfd39724c84bf43d0\": container with ID starting with 21ef2392470a57fa0b46057135757fec0bf9fb12ebfe6d2cfd39724c84bf43d0 not found: ID does not exist"
Jul 28 01:26:16.014423 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.014366    2578 scope.go:117] "RemoveContainer" containerID="32d4281118ec40443f17ce489c3be9723680e61b7bec06d7c9b2a7cfa1731220"
Jul 28 01:26:16.022038 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.022024    2578 scope.go:117] "RemoveContainer" containerID="32d4281118ec40443f17ce489c3be9723680e61b7bec06d7c9b2a7cfa1731220"
Jul 28 01:26:16.022313 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:26:16.022296    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32d4281118ec40443f17ce489c3be9723680e61b7bec06d7c9b2a7cfa1731220\": container with ID starting with 32d4281118ec40443f17ce489c3be9723680e61b7bec06d7c9b2a7cfa1731220 not found: ID does not exist" containerID="32d4281118ec40443f17ce489c3be9723680e61b7bec06d7c9b2a7cfa1731220"
Jul 28 01:26:16.022395 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.022319    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32d4281118ec40443f17ce489c3be9723680e61b7bec06d7c9b2a7cfa1731220"} err="failed to get container status \"32d4281118ec40443f17ce489c3be9723680e61b7bec06d7c9b2a7cfa1731220\": rpc error: code = NotFound desc = could not find container \"32d4281118ec40443f17ce489c3be9723680e61b7bec06d7c9b2a7cfa1731220\": container with ID starting with 32d4281118ec40443f17ce489c3be9723680e61b7bec06d7c9b2a7cfa1731220 not found: ID does not exist"
Jul 28 01:26:16.063518 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.063492    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e15223c3-401a-4621-9dd9-cf5fddfaa996-plugins-config-volume\") pod \"e15223c3-401a-4621-9dd9-cf5fddfaa996\" (UID: \"e15223c3-401a-4621-9dd9-cf5fddfaa996\") "
Jul 28 01:26:16.063646 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.063558    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwc6r\" (UniqueName: \"kubernetes.io/projected/e15223c3-401a-4621-9dd9-cf5fddfaa996-kube-api-access-fwc6r\") pod \"e15223c3-401a-4621-9dd9-cf5fddfaa996\" (UID: \"e15223c3-401a-4621-9dd9-cf5fddfaa996\") "
Jul 28 01:26:16.063646 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.063584    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c07e842-ad0b-4709-9c0a-ab72befafa33-openshift-service-ca\") pod \"7c07e842-ad0b-4709-9c0a-ab72befafa33\" (UID: \"7c07e842-ad0b-4709-9c0a-ab72befafa33\") "
Jul 28 01:26:16.063646 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.063608    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c07e842-ad0b-4709-9c0a-ab72befafa33-maas-api-tls\") pod \"7c07e842-ad0b-4709-9c0a-ab72befafa33\" (UID: \"7c07e842-ad0b-4709-9c0a-ab72befafa33\") "
Jul 28 01:26:16.063778 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.063714    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5vvj\" (UniqueName: \"kubernetes.io/projected/7c07e842-ad0b-4709-9c0a-ab72befafa33-kube-api-access-q5vvj\") pod \"7c07e842-ad0b-4709-9c0a-ab72befafa33\" (UID: \"7c07e842-ad0b-4709-9c0a-ab72befafa33\") "
Jul 28 01:26:16.063938 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.063910    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e15223c3-401a-4621-9dd9-cf5fddfaa996-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e15223c3-401a-4621-9dd9-cf5fddfaa996" (UID: "e15223c3-401a-4621-9dd9-cf5fddfaa996"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:26:16.063938 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.063929    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c07e842-ad0b-4709-9c0a-ab72befafa33-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7c07e842-ad0b-4709-9c0a-ab72befafa33" (UID: "7c07e842-ad0b-4709-9c0a-ab72befafa33"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:26:16.064314 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.064046    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c07e842-ad0b-4709-9c0a-ab72befafa33-openshift-service-ca\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:26:16.064314 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.064059    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e15223c3-401a-4621-9dd9-cf5fddfaa996-plugins-config-volume\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:26:16.065809 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.065779    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c07e842-ad0b-4709-9c0a-ab72befafa33-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7c07e842-ad0b-4709-9c0a-ab72befafa33" (UID: "7c07e842-ad0b-4709-9c0a-ab72befafa33"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:26:16.065899 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.065806    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e15223c3-401a-4621-9dd9-cf5fddfaa996-kube-api-access-fwc6r" (OuterVolumeSpecName: "kube-api-access-fwc6r") pod "e15223c3-401a-4621-9dd9-cf5fddfaa996" (UID: "e15223c3-401a-4621-9dd9-cf5fddfaa996"). InnerVolumeSpecName "kube-api-access-fwc6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:26:16.065899 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.065835    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c07e842-ad0b-4709-9c0a-ab72befafa33-kube-api-access-q5vvj" (OuterVolumeSpecName: "kube-api-access-q5vvj") pod "7c07e842-ad0b-4709-9c0a-ab72befafa33" (UID: "7c07e842-ad0b-4709-9c0a-ab72befafa33"). InnerVolumeSpecName "kube-api-access-q5vvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:26:16.164488 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.164411    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb1914fe-6176-45e9-b9d1-e886271ac3fb-plugins-config-volume\") pod \"cb1914fe-6176-45e9-b9d1-e886271ac3fb\" (UID: \"cb1914fe-6176-45e9-b9d1-e886271ac3fb\") "
Jul 28 01:26:16.164632 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.164556    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2s59q\" (UniqueName: \"kubernetes.io/projected/cb1914fe-6176-45e9-b9d1-e886271ac3fb-kube-api-access-2s59q\") pod \"cb1914fe-6176-45e9-b9d1-e886271ac3fb\" (UID: \"cb1914fe-6176-45e9-b9d1-e886271ac3fb\") "
Jul 28 01:26:16.164741 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.164729    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5vvj\" (UniqueName: \"kubernetes.io/projected/7c07e842-ad0b-4709-9c0a-ab72befafa33-kube-api-access-q5vvj\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:26:16.164777 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.164744    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwc6r\" (UniqueName: \"kubernetes.io/projected/e15223c3-401a-4621-9dd9-cf5fddfaa996-kube-api-access-fwc6r\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:26:16.164777 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.164755    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c07e842-ad0b-4709-9c0a-ab72befafa33-maas-api-tls\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:26:16.164777 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.164744    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb1914fe-6176-45e9-b9d1-e886271ac3fb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cb1914fe-6176-45e9-b9d1-e886271ac3fb" (UID: "cb1914fe-6176-45e9-b9d1-e886271ac3fb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:26:16.166572 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.166537    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb1914fe-6176-45e9-b9d1-e886271ac3fb-kube-api-access-2s59q" (OuterVolumeSpecName: "kube-api-access-2s59q") pod "cb1914fe-6176-45e9-b9d1-e886271ac3fb" (UID: "cb1914fe-6176-45e9-b9d1-e886271ac3fb"). InnerVolumeSpecName "kube-api-access-2s59q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:26:16.265308 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.265270    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2s59q\" (UniqueName: \"kubernetes.io/projected/cb1914fe-6176-45e9-b9d1-e886271ac3fb-kube-api-access-2s59q\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:26:16.265308 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.265301    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb1914fe-6176-45e9-b9d1-e886271ac3fb-plugins-config-volume\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:26:16.330159 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.330122    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw"]
Jul 28 01:26:16.335670 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.335643    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0dbe3ebb-69f846bb86-sjsqw"]
Jul 28 01:26:16.347602 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.347573    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx"]
Jul 28 01:26:16.351744 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.351724    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0dbe3ebb-5f4578fdb8-dt7gx"]
Jul 28 01:26:16.402896 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.402871    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c07e842-ad0b-4709-9c0a-ab72befafa33" path="/var/lib/kubelet/pods/7c07e842-ad0b-4709-9c0a-ab72befafa33/volumes"
Jul 28 01:26:16.403255 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:16.403240    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e15223c3-401a-4621-9dd9-cf5fddfaa996" path="/var/lib/kubelet/pods/e15223c3-401a-4621-9dd9-cf5fddfaa996/volumes"
Jul 28 01:26:17.015834 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:17.015751    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j" event={"ID":"cb1914fe-6176-45e9-b9d1-e886271ac3fb","Type":"ContainerDied","Data":"5ea6c38b3f502410ef7405554beda79f1976fcc77be859e26e749dda9cd31a05"}
Jul 28 01:26:17.015834 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:17.015808    2578 scope.go:117] "RemoveContainer" containerID="37a90c0d1e79648187bb9b41b177f34a0b52451f99ddc077100fde3527a20462"
Jul 28 01:26:17.016310 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:17.015759    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j"
Jul 28 01:26:17.042540 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:17.042513    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j"]
Jul 28 01:26:17.049673 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:17.049646    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0dbe3ebb-857dd5c7c6-4975j"]
Jul 28 01:26:18.402036 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:18.402003    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb1914fe-6176-45e9-b9d1-e886271ac3fb" path="/var/lib/kubelet/pods/cb1914fe-6176-45e9-b9d1-e886271ac3fb/volumes"
Jul 28 01:26:19.610328 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:19.610299    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0dbe3ebb-qgzwr"]
Jul 28 01:26:19.612671 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:19.612649    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0dbe3ebb-qgzwr"]
Jul 28 01:26:20.402693 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:20.402666    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26ac1248-ce1f-47eb-be35-23015356a348" path="/var/lib/kubelet/pods/26ac1248-ce1f-47eb-be35-23015356a348/volumes"
Jul 28 01:26:23.570672 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.570638    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n"]
Jul 28 01:26:23.571110 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.571058    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c07e842-ad0b-4709-9c0a-ab72befafa33" containerName="maas-api"
Jul 28 01:26:23.571110 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.571071    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c07e842-ad0b-4709-9c0a-ab72befafa33" containerName="maas-api"
Jul 28 01:26:23.571110 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.571082    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb1914fe-6176-45e9-b9d1-e886271ac3fb" containerName="payload-pre-processing"
Jul 28 01:26:23.571110 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.571088    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb1914fe-6176-45e9-b9d1-e886271ac3fb" containerName="payload-pre-processing"
Jul 28 01:26:23.571110 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.571103    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e15223c3-401a-4621-9dd9-cf5fddfaa996" containerName="payload-processing"
Jul 28 01:26:23.571110 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.571108    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e15223c3-401a-4621-9dd9-cf5fddfaa996" containerName="payload-processing"
Jul 28 01:26:23.571444 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.571125    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26ac1248-ce1f-47eb-be35-23015356a348" containerName="revoke-keys"
Jul 28 01:26:23.571444 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.571131    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="26ac1248-ce1f-47eb-be35-23015356a348" containerName="revoke-keys"
Jul 28 01:26:23.571444 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.571206    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e15223c3-401a-4621-9dd9-cf5fddfaa996" containerName="payload-processing"
Jul 28 01:26:23.571444 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.571254    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c07e842-ad0b-4709-9c0a-ab72befafa33" containerName="maas-api"
Jul 28 01:26:23.571444 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.571260    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="26ac1248-ce1f-47eb-be35-23015356a348" containerName="revoke-keys"
Jul 28 01:26:23.571444 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.571267    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb1914fe-6176-45e9-b9d1-e886271ac3fb" containerName="payload-pre-processing"
Jul 28 01:26:23.582473 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.582443    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n"
Jul 28 01:26:23.585395 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.585372    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 01:26:23.585839 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.585811    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lrljm\""
Jul 28 01:26:23.586445 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.586424    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 01:26:23.586715 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.586426    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-c08eabd1-serving-cert\""
Jul 28 01:26:23.587247 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.587206    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n"]
Jul 28 01:26:23.633717 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.633682    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3264f23c-f1cf-4397-8555-96910475df0e-openshift-service-ca\") pod \"maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n\" (UID: \"3264f23c-f1cf-4397-8555-96910475df0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n"
Jul 28 01:26:23.633876 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.633726    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3264f23c-f1cf-4397-8555-96910475df0e-maas-api-tls\") pod \"maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n\" (UID: \"3264f23c-f1cf-4397-8555-96910475df0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n"
Jul 28 01:26:23.633876 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.633781    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx2zg\" (UniqueName: \"kubernetes.io/projected/3264f23c-f1cf-4397-8555-96910475df0e-kube-api-access-wx2zg\") pod \"maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n\" (UID: \"3264f23c-f1cf-4397-8555-96910475df0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n"
Jul 28 01:26:23.734576 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.734543    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3264f23c-f1cf-4397-8555-96910475df0e-openshift-service-ca\") pod \"maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n\" (UID: \"3264f23c-f1cf-4397-8555-96910475df0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n"
Jul 28 01:26:23.734754 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.734589    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3264f23c-f1cf-4397-8555-96910475df0e-maas-api-tls\") pod \"maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n\" (UID: \"3264f23c-f1cf-4397-8555-96910475df0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n"
Jul 28 01:26:23.734754 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.734632    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx2zg\" (UniqueName: \"kubernetes.io/projected/3264f23c-f1cf-4397-8555-96910475df0e-kube-api-access-wx2zg\") pod \"maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n\" (UID: \"3264f23c-f1cf-4397-8555-96910475df0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n"
Jul 28 01:26:23.735280 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.735251    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3264f23c-f1cf-4397-8555-96910475df0e-openshift-service-ca\") pod \"maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n\" (UID: \"3264f23c-f1cf-4397-8555-96910475df0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n"
Jul 28 01:26:23.736926 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.736909    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3264f23c-f1cf-4397-8555-96910475df0e-maas-api-tls\") pod \"maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n\" (UID: \"3264f23c-f1cf-4397-8555-96910475df0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n"
Jul 28 01:26:23.744002 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.743966    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx2zg\" (UniqueName: \"kubernetes.io/projected/3264f23c-f1cf-4397-8555-96910475df0e-kube-api-access-wx2zg\") pod \"maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n\" (UID: \"3264f23c-f1cf-4397-8555-96910475df0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n"
Jul 28 01:26:23.895317 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:23.895281    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n"
Jul 28 01:26:24.044279 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:24.044254    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n"]
Jul 28 01:26:24.045616 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:26:24.045587    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3264f23c_f1cf_4397_8555_96910475df0e.slice/crio-0070b3e4f90109e1855afedda18381dd69d75dd3550ee7fac721ca1817638635 WatchSource:0}: Error finding container 0070b3e4f90109e1855afedda18381dd69d75dd3550ee7fac721ca1817638635: Status 404 returned error can't find the container with id 0070b3e4f90109e1855afedda18381dd69d75dd3550ee7fac721ca1817638635
Jul 28 01:26:24.047139 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:24.047111    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 01:26:24.052376 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:24.052347    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n" event={"ID":"3264f23c-f1cf-4397-8555-96910475df0e","Type":"ContainerStarted","Data":"0070b3e4f90109e1855afedda18381dd69d75dd3550ee7fac721ca1817638635"}
Jul 28 01:26:24.466471 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:24.466433    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92"]
Jul 28 01:26:24.471011 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:24.470995    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92"
Jul 28 01:26:24.475512 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:24.475491    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-c08eabd1\""
Jul 28 01:26:24.475789 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:24.475773    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-c08eabd1-dockercfg-fbnng\""
Jul 28 01:26:24.484573 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:24.484548    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92"]
Jul 28 01:26:24.509642 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:24.509613    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg"]
Jul 28 01:26:24.512920 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:24.512905    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg"
Jul 28 01:26:24.518349 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:24.518320    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg"]
Jul 28 01:26:24.542160 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:24.542131    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c510f1a-4862-4775-910e-25d3bf4b6481-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92\" (UID: \"4c510f1a-4862-4775-910e-25d3bf4b6481\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92"
Jul 28 01:26:24.542327 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:24.542188    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzpcw\" (UniqueName: \"kubernetes.io/projected/4c510f1a-4862-4775-910e-25d3bf4b6481-kube-api-access-wzpcw\") pod \"payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92\" (UID: \"4c510f1a-4862-4775-910e-25d3bf4b6481\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92"
Jul 28 01:26:24.542327 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:24.542267    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52f9a215-3c63-4396-9c5e-0ab03d6f33df-plugins-config-volume\") pod \"payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg\" (UID: \"52f9a215-3c63-4396-9c5e-0ab03d6f33df\") " pod="openshift-ingress/payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg"
Jul 28 01:26:24.542327 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:24.542297    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66nrm\" (UniqueName: \"kubernetes.io/projected/52f9a215-3c63-4396-9c5e-0ab03d6f33df-kube-api-access-66nrm\") pod \"payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg\" (UID: \"52f9a215-3c63-4396-9c5e-0ab03d6f33df\") " pod="openshift-ingress/payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg"
Jul 28 01:26:24.643240 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:24.643201    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52f9a215-3c63-4396-9c5e-0ab03d6f33df-plugins-config-volume\") pod \"payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg\" (UID: \"52f9a215-3c63-4396-9c5e-0ab03d6f33df\") " pod="openshift-ingress/payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg"
Jul 28 01:26:24.643733 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:24.643257    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66nrm\" (UniqueName: \"kubernetes.io/projected/52f9a215-3c63-4396-9c5e-0ab03d6f33df-kube-api-access-66nrm\") pod \"payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg\" (UID: \"52f9a215-3c63-4396-9c5e-0ab03d6f33df\") " pod="openshift-ingress/payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg"
Jul 28 01:26:24.643733 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:24.643309    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c510f1a-4862-4775-910e-25d3bf4b6481-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92\" (UID: \"4c510f1a-4862-4775-910e-25d3bf4b6481\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92"
Jul 28 01:26:24.643733 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:24.643357    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzpcw\" (UniqueName: \"kubernetes.io/projected/4c510f1a-4862-4775-910e-25d3bf4b6481-kube-api-access-wzpcw\") pod \"payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92\" (UID: \"4c510f1a-4862-4775-910e-25d3bf4b6481\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92"
Jul 28 01:26:24.643981 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:24.643959    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c510f1a-4862-4775-910e-25d3bf4b6481-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92\" (UID: \"4c510f1a-4862-4775-910e-25d3bf4b6481\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92"
Jul 28 01:26:24.644083 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:24.644064    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52f9a215-3c63-4396-9c5e-0ab03d6f33df-plugins-config-volume\") pod \"payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg\" (UID: \"52f9a215-3c63-4396-9c5e-0ab03d6f33df\") " pod="openshift-ingress/payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg"
Jul 28 01:26:24.666049 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:24.666017    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzpcw\" (UniqueName: \"kubernetes.io/projected/4c510f1a-4862-4775-910e-25d3bf4b6481-kube-api-access-wzpcw\") pod \"payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92\" (UID: \"4c510f1a-4862-4775-910e-25d3bf4b6481\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92"
Jul 28 01:26:24.667082 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:24.667056    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66nrm\" (UniqueName: \"kubernetes.io/projected/52f9a215-3c63-4396-9c5e-0ab03d6f33df-kube-api-access-66nrm\") pod \"payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg\" (UID: \"52f9a215-3c63-4396-9c5e-0ab03d6f33df\") " pod="openshift-ingress/payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg"
Jul 28 01:26:24.782144 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:24.782061    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92"
Jul 28 01:26:24.825133 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:24.825090    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg"
Jul 28 01:26:24.932848 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:24.932813    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92"]
Jul 28 01:26:24.934159 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:26:24.934130    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c510f1a_4862_4775_910e_25d3bf4b6481.slice/crio-59e56b2a5ce0f332529a616ab042cbe5af27e797c500ac644d04d78d1a0e92b3 WatchSource:0}: Error finding container 59e56b2a5ce0f332529a616ab042cbe5af27e797c500ac644d04d78d1a0e92b3: Status 404 returned error can't find the container with id 59e56b2a5ce0f332529a616ab042cbe5af27e797c500ac644d04d78d1a0e92b3
Jul 28 01:26:24.982608 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:24.982586    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg"]
Jul 28 01:26:24.994341 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:26:24.994317    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52f9a215_3c63_4396_9c5e_0ab03d6f33df.slice/crio-a6cac5cc3d9da4a55198172766c9205a6d5a658bf43d50ea58fadc164a813510 WatchSource:0}: Error finding container a6cac5cc3d9da4a55198172766c9205a6d5a658bf43d50ea58fadc164a813510: Status 404 returned error can't find the container with id a6cac5cc3d9da4a55198172766c9205a6d5a658bf43d50ea58fadc164a813510
Jul 28 01:26:25.058977 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:25.058929    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92" event={"ID":"4c510f1a-4862-4775-910e-25d3bf4b6481","Type":"ContainerStarted","Data":"1a43c00ab83b52330749d05fa426b2eccaf22b34af73db4697a36a9965e7e4a2"}
Jul 28 01:26:25.059120 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:25.059005    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92"
Jul 28 01:26:25.059120 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:25.059029    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92" event={"ID":"4c510f1a-4862-4775-910e-25d3bf4b6481","Type":"ContainerStarted","Data":"59e56b2a5ce0f332529a616ab042cbe5af27e797c500ac644d04d78d1a0e92b3"}
Jul 28 01:26:25.061033 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:25.061007    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n" event={"ID":"3264f23c-f1cf-4397-8555-96910475df0e","Type":"ContainerStarted","Data":"f100a6f9164af159a601c44b3934d68d367751bab779aee16aa7b084c64119f1"}
Jul 28 01:26:25.061164 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:25.061057    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n"
Jul 28 01:26:25.062799 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:25.062768    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg" event={"ID":"52f9a215-3c63-4396-9c5e-0ab03d6f33df","Type":"ContainerStarted","Data":"93d0156e1f89ee8ae60977ff4e86b96f3e94d1640c4ec65249855bd8b9214c4b"}
Jul 28 01:26:25.062908 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:25.062834    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg"
Jul 28 01:26:25.062908 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:25.062864    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg" event={"ID":"52f9a215-3c63-4396-9c5e-0ab03d6f33df","Type":"ContainerStarted","Data":"a6cac5cc3d9da4a55198172766c9205a6d5a658bf43d50ea58fadc164a813510"}
Jul 28 01:26:25.078310 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:25.078258    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92" podStartSLOduration=1.078242344 podStartE2EDuration="1.078242344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:26:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:26:25.076398783 +0000 UTC m=+1955.241206891" watchObservedRunningTime="2026-07-28 01:26:25.078242344 +0000 UTC m=+1955.243050447"
Jul 28 01:26:25.097229 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:25.097168    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg" podStartSLOduration=1.097152637 podStartE2EDuration="1.097152637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:26:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:26:25.094995151 +0000 UTC m=+1955.259803250" watchObservedRunningTime="2026-07-28 01:26:25.097152637 +0000 UTC m=+1955.261960740"
Jul 28 01:26:25.114543 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:25.114489    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n" podStartSLOduration=1.5670631689999999 podStartE2EDuration="2.114471594s" totalImagesPullingTime="547.408425ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:26:25.113290136 +0000 UTC m=+1955.278098265" watchObservedRunningTime="2026-07-28 01:26:25.114471594 +0000 UTC m=+1955.279279700"
Jul 28 01:26:31.073823 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:31.073792    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n"
Jul 28 01:26:36.070411 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:36.070358    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92" podUID="4c510f1a-4862-4775-910e-25d3bf4b6481" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.40:9004: connect: connection refused"
Jul 28 01:26:36.070972 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:36.070808    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg"
Jul 28 01:26:46.069526 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:26:46.069451    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92"
Jul 28 01:29:32.174900 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:32.174863    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c08eabd1-rjbdd"]
Jul 28 01:29:32.178363 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:32.178347    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c08eabd1-rjbdd"
Jul 28 01:29:32.181108 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:32.181082    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8ph8w\""
Jul 28 01:29:32.197394 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:32.197366    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c08eabd1-rjbdd"]
Jul 28 01:29:32.264748 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:32.264708    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7532cff1-0d9e-4754-b63a-f88dfec06cb7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c08eabd1-rjbdd\" (UID: \"7532cff1-0d9e-4754-b63a-f88dfec06cb7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c08eabd1-rjbdd"
Jul 28 01:29:32.365740 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:32.365699    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7532cff1-0d9e-4754-b63a-f88dfec06cb7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c08eabd1-rjbdd\" (UID: \"7532cff1-0d9e-4754-b63a-f88dfec06cb7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c08eabd1-rjbdd"
Jul 28 01:29:32.366373 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:32.366356    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7532cff1-0d9e-4754-b63a-f88dfec06cb7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c08eabd1-rjbdd\" (UID: \"7532cff1-0d9e-4754-b63a-f88dfec06cb7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c08eabd1-rjbdd"
Jul 28 01:29:32.489066 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:32.488975    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c08eabd1-rjbdd"
Jul 28 01:29:32.621600 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:32.621573    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c08eabd1-rjbdd"]
Jul 28 01:29:32.624042 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:29:32.624014    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7532cff1_0d9e_4754_b63a_f88dfec06cb7.slice/crio-812eace9a4b739c5739ba4aaaa4817b6e673711cb6dc5b992d03458082dc5737 WatchSource:0}: Error finding container 812eace9a4b739c5739ba4aaaa4817b6e673711cb6dc5b992d03458082dc5737: Status 404 returned error can't find the container with id 812eace9a4b739c5739ba4aaaa4817b6e673711cb6dc5b992d03458082dc5737
Jul 28 01:29:33.018352 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:33.018310    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c08eabd1-rjbdd" event={"ID":"7532cff1-0d9e-4754-b63a-f88dfec06cb7","Type":"ContainerStarted","Data":"9832685f743c81dbb679c2e0c300a01f7e7876cf3dd848f4413de1a203c69285"}
Jul 28 01:29:33.018515 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:33.018404    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c08eabd1-rjbdd" event={"ID":"7532cff1-0d9e-4754-b63a-f88dfec06cb7","Type":"ContainerStarted","Data":"812eace9a4b739c5739ba4aaaa4817b6e673711cb6dc5b992d03458082dc5737"}
Jul 28 01:29:33.035721 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:33.035671    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c08eabd1-rjbdd" podStartSLOduration=1.035656885 podStartE2EDuration="1.035656885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:29:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:29:33.032784845 +0000 UTC m=+2143.197592974" watchObservedRunningTime="2026-07-28 01:29:33.035656885 +0000 UTC m=+2143.200464991"
Jul 28 01:29:34.024694 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:34.024656    2578 generic.go:356] "Generic (PLEG): container finished" podID="7532cff1-0d9e-4754-b63a-f88dfec06cb7" containerID="9832685f743c81dbb679c2e0c300a01f7e7876cf3dd848f4413de1a203c69285" exitCode=0
Jul 28 01:29:34.025106 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:34.024743    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c08eabd1-rjbdd" event={"ID":"7532cff1-0d9e-4754-b63a-f88dfec06cb7","Type":"ContainerDied","Data":"9832685f743c81dbb679c2e0c300a01f7e7876cf3dd848f4413de1a203c69285"}
Jul 28 01:29:35.167865 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:35.167843    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c08eabd1-rjbdd"
Jul 28 01:29:35.296566 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:35.296471    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7532cff1-0d9e-4754-b63a-f88dfec06cb7-maas-api-service-ca\") pod \"7532cff1-0d9e-4754-b63a-f88dfec06cb7\" (UID: \"7532cff1-0d9e-4754-b63a-f88dfec06cb7\") "
Jul 28 01:29:35.296960 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:35.296933    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7532cff1-0d9e-4754-b63a-f88dfec06cb7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7532cff1-0d9e-4754-b63a-f88dfec06cb7" (UID: "7532cff1-0d9e-4754-b63a-f88dfec06cb7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:29:35.398064 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:35.398035    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7532cff1-0d9e-4754-b63a-f88dfec06cb7-maas-api-service-ca\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:29:36.037550 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:36.037524    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c08eabd1-rjbdd"
Jul 28 01:29:36.037550 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:36.037542    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c08eabd1-rjbdd" event={"ID":"7532cff1-0d9e-4754-b63a-f88dfec06cb7","Type":"ContainerDied","Data":"812eace9a4b739c5739ba4aaaa4817b6e673711cb6dc5b992d03458082dc5737"}
Jul 28 01:29:36.037749 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:36.037583    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="812eace9a4b739c5739ba4aaaa4817b6e673711cb6dc5b992d03458082dc5737"
Jul 28 01:29:42.606352 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:42.606259    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n"]
Jul 28 01:29:42.606819 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:42.606509    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n" podUID="3264f23c-f1cf-4397-8555-96910475df0e" containerName="maas-api" containerID="cri-o://f100a6f9164af159a601c44b3934d68d367751bab779aee16aa7b084c64119f1" gracePeriod=30
Jul 28 01:29:42.658344 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:42.658307    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg"]
Jul 28 01:29:42.658578 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:42.658552    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg" podUID="52f9a215-3c63-4396-9c5e-0ab03d6f33df" containerName="payload-processing" containerID="cri-o://93d0156e1f89ee8ae60977ff4e86b96f3e94d1640c4ec65249855bd8b9214c4b" gracePeriod=30
Jul 28 01:29:42.665628 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:42.665588    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92"]
Jul 28 01:29:42.666428 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:42.666387    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92" podUID="4c510f1a-4862-4775-910e-25d3bf4b6481" containerName="payload-pre-processing" containerID="cri-o://1a43c00ab83b52330749d05fa426b2eccaf22b34af73db4697a36a9965e7e4a2" gracePeriod=30
Jul 28 01:29:42.923204 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:42.923168    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n"
Jul 28 01:29:42.952715 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:42.952689    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg"
Jul 28 01:29:42.960477 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:42.960445    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92"
Jul 28 01:29:42.976624 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:42.976595    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66nrm\" (UniqueName: \"kubernetes.io/projected/52f9a215-3c63-4396-9c5e-0ab03d6f33df-kube-api-access-66nrm\") pod \"52f9a215-3c63-4396-9c5e-0ab03d6f33df\" (UID: \"52f9a215-3c63-4396-9c5e-0ab03d6f33df\") "
Jul 28 01:29:42.976777 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:42.976669    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wx2zg\" (UniqueName: \"kubernetes.io/projected/3264f23c-f1cf-4397-8555-96910475df0e-kube-api-access-wx2zg\") pod \"3264f23c-f1cf-4397-8555-96910475df0e\" (UID: \"3264f23c-f1cf-4397-8555-96910475df0e\") "
Jul 28 01:29:42.976777 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:42.976733    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3264f23c-f1cf-4397-8555-96910475df0e-maas-api-tls\") pod \"3264f23c-f1cf-4397-8555-96910475df0e\" (UID: \"3264f23c-f1cf-4397-8555-96910475df0e\") "
Jul 28 01:29:42.976896 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:42.976783    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3264f23c-f1cf-4397-8555-96910475df0e-openshift-service-ca\") pod \"3264f23c-f1cf-4397-8555-96910475df0e\" (UID: \"3264f23c-f1cf-4397-8555-96910475df0e\") "
Jul 28 01:29:42.976896 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:42.976835    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52f9a215-3c63-4396-9c5e-0ab03d6f33df-plugins-config-volume\") pod \"52f9a215-3c63-4396-9c5e-0ab03d6f33df\" (UID: \"52f9a215-3c63-4396-9c5e-0ab03d6f33df\") "
Jul 28 01:29:42.977588 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:42.977555    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3264f23c-f1cf-4397-8555-96910475df0e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3264f23c-f1cf-4397-8555-96910475df0e" (UID: "3264f23c-f1cf-4397-8555-96910475df0e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:29:42.978560 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:42.978532    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52f9a215-3c63-4396-9c5e-0ab03d6f33df-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "52f9a215-3c63-4396-9c5e-0ab03d6f33df" (UID: "52f9a215-3c63-4396-9c5e-0ab03d6f33df"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:29:42.979865 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:42.979826    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3264f23c-f1cf-4397-8555-96910475df0e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3264f23c-f1cf-4397-8555-96910475df0e" (UID: "3264f23c-f1cf-4397-8555-96910475df0e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:29:42.979967 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:42.979833    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52f9a215-3c63-4396-9c5e-0ab03d6f33df-kube-api-access-66nrm" (OuterVolumeSpecName: "kube-api-access-66nrm") pod "52f9a215-3c63-4396-9c5e-0ab03d6f33df" (UID: "52f9a215-3c63-4396-9c5e-0ab03d6f33df"). InnerVolumeSpecName "kube-api-access-66nrm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:29:42.980012 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:42.979967    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3264f23c-f1cf-4397-8555-96910475df0e-kube-api-access-wx2zg" (OuterVolumeSpecName: "kube-api-access-wx2zg") pod "3264f23c-f1cf-4397-8555-96910475df0e" (UID: "3264f23c-f1cf-4397-8555-96910475df0e"). InnerVolumeSpecName "kube-api-access-wx2zg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:29:43.073869 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.073823    2578 generic.go:356] "Generic (PLEG): container finished" podID="4c510f1a-4862-4775-910e-25d3bf4b6481" containerID="1a43c00ab83b52330749d05fa426b2eccaf22b34af73db4697a36a9965e7e4a2" exitCode=0
Jul 28 01:29:43.074043 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.073882    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92"
Jul 28 01:29:43.074043 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.073908    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92" event={"ID":"4c510f1a-4862-4775-910e-25d3bf4b6481","Type":"ContainerDied","Data":"1a43c00ab83b52330749d05fa426b2eccaf22b34af73db4697a36a9965e7e4a2"}
Jul 28 01:29:43.074043 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.073996    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92" event={"ID":"4c510f1a-4862-4775-910e-25d3bf4b6481","Type":"ContainerDied","Data":"59e56b2a5ce0f332529a616ab042cbe5af27e797c500ac644d04d78d1a0e92b3"}
Jul 28 01:29:43.074043 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.074012    2578 scope.go:117] "RemoveContainer" containerID="1a43c00ab83b52330749d05fa426b2eccaf22b34af73db4697a36a9965e7e4a2"
Jul 28 01:29:43.075365 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.075340    2578 generic.go:356] "Generic (PLEG): container finished" podID="3264f23c-f1cf-4397-8555-96910475df0e" containerID="f100a6f9164af159a601c44b3934d68d367751bab779aee16aa7b084c64119f1" exitCode=0
Jul 28 01:29:43.075467 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.075401    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n" event={"ID":"3264f23c-f1cf-4397-8555-96910475df0e","Type":"ContainerDied","Data":"f100a6f9164af159a601c44b3934d68d367751bab779aee16aa7b084c64119f1"}
Jul 28 01:29:43.075467 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.075444    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n" event={"ID":"3264f23c-f1cf-4397-8555-96910475df0e","Type":"ContainerDied","Data":"0070b3e4f90109e1855afedda18381dd69d75dd3550ee7fac721ca1817638635"}
Jul 28 01:29:43.075581 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.075479    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n"
Jul 28 01:29:43.076895 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.076869    2578 generic.go:356] "Generic (PLEG): container finished" podID="52f9a215-3c63-4396-9c5e-0ab03d6f33df" containerID="93d0156e1f89ee8ae60977ff4e86b96f3e94d1640c4ec65249855bd8b9214c4b" exitCode=0
Jul 28 01:29:43.077071 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.076906    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg" event={"ID":"52f9a215-3c63-4396-9c5e-0ab03d6f33df","Type":"ContainerDied","Data":"93d0156e1f89ee8ae60977ff4e86b96f3e94d1640c4ec65249855bd8b9214c4b"}
Jul 28 01:29:43.077071 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.076927    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg"
Jul 28 01:29:43.077071 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.076943    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg" event={"ID":"52f9a215-3c63-4396-9c5e-0ab03d6f33df","Type":"ContainerDied","Data":"a6cac5cc3d9da4a55198172766c9205a6d5a658bf43d50ea58fadc164a813510"}
Jul 28 01:29:43.077562 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.077537    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzpcw\" (UniqueName: \"kubernetes.io/projected/4c510f1a-4862-4775-910e-25d3bf4b6481-kube-api-access-wzpcw\") pod \"4c510f1a-4862-4775-910e-25d3bf4b6481\" (UID: \"4c510f1a-4862-4775-910e-25d3bf4b6481\") "
Jul 28 01:29:43.077679 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.077616    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c510f1a-4862-4775-910e-25d3bf4b6481-plugins-config-volume\") pod \"4c510f1a-4862-4775-910e-25d3bf4b6481\" (UID: \"4c510f1a-4862-4775-910e-25d3bf4b6481\") "
Jul 28 01:29:43.077848 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.077830    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52f9a215-3c63-4396-9c5e-0ab03d6f33df-plugins-config-volume\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:29:43.077931 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.077853    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66nrm\" (UniqueName: \"kubernetes.io/projected/52f9a215-3c63-4396-9c5e-0ab03d6f33df-kube-api-access-66nrm\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:29:43.077931 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.077866    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wx2zg\" (UniqueName: \"kubernetes.io/projected/3264f23c-f1cf-4397-8555-96910475df0e-kube-api-access-wx2zg\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:29:43.077931 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.077877    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3264f23c-f1cf-4397-8555-96910475df0e-maas-api-tls\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:29:43.077931 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.077890    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3264f23c-f1cf-4397-8555-96910475df0e-openshift-service-ca\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:29:43.078139 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.078052    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c510f1a-4862-4775-910e-25d3bf4b6481-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4c510f1a-4862-4775-910e-25d3bf4b6481" (UID: "4c510f1a-4862-4775-910e-25d3bf4b6481"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:29:43.079572 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.079551    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c510f1a-4862-4775-910e-25d3bf4b6481-kube-api-access-wzpcw" (OuterVolumeSpecName: "kube-api-access-wzpcw") pod "4c510f1a-4862-4775-910e-25d3bf4b6481" (UID: "4c510f1a-4862-4775-910e-25d3bf4b6481"). InnerVolumeSpecName "kube-api-access-wzpcw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:29:43.084836 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.084821    2578 scope.go:117] "RemoveContainer" containerID="1a43c00ab83b52330749d05fa426b2eccaf22b34af73db4697a36a9965e7e4a2"
Jul 28 01:29:43.085160 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:29:43.085140    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a43c00ab83b52330749d05fa426b2eccaf22b34af73db4697a36a9965e7e4a2\": container with ID starting with 1a43c00ab83b52330749d05fa426b2eccaf22b34af73db4697a36a9965e7e4a2 not found: ID does not exist" containerID="1a43c00ab83b52330749d05fa426b2eccaf22b34af73db4697a36a9965e7e4a2"
Jul 28 01:29:43.085232 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.085168    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a43c00ab83b52330749d05fa426b2eccaf22b34af73db4697a36a9965e7e4a2"} err="failed to get container status \"1a43c00ab83b52330749d05fa426b2eccaf22b34af73db4697a36a9965e7e4a2\": rpc error: code = NotFound desc = could not find container \"1a43c00ab83b52330749d05fa426b2eccaf22b34af73db4697a36a9965e7e4a2\": container with ID starting with 1a43c00ab83b52330749d05fa426b2eccaf22b34af73db4697a36a9965e7e4a2 not found: ID does not exist"
Jul 28 01:29:43.085232 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.085185    2578 scope.go:117] "RemoveContainer" containerID="f100a6f9164af159a601c44b3934d68d367751bab779aee16aa7b084c64119f1"
Jul 28 01:29:43.093614 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.093598    2578 scope.go:117] "RemoveContainer" containerID="f100a6f9164af159a601c44b3934d68d367751bab779aee16aa7b084c64119f1"
Jul 28 01:29:43.093855 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:29:43.093836    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f100a6f9164af159a601c44b3934d68d367751bab779aee16aa7b084c64119f1\": container with ID starting with f100a6f9164af159a601c44b3934d68d367751bab779aee16aa7b084c64119f1 not found: ID does not exist" containerID="f100a6f9164af159a601c44b3934d68d367751bab779aee16aa7b084c64119f1"
Jul 28 01:29:43.093916 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.093860    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f100a6f9164af159a601c44b3934d68d367751bab779aee16aa7b084c64119f1"} err="failed to get container status \"f100a6f9164af159a601c44b3934d68d367751bab779aee16aa7b084c64119f1\": rpc error: code = NotFound desc = could not find container \"f100a6f9164af159a601c44b3934d68d367751bab779aee16aa7b084c64119f1\": container with ID starting with f100a6f9164af159a601c44b3934d68d367751bab779aee16aa7b084c64119f1 not found: ID does not exist"
Jul 28 01:29:43.093916 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.093875    2578 scope.go:117] "RemoveContainer" containerID="93d0156e1f89ee8ae60977ff4e86b96f3e94d1640c4ec65249855bd8b9214c4b"
Jul 28 01:29:43.102031 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.101925    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n"]
Jul 28 01:29:43.102104 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.102059    2578 scope.go:117] "RemoveContainer" containerID="93d0156e1f89ee8ae60977ff4e86b96f3e94d1640c4ec65249855bd8b9214c4b"
Jul 28 01:29:43.102904 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:29:43.102495    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93d0156e1f89ee8ae60977ff4e86b96f3e94d1640c4ec65249855bd8b9214c4b\": container with ID starting with 93d0156e1f89ee8ae60977ff4e86b96f3e94d1640c4ec65249855bd8b9214c4b not found: ID does not exist" containerID="93d0156e1f89ee8ae60977ff4e86b96f3e94d1640c4ec65249855bd8b9214c4b"
Jul 28 01:29:43.102904 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.102554    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93d0156e1f89ee8ae60977ff4e86b96f3e94d1640c4ec65249855bd8b9214c4b"} err="failed to get container status \"93d0156e1f89ee8ae60977ff4e86b96f3e94d1640c4ec65249855bd8b9214c4b\": rpc error: code = NotFound desc = could not find container \"93d0156e1f89ee8ae60977ff4e86b96f3e94d1640c4ec65249855bd8b9214c4b\": container with ID starting with 93d0156e1f89ee8ae60977ff4e86b96f3e94d1640c4ec65249855bd8b9214c4b not found: ID does not exist"
Jul 28 01:29:43.106330 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.106303    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c08eabd1-84454bb6c6-wpt8n"]
Jul 28 01:29:43.116922 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.116899    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg"]
Jul 28 01:29:43.119859 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.119836    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-c08eabd1-968885bf-dslxg"]
Jul 28 01:29:43.178740 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.178645    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzpcw\" (UniqueName: \"kubernetes.io/projected/4c510f1a-4862-4775-910e-25d3bf4b6481-kube-api-access-wzpcw\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:29:43.178740 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.178680    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c510f1a-4862-4775-910e-25d3bf4b6481-plugins-config-volume\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:29:43.397092 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.397063    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92"]
Jul 28 01:29:43.400694 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:43.400666    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-c08eabd1-677b795cc6-mgh92"]
Jul 28 01:29:44.402294 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:44.402264    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3264f23c-f1cf-4397-8555-96910475df0e" path="/var/lib/kubelet/pods/3264f23c-f1cf-4397-8555-96910475df0e/volumes"
Jul 28 01:29:44.402673 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:44.402633    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c510f1a-4862-4775-910e-25d3bf4b6481" path="/var/lib/kubelet/pods/4c510f1a-4862-4775-910e-25d3bf4b6481/volumes"
Jul 28 01:29:44.402962 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:44.402949    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52f9a215-3c63-4396-9c5e-0ab03d6f33df" path="/var/lib/kubelet/pods/52f9a215-3c63-4396-9c5e-0ab03d6f33df/volumes"
Jul 28 01:29:47.269030 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:47.268996    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c08eabd1-rjbdd"]
Jul 28 01:29:47.276685 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:47.276656    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c08eabd1-rjbdd"]
Jul 28 01:29:48.402446 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:48.402415    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7532cff1-0d9e-4754-b63a-f88dfec06cb7" path="/var/lib/kubelet/pods/7532cff1-0d9e-4754-b63a-f88dfec06cb7/volumes"
Jul 28 01:29:50.676248 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.676194    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g"]
Jul 28 01:29:50.676846 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.676819    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c510f1a-4862-4775-910e-25d3bf4b6481" containerName="payload-pre-processing"
Jul 28 01:29:50.676846 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.676847    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c510f1a-4862-4775-910e-25d3bf4b6481" containerName="payload-pre-processing"
Jul 28 01:29:50.677021 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.676877    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3264f23c-f1cf-4397-8555-96910475df0e" containerName="maas-api"
Jul 28 01:29:50.677021 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.676886    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3264f23c-f1cf-4397-8555-96910475df0e" containerName="maas-api"
Jul 28 01:29:50.677021 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.676897    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52f9a215-3c63-4396-9c5e-0ab03d6f33df" containerName="payload-processing"
Jul 28 01:29:50.677021 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.676906    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="52f9a215-3c63-4396-9c5e-0ab03d6f33df" containerName="payload-processing"
Jul 28 01:29:50.677021 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.676915    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7532cff1-0d9e-4754-b63a-f88dfec06cb7" containerName="revoke-keys"
Jul 28 01:29:50.677021 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.676924    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7532cff1-0d9e-4754-b63a-f88dfec06cb7" containerName="revoke-keys"
Jul 28 01:29:50.677245 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.677024    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3264f23c-f1cf-4397-8555-96910475df0e" containerName="maas-api"
Jul 28 01:29:50.677245 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.677036    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="52f9a215-3c63-4396-9c5e-0ab03d6f33df" containerName="payload-processing"
Jul 28 01:29:50.677245 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.677047    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c510f1a-4862-4775-910e-25d3bf4b6481" containerName="payload-pre-processing"
Jul 28 01:29:50.677245 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.677057    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7532cff1-0d9e-4754-b63a-f88dfec06cb7" containerName="revoke-keys"
Jul 28 01:29:50.682926 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.682905    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g"
Jul 28 01:29:50.685833 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.685811    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e0fd3dac-serving-cert\""
Jul 28 01:29:50.686029 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.685830    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 01:29:50.686149 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.685884    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lrljm\""
Jul 28 01:29:50.686567 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.686404    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 01:29:50.689056 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.689033    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g"]
Jul 28 01:29:50.747469 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.747433    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65bd7366-0d6c-475a-841f-2a93b2ac99f0-openshift-service-ca\") pod \"maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g\" (UID: \"65bd7366-0d6c-475a-841f-2a93b2ac99f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g"
Jul 28 01:29:50.747630 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.747490    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nzzk\" (UniqueName: \"kubernetes.io/projected/65bd7366-0d6c-475a-841f-2a93b2ac99f0-kube-api-access-8nzzk\") pod \"maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g\" (UID: \"65bd7366-0d6c-475a-841f-2a93b2ac99f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g"
Jul 28 01:29:50.747630 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.747578    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65bd7366-0d6c-475a-841f-2a93b2ac99f0-maas-api-tls\") pod \"maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g\" (UID: \"65bd7366-0d6c-475a-841f-2a93b2ac99f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g"
Jul 28 01:29:50.849067 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.848975    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65bd7366-0d6c-475a-841f-2a93b2ac99f0-openshift-service-ca\") pod \"maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g\" (UID: \"65bd7366-0d6c-475a-841f-2a93b2ac99f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g"
Jul 28 01:29:50.849067 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.849027    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nzzk\" (UniqueName: \"kubernetes.io/projected/65bd7366-0d6c-475a-841f-2a93b2ac99f0-kube-api-access-8nzzk\") pod \"maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g\" (UID: \"65bd7366-0d6c-475a-841f-2a93b2ac99f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g"
Jul 28 01:29:50.849067 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.849061    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65bd7366-0d6c-475a-841f-2a93b2ac99f0-maas-api-tls\") pod \"maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g\" (UID: \"65bd7366-0d6c-475a-841f-2a93b2ac99f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g"
Jul 28 01:29:50.849699 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.849674    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65bd7366-0d6c-475a-841f-2a93b2ac99f0-openshift-service-ca\") pod \"maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g\" (UID: \"65bd7366-0d6c-475a-841f-2a93b2ac99f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g"
Jul 28 01:29:50.851490 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.851472    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65bd7366-0d6c-475a-841f-2a93b2ac99f0-maas-api-tls\") pod \"maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g\" (UID: \"65bd7366-0d6c-475a-841f-2a93b2ac99f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g"
Jul 28 01:29:50.857435 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.857406    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nzzk\" (UniqueName: \"kubernetes.io/projected/65bd7366-0d6c-475a-841f-2a93b2ac99f0-kube-api-access-8nzzk\") pod \"maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g\" (UID: \"65bd7366-0d6c-475a-841f-2a93b2ac99f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g"
Jul 28 01:29:50.996291 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:50.996252    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g"
Jul 28 01:29:51.134178 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:51.134151    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g"]
Jul 28 01:29:51.135522 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:29:51.135493    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65bd7366_0d6c_475a_841f_2a93b2ac99f0.slice/crio-d9b60b793e5a6824adac58b7f8031f7831422901be212842c86ca07e00ac371d WatchSource:0}: Error finding container d9b60b793e5a6824adac58b7f8031f7831422901be212842c86ca07e00ac371d: Status 404 returned error can't find the container with id d9b60b793e5a6824adac58b7f8031f7831422901be212842c86ca07e00ac371d
Jul 28 01:29:51.455249 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:51.455140    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9"]
Jul 28 01:29:51.459994 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:51.459966    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9"
Jul 28 01:29:51.462564 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:51.462535    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-e0fd3dac\""
Jul 28 01:29:51.462696 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:51.462535    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-e0fd3dac-dockercfg-ms84q\""
Jul 28 01:29:51.471014 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:51.470931    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9"]
Jul 28 01:29:51.474716 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:51.474693    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk"]
Jul 28 01:29:51.478243 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:51.478224    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk"
Jul 28 01:29:51.487045 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:51.487023    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk"]
Jul 28 01:29:51.558647 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:51.558616    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6e9ecbd-2117-4a05-a300-089bfe8c8eb2-plugins-config-volume\") pod \"payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk\" (UID: \"d6e9ecbd-2117-4a05-a300-089bfe8c8eb2\") " pod="openshift-ingress/payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk"
Jul 28 01:29:51.558647 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:51.558650    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8615174-1122-40d4-9dac-dbd389dad4da-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9\" (UID: \"b8615174-1122-40d4-9dac-dbd389dad4da\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9"
Jul 28 01:29:51.558900 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:51.558776    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wv5c\" (UniqueName: \"kubernetes.io/projected/d6e9ecbd-2117-4a05-a300-089bfe8c8eb2-kube-api-access-9wv5c\") pod \"payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk\" (UID: \"d6e9ecbd-2117-4a05-a300-089bfe8c8eb2\") " pod="openshift-ingress/payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk"
Jul 28 01:29:51.558900 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:51.558827    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg55j\" (UniqueName: \"kubernetes.io/projected/b8615174-1122-40d4-9dac-dbd389dad4da-kube-api-access-rg55j\") pod \"payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9\" (UID: \"b8615174-1122-40d4-9dac-dbd389dad4da\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9"
Jul 28 01:29:51.659524 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:51.659504    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wv5c\" (UniqueName: \"kubernetes.io/projected/d6e9ecbd-2117-4a05-a300-089bfe8c8eb2-kube-api-access-9wv5c\") pod \"payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk\" (UID: \"d6e9ecbd-2117-4a05-a300-089bfe8c8eb2\") " pod="openshift-ingress/payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk"
Jul 28 01:29:51.659598 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:51.659552    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg55j\" (UniqueName: \"kubernetes.io/projected/b8615174-1122-40d4-9dac-dbd389dad4da-kube-api-access-rg55j\") pod \"payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9\" (UID: \"b8615174-1122-40d4-9dac-dbd389dad4da\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9"
Jul 28 01:29:51.659718 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:51.659615    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6e9ecbd-2117-4a05-a300-089bfe8c8eb2-plugins-config-volume\") pod \"payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk\" (UID: \"d6e9ecbd-2117-4a05-a300-089bfe8c8eb2\") " pod="openshift-ingress/payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk"
Jul 28 01:29:51.659718 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:51.659641    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8615174-1122-40d4-9dac-dbd389dad4da-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9\" (UID: \"b8615174-1122-40d4-9dac-dbd389dad4da\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9"
Jul 28 01:29:51.660603 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:51.660577    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8615174-1122-40d4-9dac-dbd389dad4da-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9\" (UID: \"b8615174-1122-40d4-9dac-dbd389dad4da\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9"
Jul 28 01:29:51.660713 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:51.660577    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6e9ecbd-2117-4a05-a300-089bfe8c8eb2-plugins-config-volume\") pod \"payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk\" (UID: \"d6e9ecbd-2117-4a05-a300-089bfe8c8eb2\") " pod="openshift-ingress/payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk"
Jul 28 01:29:51.668545 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:51.668513    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wv5c\" (UniqueName: \"kubernetes.io/projected/d6e9ecbd-2117-4a05-a300-089bfe8c8eb2-kube-api-access-9wv5c\") pod \"payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk\" (UID: \"d6e9ecbd-2117-4a05-a300-089bfe8c8eb2\") " pod="openshift-ingress/payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk"
Jul 28 01:29:51.668895 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:51.668874    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg55j\" (UniqueName: \"kubernetes.io/projected/b8615174-1122-40d4-9dac-dbd389dad4da-kube-api-access-rg55j\") pod \"payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9\" (UID: \"b8615174-1122-40d4-9dac-dbd389dad4da\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9"
Jul 28 01:29:51.773462 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:51.773424    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9"
Jul 28 01:29:51.790515 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:51.790490    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk"
Jul 28 01:29:51.939892 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:51.939867    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9"]
Jul 28 01:29:51.940491 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:29:51.940468    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8615174_1122_40d4_9dac_dbd389dad4da.slice/crio-271baf51d08f9b9a6c1e4ca5ceff3e9f000f7708d7f2ccafe282a33476ee552d WatchSource:0}: Error finding container 271baf51d08f9b9a6c1e4ca5ceff3e9f000f7708d7f2ccafe282a33476ee552d: Status 404 returned error can't find the container with id 271baf51d08f9b9a6c1e4ca5ceff3e9f000f7708d7f2ccafe282a33476ee552d
Jul 28 01:29:51.978232 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:51.978185    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk"]
Jul 28 01:29:51.982365 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:29:51.982339    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6e9ecbd_2117_4a05_a300_089bfe8c8eb2.slice/crio-4402d7a6dab91dc50198cb6b165450a4a06661bf8d77f1e620c1726af8271bd8 WatchSource:0}: Error finding container 4402d7a6dab91dc50198cb6b165450a4a06661bf8d77f1e620c1726af8271bd8: Status 404 returned error can't find the container with id 4402d7a6dab91dc50198cb6b165450a4a06661bf8d77f1e620c1726af8271bd8
Jul 28 01:29:52.136788 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:52.136725    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g" event={"ID":"65bd7366-0d6c-475a-841f-2a93b2ac99f0","Type":"ContainerStarted","Data":"33e8c3b74557defaca6bbda5ddf04624ef545f4bf4c5d8950ceaf70c602b03be"}
Jul 28 01:29:52.136999 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:52.136834    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g" event={"ID":"65bd7366-0d6c-475a-841f-2a93b2ac99f0","Type":"ContainerStarted","Data":"d9b60b793e5a6824adac58b7f8031f7831422901be212842c86ca07e00ac371d"}
Jul 28 01:29:52.136999 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:52.136869    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g"
Jul 28 01:29:52.138422 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:52.138394    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9" event={"ID":"b8615174-1122-40d4-9dac-dbd389dad4da","Type":"ContainerStarted","Data":"07718f6784d51040f10ac8e48f09878e94f5242e9d0749b6b47f8b4e0de72b7b"}
Jul 28 01:29:52.138518 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:52.138450    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9"
Jul 28 01:29:52.138518 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:52.138472    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9" event={"ID":"b8615174-1122-40d4-9dac-dbd389dad4da","Type":"ContainerStarted","Data":"271baf51d08f9b9a6c1e4ca5ceff3e9f000f7708d7f2ccafe282a33476ee552d"}
Jul 28 01:29:52.139765 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:52.139743    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk" event={"ID":"d6e9ecbd-2117-4a05-a300-089bfe8c8eb2","Type":"ContainerStarted","Data":"53b029c625a357bab718b2d67107d56a4c81ec3830e642923c38198af162a06b"}
Jul 28 01:29:52.139863 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:52.139780    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk" event={"ID":"d6e9ecbd-2117-4a05-a300-089bfe8c8eb2","Type":"ContainerStarted","Data":"4402d7a6dab91dc50198cb6b165450a4a06661bf8d77f1e620c1726af8271bd8"}
Jul 28 01:29:52.139863 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:52.139829    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk"
Jul 28 01:29:52.225663 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:52.225567    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g" podStartSLOduration=1.7077242 podStartE2EDuration="2.225550022s" totalImagesPullingTime="517.825822ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:29:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:29:52.221547823 +0000 UTC m=+2162.386355940" watchObservedRunningTime="2026-07-28 01:29:52.225550022 +0000 UTC m=+2162.390358153"
Jul 28 01:29:52.291117 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:52.291065    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9" podStartSLOduration=1.2910501220000001 podStartE2EDuration="1.291050122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:29:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:29:52.265454411 +0000 UTC m=+2162.430262518" watchObservedRunningTime="2026-07-28 01:29:52.291050122 +0000 UTC m=+2162.455858228"
Jul 28 01:29:52.291677 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:52.291648    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk" podStartSLOduration=1.291639772 podStartE2EDuration="1.291639772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:29:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:29:52.288476343 +0000 UTC m=+2162.453284451" watchObservedRunningTime="2026-07-28 01:29:52.291639772 +0000 UTC m=+2162.456447878"
Jul 28 01:29:58.152265 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:29:58.152233    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g"
Jul 28 01:30:00.132042 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:00.132002    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e0fd3dac-29753370-qfj64"]
Jul 28 01:30:00.135566 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:00.135549    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753370-sz8nm"]
Jul 28 01:30:00.135681 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:00.135662    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e0fd3dac-29753370-qfj64"
Jul 28 01:30:00.138170 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:00.138148    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8ph8w\""
Jul 28 01:30:00.138829 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:00.138815    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753370-sz8nm"
Jul 28 01:30:00.144030 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:00.143693    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753370-sz8nm"]
Jul 28 01:30:00.147770 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:00.147742    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e0fd3dac-29753370-qfj64"]
Jul 28 01:30:00.150040 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:00.149812    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e0fd3dac-29753370-qfj64"
Jul 28 01:30:00.165918 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:00.165173    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753370-sz8nm"
Jul 28 01:30:00.349271 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:00.349246    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753370-sz8nm"]
Jul 28 01:30:00.351147 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:30:00.351118    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod414afbe9_fbbf_4b8f_aa86_1cd59624646b.slice/crio-fd297464aa43722c888f761a54a60681b59f4935b370603dd6fa8d24b562e0f3 WatchSource:0}: Error finding container fd297464aa43722c888f761a54a60681b59f4935b370603dd6fa8d24b562e0f3: Status 404 returned error can't find the container with id fd297464aa43722c888f761a54a60681b59f4935b370603dd6fa8d24b562e0f3
Jul 28 01:30:00.532467 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:00.532441    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e0fd3dac-29753370-qfj64"]
Jul 28 01:30:00.533597 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:30:00.533571    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod519c1102_6f01_48d9_98ab_480451fc99d5.slice/crio-538ea56c69a032c775788d623edf997abfc10605a7f2cb1e9c38d5e3f44dcbef WatchSource:0}: Error finding container 538ea56c69a032c775788d623edf997abfc10605a7f2cb1e9c38d5e3f44dcbef: Status 404 returned error can't find the container with id 538ea56c69a032c775788d623edf997abfc10605a7f2cb1e9c38d5e3f44dcbef
Jul 28 01:30:01.199129 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:01.199096    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e0fd3dac-29753370-qfj64" event={"ID":"519c1102-6f01-48d9-98ab-480451fc99d5","Type":"ContainerStarted","Data":"1fc20ea03e4f2b932792ee3f9f5a55b08664596edd22271b53095884f10a40e8"}
Jul 28 01:30:01.199536 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:01.199156    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e0fd3dac-29753370-qfj64" event={"ID":"519c1102-6f01-48d9-98ab-480451fc99d5","Type":"ContainerStarted","Data":"538ea56c69a032c775788d623edf997abfc10605a7f2cb1e9c38d5e3f44dcbef"}
Jul 28 01:30:01.200768 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:01.200738    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753370-sz8nm" event={"ID":"414afbe9-fbbf-4b8f-aa86-1cd59624646b","Type":"ContainerStarted","Data":"597236aa669f613106575bedfa0770cc138cd2a6b94066272d5f215034510559"}
Jul 28 01:30:01.200880 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:01.200786    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753370-sz8nm" event={"ID":"414afbe9-fbbf-4b8f-aa86-1cd59624646b","Type":"ContainerStarted","Data":"fd297464aa43722c888f761a54a60681b59f4935b370603dd6fa8d24b562e0f3"}
Jul 28 01:30:01.215633 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:01.215587    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e0fd3dac-29753370-qfj64" podStartSLOduration=1.215571574 podStartE2EDuration="1.215571574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:30:01.213420829 +0000 UTC m=+2171.378228941" watchObservedRunningTime="2026-07-28 01:30:01.215571574 +0000 UTC m=+2171.380379680"
Jul 28 01:30:01.229708 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:01.229646    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753370-sz8nm" podStartSLOduration=1.22962988 podStartE2EDuration="1.22962988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:30:01.228749738 +0000 UTC m=+2171.393557844" watchObservedRunningTime="2026-07-28 01:30:01.22962988 +0000 UTC m=+2171.394437987"
Jul 28 01:30:02.207037 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:02.207004    2578 generic.go:356] "Generic (PLEG): container finished" podID="519c1102-6f01-48d9-98ab-480451fc99d5" containerID="1fc20ea03e4f2b932792ee3f9f5a55b08664596edd22271b53095884f10a40e8" exitCode=0
Jul 28 01:30:02.207500 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:02.207100    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e0fd3dac-29753370-qfj64" event={"ID":"519c1102-6f01-48d9-98ab-480451fc99d5","Type":"ContainerDied","Data":"1fc20ea03e4f2b932792ee3f9f5a55b08664596edd22271b53095884f10a40e8"}
Jul 28 01:30:02.208605 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:02.208585    2578 generic.go:356] "Generic (PLEG): container finished" podID="414afbe9-fbbf-4b8f-aa86-1cd59624646b" containerID="597236aa669f613106575bedfa0770cc138cd2a6b94066272d5f215034510559" exitCode=0
Jul 28 01:30:02.208670 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:02.208621    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753370-sz8nm" event={"ID":"414afbe9-fbbf-4b8f-aa86-1cd59624646b","Type":"ContainerDied","Data":"597236aa669f613106575bedfa0770cc138cd2a6b94066272d5f215034510559"}
Jul 28 01:30:03.147675 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:03.147645    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9"
Jul 28 01:30:03.147881 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:03.147836    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk"
Jul 28 01:30:03.348711 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:03.348684    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753370-sz8nm"
Jul 28 01:30:03.384610 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:03.384587    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e0fd3dac-29753370-qfj64"
Jul 28 01:30:04.220299 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:04.220272    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e0fd3dac-29753370-qfj64"
Jul 28 01:30:04.220299 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:04.220290    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e0fd3dac-29753370-qfj64" event={"ID":"519c1102-6f01-48d9-98ab-480451fc99d5","Type":"ContainerDied","Data":"538ea56c69a032c775788d623edf997abfc10605a7f2cb1e9c38d5e3f44dcbef"}
Jul 28 01:30:04.220528 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:04.220337    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="538ea56c69a032c775788d623edf997abfc10605a7f2cb1e9c38d5e3f44dcbef"
Jul 28 01:30:04.221617 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:04.221602    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753370-sz8nm"
Jul 28 01:30:04.221727 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:04.221647    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753370-sz8nm" event={"ID":"414afbe9-fbbf-4b8f-aa86-1cd59624646b","Type":"ContainerDied","Data":"fd297464aa43722c888f761a54a60681b59f4935b370603dd6fa8d24b562e0f3"}
Jul 28 01:30:04.221727 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:30:04.221700    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fd297464aa43722c888f761a54a60681b59f4935b370603dd6fa8d24b562e0f3"
Jul 28 01:32:50.552469 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:32:50.552393    2578 scope.go:117] "RemoveContainer" containerID="bf55e35fb6d33b96ef631e464b65ee1c9e976861d3c91a75a0198e4d1d214ed9"
Jul 28 01:34:57.373828 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:34:57.373793    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0fd3dac-cfmxj"]
Jul 28 01:34:57.376340 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:34:57.374200    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="519c1102-6f01-48d9-98ab-480451fc99d5" containerName="cleanup"
Jul 28 01:34:57.376340 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:34:57.374226    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="519c1102-6f01-48d9-98ab-480451fc99d5" containerName="cleanup"
Jul 28 01:34:57.376340 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:34:57.374244    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="414afbe9-fbbf-4b8f-aa86-1cd59624646b" containerName="cleanup"
Jul 28 01:34:57.376340 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:34:57.374249    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="414afbe9-fbbf-4b8f-aa86-1cd59624646b" containerName="cleanup"
Jul 28 01:34:57.376340 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:34:57.374340    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="519c1102-6f01-48d9-98ab-480451fc99d5" containerName="cleanup"
Jul 28 01:34:57.376340 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:34:57.374350    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="414afbe9-fbbf-4b8f-aa86-1cd59624646b" containerName="cleanup"
Jul 28 01:34:57.377408 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:34:57.377393    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0fd3dac-cfmxj"
Jul 28 01:34:57.380049 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:34:57.380030    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8ph8w\""
Jul 28 01:34:57.391947 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:34:57.391924    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0fd3dac-cfmxj"]
Jul 28 01:34:57.548164 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:34:57.548128    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b35e237a-ac0c-45d6-97d5-20f975a6d93e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e0fd3dac-cfmxj\" (UID: \"b35e237a-ac0c-45d6-97d5-20f975a6d93e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0fd3dac-cfmxj"
Jul 28 01:34:57.648883 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:34:57.648785    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b35e237a-ac0c-45d6-97d5-20f975a6d93e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e0fd3dac-cfmxj\" (UID: \"b35e237a-ac0c-45d6-97d5-20f975a6d93e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0fd3dac-cfmxj"
Jul 28 01:34:57.649514 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:34:57.649488    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b35e237a-ac0c-45d6-97d5-20f975a6d93e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e0fd3dac-cfmxj\" (UID: \"b35e237a-ac0c-45d6-97d5-20f975a6d93e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0fd3dac-cfmxj"
Jul 28 01:34:57.687360 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:34:57.687327    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0fd3dac-cfmxj"
Jul 28 01:34:57.830272 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:34:57.830246    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0fd3dac-cfmxj"]
Jul 28 01:34:57.832552 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:34:57.832522    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb35e237a_ac0c_45d6_97d5_20f975a6d93e.slice/crio-111fca15e86f284a8f469a82df8fd61635b4f7009c15c72004794fa28ea28890 WatchSource:0}: Error finding container 111fca15e86f284a8f469a82df8fd61635b4f7009c15c72004794fa28ea28890: Status 404 returned error can't find the container with id 111fca15e86f284a8f469a82df8fd61635b4f7009c15c72004794fa28ea28890
Jul 28 01:34:58.775765 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:34:58.775674    2578 generic.go:356] "Generic (PLEG): container finished" podID="b35e237a-ac0c-45d6-97d5-20f975a6d93e" containerID="d3e780648e6916632228a4a21e49bc4a7d2df2e4ab0c9d93ed1ddaa68859c9de" exitCode=0
Jul 28 01:34:58.776310 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:34:58.775765    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0fd3dac-cfmxj" event={"ID":"b35e237a-ac0c-45d6-97d5-20f975a6d93e","Type":"ContainerDied","Data":"d3e780648e6916632228a4a21e49bc4a7d2df2e4ab0c9d93ed1ddaa68859c9de"}
Jul 28 01:34:58.776310 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:34:58.775816    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0fd3dac-cfmxj" event={"ID":"b35e237a-ac0c-45d6-97d5-20f975a6d93e","Type":"ContainerStarted","Data":"111fca15e86f284a8f469a82df8fd61635b4f7009c15c72004794fa28ea28890"}
Jul 28 01:34:59.915635 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:34:59.915611    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0fd3dac-cfmxj"
Jul 28 01:35:00.071943 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:00.071858    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b35e237a-ac0c-45d6-97d5-20f975a6d93e-maas-api-service-ca\") pod \"b35e237a-ac0c-45d6-97d5-20f975a6d93e\" (UID: \"b35e237a-ac0c-45d6-97d5-20f975a6d93e\") "
Jul 28 01:35:00.072286 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:00.072264    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b35e237a-ac0c-45d6-97d5-20f975a6d93e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b35e237a-ac0c-45d6-97d5-20f975a6d93e" (UID: "b35e237a-ac0c-45d6-97d5-20f975a6d93e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:35:00.172889 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:00.172853    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b35e237a-ac0c-45d6-97d5-20f975a6d93e-maas-api-service-ca\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:35:00.787861 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:00.787786    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0fd3dac-cfmxj"
Jul 28 01:35:00.788004 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:00.787778    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0fd3dac-cfmxj" event={"ID":"b35e237a-ac0c-45d6-97d5-20f975a6d93e","Type":"ContainerDied","Data":"111fca15e86f284a8f469a82df8fd61635b4f7009c15c72004794fa28ea28890"}
Jul 28 01:35:00.788004 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:00.787911    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="111fca15e86f284a8f469a82df8fd61635b4f7009c15c72004794fa28ea28890"
Jul 28 01:35:07.799119 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:07.799023    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g"]
Jul 28 01:35:07.799876 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:07.799837    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g" podUID="65bd7366-0d6c-475a-841f-2a93b2ac99f0" containerName="maas-api" containerID="cri-o://33e8c3b74557defaca6bbda5ddf04624ef545f4bf4c5d8950ceaf70c602b03be" gracePeriod=30
Jul 28 01:35:07.830232 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:07.830169    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e0fd3dac-29753370-qfj64"]
Jul 28 01:35:07.835485 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:07.835300    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e0fd3dac-29753370-qfj64"]
Jul 28 01:35:07.854424 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:07.854394    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk"]
Jul 28 01:35:07.854696 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:07.854668    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk" podUID="d6e9ecbd-2117-4a05-a300-089bfe8c8eb2" containerName="payload-processing" containerID="cri-o://53b029c625a357bab718b2d67107d56a4c81ec3830e642923c38198af162a06b" gracePeriod=30
Jul 28 01:35:07.866973 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:07.866943    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9"]
Jul 28 01:35:07.867413 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:07.867362    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9" podUID="b8615174-1122-40d4-9dac-dbd389dad4da" containerName="payload-pre-processing" containerID="cri-o://07718f6784d51040f10ac8e48f09878e94f5242e9d0749b6b47f8b4e0de72b7b" gracePeriod=30
Jul 28 01:35:08.123061 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.123032    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g"
Jul 28 01:35:08.150847 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.150819    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nzzk\" (UniqueName: \"kubernetes.io/projected/65bd7366-0d6c-475a-841f-2a93b2ac99f0-kube-api-access-8nzzk\") pod \"65bd7366-0d6c-475a-841f-2a93b2ac99f0\" (UID: \"65bd7366-0d6c-475a-841f-2a93b2ac99f0\") "
Jul 28 01:35:08.151108 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.150872    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65bd7366-0d6c-475a-841f-2a93b2ac99f0-openshift-service-ca\") pod \"65bd7366-0d6c-475a-841f-2a93b2ac99f0\" (UID: \"65bd7366-0d6c-475a-841f-2a93b2ac99f0\") "
Jul 28 01:35:08.151108 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.150935    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65bd7366-0d6c-475a-841f-2a93b2ac99f0-maas-api-tls\") pod \"65bd7366-0d6c-475a-841f-2a93b2ac99f0\" (UID: \"65bd7366-0d6c-475a-841f-2a93b2ac99f0\") "
Jul 28 01:35:08.151625 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.151592    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65bd7366-0d6c-475a-841f-2a93b2ac99f0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "65bd7366-0d6c-475a-841f-2a93b2ac99f0" (UID: "65bd7366-0d6c-475a-841f-2a93b2ac99f0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:35:08.153168 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.153115    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65bd7366-0d6c-475a-841f-2a93b2ac99f0-kube-api-access-8nzzk" (OuterVolumeSpecName: "kube-api-access-8nzzk") pod "65bd7366-0d6c-475a-841f-2a93b2ac99f0" (UID: "65bd7366-0d6c-475a-841f-2a93b2ac99f0"). InnerVolumeSpecName "kube-api-access-8nzzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:35:08.153537 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.153511    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65bd7366-0d6c-475a-841f-2a93b2ac99f0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "65bd7366-0d6c-475a-841f-2a93b2ac99f0" (UID: "65bd7366-0d6c-475a-841f-2a93b2ac99f0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:35:08.155957 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.155935    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk"
Jul 28 01:35:08.173873 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.173585    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9"
Jul 28 01:35:08.252512 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.252482    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rg55j\" (UniqueName: \"kubernetes.io/projected/b8615174-1122-40d4-9dac-dbd389dad4da-kube-api-access-rg55j\") pod \"b8615174-1122-40d4-9dac-dbd389dad4da\" (UID: \"b8615174-1122-40d4-9dac-dbd389dad4da\") "
Jul 28 01:35:08.252700 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.252525    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6e9ecbd-2117-4a05-a300-089bfe8c8eb2-plugins-config-volume\") pod \"d6e9ecbd-2117-4a05-a300-089bfe8c8eb2\" (UID: \"d6e9ecbd-2117-4a05-a300-089bfe8c8eb2\") "
Jul 28 01:35:08.252700 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.252553    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wv5c\" (UniqueName: \"kubernetes.io/projected/d6e9ecbd-2117-4a05-a300-089bfe8c8eb2-kube-api-access-9wv5c\") pod \"d6e9ecbd-2117-4a05-a300-089bfe8c8eb2\" (UID: \"d6e9ecbd-2117-4a05-a300-089bfe8c8eb2\") "
Jul 28 01:35:08.252700 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.252604    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8615174-1122-40d4-9dac-dbd389dad4da-plugins-config-volume\") pod \"b8615174-1122-40d4-9dac-dbd389dad4da\" (UID: \"b8615174-1122-40d4-9dac-dbd389dad4da\") "
Jul 28 01:35:08.252863 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.252795    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nzzk\" (UniqueName: \"kubernetes.io/projected/65bd7366-0d6c-475a-841f-2a93b2ac99f0-kube-api-access-8nzzk\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:35:08.252863 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.252818    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65bd7366-0d6c-475a-841f-2a93b2ac99f0-openshift-service-ca\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:35:08.252863 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.252833    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65bd7366-0d6c-475a-841f-2a93b2ac99f0-maas-api-tls\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:35:08.253051 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.252963    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6e9ecbd-2117-4a05-a300-089bfe8c8eb2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d6e9ecbd-2117-4a05-a300-089bfe8c8eb2" (UID: "d6e9ecbd-2117-4a05-a300-089bfe8c8eb2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:35:08.253051 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.253012    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8615174-1122-40d4-9dac-dbd389dad4da-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b8615174-1122-40d4-9dac-dbd389dad4da" (UID: "b8615174-1122-40d4-9dac-dbd389dad4da"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:35:08.254644 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.254620    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8615174-1122-40d4-9dac-dbd389dad4da-kube-api-access-rg55j" (OuterVolumeSpecName: "kube-api-access-rg55j") pod "b8615174-1122-40d4-9dac-dbd389dad4da" (UID: "b8615174-1122-40d4-9dac-dbd389dad4da"). InnerVolumeSpecName "kube-api-access-rg55j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:35:08.254738 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.254651    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6e9ecbd-2117-4a05-a300-089bfe8c8eb2-kube-api-access-9wv5c" (OuterVolumeSpecName: "kube-api-access-9wv5c") pod "d6e9ecbd-2117-4a05-a300-089bfe8c8eb2" (UID: "d6e9ecbd-2117-4a05-a300-089bfe8c8eb2"). InnerVolumeSpecName "kube-api-access-9wv5c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:35:08.354090 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.353991    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8615174-1122-40d4-9dac-dbd389dad4da-plugins-config-volume\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:35:08.354090 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.354034    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rg55j\" (UniqueName: \"kubernetes.io/projected/b8615174-1122-40d4-9dac-dbd389dad4da-kube-api-access-rg55j\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:35:08.354090 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.354044    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6e9ecbd-2117-4a05-a300-089bfe8c8eb2-plugins-config-volume\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:35:08.354090 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.354052    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wv5c\" (UniqueName: \"kubernetes.io/projected/d6e9ecbd-2117-4a05-a300-089bfe8c8eb2-kube-api-access-9wv5c\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:35:08.402439 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.402404    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="519c1102-6f01-48d9-98ab-480451fc99d5" path="/var/lib/kubelet/pods/519c1102-6f01-48d9-98ab-480451fc99d5/volumes"
Jul 28 01:35:08.836053 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.836018    2578 generic.go:356] "Generic (PLEG): container finished" podID="b8615174-1122-40d4-9dac-dbd389dad4da" containerID="07718f6784d51040f10ac8e48f09878e94f5242e9d0749b6b47f8b4e0de72b7b" exitCode=0
Jul 28 01:35:08.836526 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.836061    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9"
Jul 28 01:35:08.836526 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.836104    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9" event={"ID":"b8615174-1122-40d4-9dac-dbd389dad4da","Type":"ContainerDied","Data":"07718f6784d51040f10ac8e48f09878e94f5242e9d0749b6b47f8b4e0de72b7b"}
Jul 28 01:35:08.836526 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.836169    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9" event={"ID":"b8615174-1122-40d4-9dac-dbd389dad4da","Type":"ContainerDied","Data":"271baf51d08f9b9a6c1e4ca5ceff3e9f000f7708d7f2ccafe282a33476ee552d"}
Jul 28 01:35:08.836526 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.836205    2578 scope.go:117] "RemoveContainer" containerID="07718f6784d51040f10ac8e48f09878e94f5242e9d0749b6b47f8b4e0de72b7b"
Jul 28 01:35:08.837663 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.837641    2578 generic.go:356] "Generic (PLEG): container finished" podID="d6e9ecbd-2117-4a05-a300-089bfe8c8eb2" containerID="53b029c625a357bab718b2d67107d56a4c81ec3830e642923c38198af162a06b" exitCode=0
Jul 28 01:35:08.837765 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.837699    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk"
Jul 28 01:35:08.837765 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.837732    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk" event={"ID":"d6e9ecbd-2117-4a05-a300-089bfe8c8eb2","Type":"ContainerDied","Data":"53b029c625a357bab718b2d67107d56a4c81ec3830e642923c38198af162a06b"}
Jul 28 01:35:08.837878 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.837788    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk" event={"ID":"d6e9ecbd-2117-4a05-a300-089bfe8c8eb2","Type":"ContainerDied","Data":"4402d7a6dab91dc50198cb6b165450a4a06661bf8d77f1e620c1726af8271bd8"}
Jul 28 01:35:08.839414 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.839390    2578 generic.go:356] "Generic (PLEG): container finished" podID="65bd7366-0d6c-475a-841f-2a93b2ac99f0" containerID="33e8c3b74557defaca6bbda5ddf04624ef545f4bf4c5d8950ceaf70c602b03be" exitCode=0
Jul 28 01:35:08.839526 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.839418    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g" event={"ID":"65bd7366-0d6c-475a-841f-2a93b2ac99f0","Type":"ContainerDied","Data":"33e8c3b74557defaca6bbda5ddf04624ef545f4bf4c5d8950ceaf70c602b03be"}
Jul 28 01:35:08.839526 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.839467    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g" event={"ID":"65bd7366-0d6c-475a-841f-2a93b2ac99f0","Type":"ContainerDied","Data":"d9b60b793e5a6824adac58b7f8031f7831422901be212842c86ca07e00ac371d"}
Jul 28 01:35:08.839526 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.839498    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g"
Jul 28 01:35:08.846122 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.846107    2578 scope.go:117] "RemoveContainer" containerID="07718f6784d51040f10ac8e48f09878e94f5242e9d0749b6b47f8b4e0de72b7b"
Jul 28 01:35:08.846393 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:35:08.846376    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07718f6784d51040f10ac8e48f09878e94f5242e9d0749b6b47f8b4e0de72b7b\": container with ID starting with 07718f6784d51040f10ac8e48f09878e94f5242e9d0749b6b47f8b4e0de72b7b not found: ID does not exist" containerID="07718f6784d51040f10ac8e48f09878e94f5242e9d0749b6b47f8b4e0de72b7b"
Jul 28 01:35:08.846455 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.846404    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07718f6784d51040f10ac8e48f09878e94f5242e9d0749b6b47f8b4e0de72b7b"} err="failed to get container status \"07718f6784d51040f10ac8e48f09878e94f5242e9d0749b6b47f8b4e0de72b7b\": rpc error: code = NotFound desc = could not find container \"07718f6784d51040f10ac8e48f09878e94f5242e9d0749b6b47f8b4e0de72b7b\": container with ID starting with 07718f6784d51040f10ac8e48f09878e94f5242e9d0749b6b47f8b4e0de72b7b not found: ID does not exist"
Jul 28 01:35:08.846455 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.846422    2578 scope.go:117] "RemoveContainer" containerID="53b029c625a357bab718b2d67107d56a4c81ec3830e642923c38198af162a06b"
Jul 28 01:35:08.854382 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.854287    2578 scope.go:117] "RemoveContainer" containerID="53b029c625a357bab718b2d67107d56a4c81ec3830e642923c38198af162a06b"
Jul 28 01:35:08.854920 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:35:08.854887    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53b029c625a357bab718b2d67107d56a4c81ec3830e642923c38198af162a06b\": container with ID starting with 53b029c625a357bab718b2d67107d56a4c81ec3830e642923c38198af162a06b not found: ID does not exist" containerID="53b029c625a357bab718b2d67107d56a4c81ec3830e642923c38198af162a06b"
Jul 28 01:35:08.855013 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.854927    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53b029c625a357bab718b2d67107d56a4c81ec3830e642923c38198af162a06b"} err="failed to get container status \"53b029c625a357bab718b2d67107d56a4c81ec3830e642923c38198af162a06b\": rpc error: code = NotFound desc = could not find container \"53b029c625a357bab718b2d67107d56a4c81ec3830e642923c38198af162a06b\": container with ID starting with 53b029c625a357bab718b2d67107d56a4c81ec3830e642923c38198af162a06b not found: ID does not exist"
Jul 28 01:35:08.855013 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.854945    2578 scope.go:117] "RemoveContainer" containerID="33e8c3b74557defaca6bbda5ddf04624ef545f4bf4c5d8950ceaf70c602b03be"
Jul 28 01:35:08.858418 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.858395    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9"]
Jul 28 01:35:08.861565 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.861545    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e0fd3dac-54cd9bf544-5gsh9"]
Jul 28 01:35:08.864054 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.864035    2578 scope.go:117] "RemoveContainer" containerID="33e8c3b74557defaca6bbda5ddf04624ef545f4bf4c5d8950ceaf70c602b03be"
Jul 28 01:35:08.864335 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:35:08.864315    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33e8c3b74557defaca6bbda5ddf04624ef545f4bf4c5d8950ceaf70c602b03be\": container with ID starting with 33e8c3b74557defaca6bbda5ddf04624ef545f4bf4c5d8950ceaf70c602b03be not found: ID does not exist" containerID="33e8c3b74557defaca6bbda5ddf04624ef545f4bf4c5d8950ceaf70c602b03be"
Jul 28 01:35:08.864409 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.864344    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33e8c3b74557defaca6bbda5ddf04624ef545f4bf4c5d8950ceaf70c602b03be"} err="failed to get container status \"33e8c3b74557defaca6bbda5ddf04624ef545f4bf4c5d8950ceaf70c602b03be\": rpc error: code = NotFound desc = could not find container \"33e8c3b74557defaca6bbda5ddf04624ef545f4bf4c5d8950ceaf70c602b03be\": container with ID starting with 33e8c3b74557defaca6bbda5ddf04624ef545f4bf4c5d8950ceaf70c602b03be not found: ID does not exist"
Jul 28 01:35:08.871013 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.870983    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk"]
Jul 28 01:35:08.877035 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.877017    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e0fd3dac-77dc586d88-4w7dk"]
Jul 28 01:35:08.887882 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.887860    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g"]
Jul 28 01:35:08.891183 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:08.891162    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e0fd3dac-5db778cb5c-c6b8g"]
Jul 28 01:35:10.402275 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:10.402235    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65bd7366-0d6c-475a-841f-2a93b2ac99f0" path="/var/lib/kubelet/pods/65bd7366-0d6c-475a-841f-2a93b2ac99f0/volumes"
Jul 28 01:35:10.402814 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:10.402797    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8615174-1122-40d4-9dac-dbd389dad4da" path="/var/lib/kubelet/pods/b8615174-1122-40d4-9dac-dbd389dad4da/volumes"
Jul 28 01:35:10.403140 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:10.403127    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6e9ecbd-2117-4a05-a300-089bfe8c8eb2" path="/var/lib/kubelet/pods/d6e9ecbd-2117-4a05-a300-089bfe8c8eb2/volumes"
Jul 28 01:35:12.459304 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:12.459266    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0fd3dac-cfmxj"]
Jul 28 01:35:12.461779 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:12.461754    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e0fd3dac-cfmxj"]
Jul 28 01:35:14.403316 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:14.403285    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b35e237a-ac0c-45d6-97d5-20f975a6d93e" path="/var/lib/kubelet/pods/b35e237a-ac0c-45d6-97d5-20f975a6d93e/volumes"
Jul 28 01:35:16.736371 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:16.736334    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m"]
Jul 28 01:35:16.736907 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:16.736887    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8615174-1122-40d4-9dac-dbd389dad4da" containerName="payload-pre-processing"
Jul 28 01:35:16.736986 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:16.736910    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8615174-1122-40d4-9dac-dbd389dad4da" containerName="payload-pre-processing"
Jul 28 01:35:16.736986 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:16.736934    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65bd7366-0d6c-475a-841f-2a93b2ac99f0" containerName="maas-api"
Jul 28 01:35:16.736986 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:16.736943    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="65bd7366-0d6c-475a-841f-2a93b2ac99f0" containerName="maas-api"
Jul 28 01:35:16.736986 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:16.736953    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6e9ecbd-2117-4a05-a300-089bfe8c8eb2" containerName="payload-processing"
Jul 28 01:35:16.736986 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:16.736962    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6e9ecbd-2117-4a05-a300-089bfe8c8eb2" containerName="payload-processing"
Jul 28 01:35:16.737281 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:16.736992    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b35e237a-ac0c-45d6-97d5-20f975a6d93e" containerName="revoke-keys"
Jul 28 01:35:16.737281 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:16.737001    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b35e237a-ac0c-45d6-97d5-20f975a6d93e" containerName="revoke-keys"
Jul 28 01:35:16.737281 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:16.737106    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b35e237a-ac0c-45d6-97d5-20f975a6d93e" containerName="revoke-keys"
Jul 28 01:35:16.737281 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:16.737135    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8615174-1122-40d4-9dac-dbd389dad4da" containerName="payload-pre-processing"
Jul 28 01:35:16.737281 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:16.737146    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="65bd7366-0d6c-475a-841f-2a93b2ac99f0" containerName="maas-api"
Jul 28 01:35:16.737281 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:16.737155    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6e9ecbd-2117-4a05-a300-089bfe8c8eb2" containerName="payload-processing"
Jul 28 01:35:16.742066 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:16.742044    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m"
Jul 28 01:35:16.745325 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:16.745303    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 01:35:16.745325 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:16.745315    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-1a2510d6-serving-cert\""
Jul 28 01:35:16.745530 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:16.745353    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 01:35:16.746008 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:16.745989    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lrljm\""
Jul 28 01:35:16.753864 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:16.753834    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m"]
Jul 28 01:35:16.841131 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:16.841086    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62c11f6f-b2c9-4639-8ff9-a62d63260a09-openshift-service-ca\") pod \"maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m\" (UID: \"62c11f6f-b2c9-4639-8ff9-a62d63260a09\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m"
Jul 28 01:35:16.841336 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:16.841157    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54d4p\" (UniqueName: \"kubernetes.io/projected/62c11f6f-b2c9-4639-8ff9-a62d63260a09-kube-api-access-54d4p\") pod \"maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m\" (UID: \"62c11f6f-b2c9-4639-8ff9-a62d63260a09\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m"
Jul 28 01:35:16.841336 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:16.841254    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62c11f6f-b2c9-4639-8ff9-a62d63260a09-maas-api-tls\") pod \"maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m\" (UID: \"62c11f6f-b2c9-4639-8ff9-a62d63260a09\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m"
Jul 28 01:35:16.942496 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:16.942450    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62c11f6f-b2c9-4639-8ff9-a62d63260a09-openshift-service-ca\") pod \"maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m\" (UID: \"62c11f6f-b2c9-4639-8ff9-a62d63260a09\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m"
Jul 28 01:35:16.942706 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:16.942593    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54d4p\" (UniqueName: \"kubernetes.io/projected/62c11f6f-b2c9-4639-8ff9-a62d63260a09-kube-api-access-54d4p\") pod \"maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m\" (UID: \"62c11f6f-b2c9-4639-8ff9-a62d63260a09\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m"
Jul 28 01:35:16.942706 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:16.942625    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62c11f6f-b2c9-4639-8ff9-a62d63260a09-maas-api-tls\") pod \"maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m\" (UID: \"62c11f6f-b2c9-4639-8ff9-a62d63260a09\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m"
Jul 28 01:35:16.943138 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:16.943115    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62c11f6f-b2c9-4639-8ff9-a62d63260a09-openshift-service-ca\") pod \"maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m\" (UID: \"62c11f6f-b2c9-4639-8ff9-a62d63260a09\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m"
Jul 28 01:35:16.945320 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:16.945295    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62c11f6f-b2c9-4639-8ff9-a62d63260a09-maas-api-tls\") pod \"maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m\" (UID: \"62c11f6f-b2c9-4639-8ff9-a62d63260a09\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m"
Jul 28 01:35:16.952302 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:16.952276    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54d4p\" (UniqueName: \"kubernetes.io/projected/62c11f6f-b2c9-4639-8ff9-a62d63260a09-kube-api-access-54d4p\") pod \"maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m\" (UID: \"62c11f6f-b2c9-4639-8ff9-a62d63260a09\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m"
Jul 28 01:35:17.055782 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.055684    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m"
Jul 28 01:35:17.215108 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.215056    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m"]
Jul 28 01:35:17.224490 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:35:17.224454    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62c11f6f_b2c9_4639_8ff9_a62d63260a09.slice/crio-67aa6a4adedc56afa051cdbbe9547ef7efaf004b349f36532fa46f77740b6c9e WatchSource:0}: Error finding container 67aa6a4adedc56afa051cdbbe9547ef7efaf004b349f36532fa46f77740b6c9e: Status 404 returned error can't find the container with id 67aa6a4adedc56afa051cdbbe9547ef7efaf004b349f36532fa46f77740b6c9e
Jul 28 01:35:17.226081 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.226058    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 01:35:17.526418 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.526376    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj"]
Jul 28 01:35:17.531372 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.531343    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj"
Jul 28 01:35:17.533975 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.533950    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-1a2510d6\""
Jul 28 01:35:17.542636 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.542606    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7"]
Jul 28 01:35:17.547228 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.547190    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj"]
Jul 28 01:35:17.547363 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.547310    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7"
Jul 28 01:35:17.550235 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.550185    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-1a2510d6-dockercfg-ffm85\""
Jul 28 01:35:17.558197 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.558168    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7"]
Jul 28 01:35:17.649547 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.649507    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad-plugins-config-volume\") pod \"payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7\" (UID: \"ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad\") " pod="openshift-ingress/payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7"
Jul 28 01:35:17.649725 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.649625    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d96r9\" (UniqueName: \"kubernetes.io/projected/ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad-kube-api-access-d96r9\") pod \"payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7\" (UID: \"ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad\") " pod="openshift-ingress/payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7"
Jul 28 01:35:17.649777 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.649760    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mhm8\" (UniqueName: \"kubernetes.io/projected/97d1856f-3cf1-41ab-b2b4-b529176d0a0b-kube-api-access-6mhm8\") pod \"payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj\" (UID: \"97d1856f-3cf1-41ab-b2b4-b529176d0a0b\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj"
Jul 28 01:35:17.649833 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.649817    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97d1856f-3cf1-41ab-b2b4-b529176d0a0b-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj\" (UID: \"97d1856f-3cf1-41ab-b2b4-b529176d0a0b\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj"
Jul 28 01:35:17.751145 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.751109    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad-plugins-config-volume\") pod \"payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7\" (UID: \"ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad\") " pod="openshift-ingress/payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7"
Jul 28 01:35:17.751672 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.751166    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d96r9\" (UniqueName: \"kubernetes.io/projected/ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad-kube-api-access-d96r9\") pod \"payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7\" (UID: \"ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad\") " pod="openshift-ingress/payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7"
Jul 28 01:35:17.751672 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.751292    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mhm8\" (UniqueName: \"kubernetes.io/projected/97d1856f-3cf1-41ab-b2b4-b529176d0a0b-kube-api-access-6mhm8\") pod \"payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj\" (UID: \"97d1856f-3cf1-41ab-b2b4-b529176d0a0b\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj"
Jul 28 01:35:17.751672 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.751345    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97d1856f-3cf1-41ab-b2b4-b529176d0a0b-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj\" (UID: \"97d1856f-3cf1-41ab-b2b4-b529176d0a0b\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj"
Jul 28 01:35:17.751978 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.751959    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad-plugins-config-volume\") pod \"payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7\" (UID: \"ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad\") " pod="openshift-ingress/payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7"
Jul 28 01:35:17.752034 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.752015    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97d1856f-3cf1-41ab-b2b4-b529176d0a0b-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj\" (UID: \"97d1856f-3cf1-41ab-b2b4-b529176d0a0b\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj"
Jul 28 01:35:17.759795 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.759763    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mhm8\" (UniqueName: \"kubernetes.io/projected/97d1856f-3cf1-41ab-b2b4-b529176d0a0b-kube-api-access-6mhm8\") pod \"payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj\" (UID: \"97d1856f-3cf1-41ab-b2b4-b529176d0a0b\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj"
Jul 28 01:35:17.759927 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.759778    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d96r9\" (UniqueName: \"kubernetes.io/projected/ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad-kube-api-access-d96r9\") pod \"payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7\" (UID: \"ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad\") " pod="openshift-ingress/payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7"
Jul 28 01:35:17.845042 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.845004    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj"
Jul 28 01:35:17.861171 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.861139    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7"
Jul 28 01:35:17.899081 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.899034    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m" event={"ID":"62c11f6f-b2c9-4639-8ff9-a62d63260a09","Type":"ContainerStarted","Data":"0f6875ad57f83ae49ba3b811da7f825811138099d7443ea6bc84bcbafab7ad41"}
Jul 28 01:35:17.899266 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.899117    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m"
Jul 28 01:35:17.899266 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.899144    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m" event={"ID":"62c11f6f-b2c9-4639-8ff9-a62d63260a09","Type":"ContainerStarted","Data":"67aa6a4adedc56afa051cdbbe9547ef7efaf004b349f36532fa46f77740b6c9e"}
Jul 28 01:35:17.919040 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:17.918976    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m" podStartSLOduration=1.382557116 podStartE2EDuration="1.918954555s" totalImagesPullingTime="536.397439ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:35:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:35:17.915325402 +0000 UTC m=+2488.080133516" watchObservedRunningTime="2026-07-28 01:35:17.918954555 +0000 UTC m=+2488.083762666"
Jul 28 01:35:18.024139 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:18.022188    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj"]
Jul 28 01:35:18.025995 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:35:18.025967    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97d1856f_3cf1_41ab_b2b4_b529176d0a0b.slice/crio-d347b7b8549bea89795c7b154b644de13bf9ca2133b0597ee741cc0a1bd943b6 WatchSource:0}: Error finding container d347b7b8549bea89795c7b154b644de13bf9ca2133b0597ee741cc0a1bd943b6: Status 404 returned error can't find the container with id d347b7b8549bea89795c7b154b644de13bf9ca2133b0597ee741cc0a1bd943b6
Jul 28 01:35:18.051912 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:18.051880    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7"]
Jul 28 01:35:18.053265 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:35:18.053239    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef16c973_4dd4_4ead_aa7f_9ee1dd34e3ad.slice/crio-53feb1a036c8be8ecc577114e4975db7af9e96886276be624da48b73908b979b WatchSource:0}: Error finding container 53feb1a036c8be8ecc577114e4975db7af9e96886276be624da48b73908b979b: Status 404 returned error can't find the container with id 53feb1a036c8be8ecc577114e4975db7af9e96886276be624da48b73908b979b
Jul 28 01:35:18.906606 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:18.906572    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7" event={"ID":"ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad","Type":"ContainerStarted","Data":"a82c490ba8190857116a3a1ffb88286e6871cf116570fe57b28764489151c47a"}
Jul 28 01:35:18.907082 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:18.906631    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7" event={"ID":"ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad","Type":"ContainerStarted","Data":"53feb1a036c8be8ecc577114e4975db7af9e96886276be624da48b73908b979b"}
Jul 28 01:35:18.907082 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:18.906661    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7"
Jul 28 01:35:18.908089 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:18.908066    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj" event={"ID":"97d1856f-3cf1-41ab-b2b4-b529176d0a0b","Type":"ContainerStarted","Data":"05b5bc2b2204c81e88fb44cbc6c07c31835bf532468190bd0ccaf451944d164b"}
Jul 28 01:35:18.908261 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:18.908108    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj" event={"ID":"97d1856f-3cf1-41ab-b2b4-b529176d0a0b","Type":"ContainerStarted","Data":"d347b7b8549bea89795c7b154b644de13bf9ca2133b0597ee741cc0a1bd943b6"}
Jul 28 01:35:18.924652 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:18.924601    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7" podStartSLOduration=1.924586846 podStartE2EDuration="1.924586846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:35:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:35:18.923570017 +0000 UTC m=+2489.088378138" watchObservedRunningTime="2026-07-28 01:35:18.924586846 +0000 UTC m=+2489.089394952"
Jul 28 01:35:18.940682 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:18.940634    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj" podStartSLOduration=1.9406197330000001 podStartE2EDuration="1.940619733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:35:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:35:18.938905861 +0000 UTC m=+2489.103713969" watchObservedRunningTime="2026-07-28 01:35:18.940619733 +0000 UTC m=+2489.105427840"
Jul 28 01:35:19.914914 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:19.914886    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj"
Jul 28 01:35:23.914747 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:23.914712    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m"
Jul 28 01:35:29.917439 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:29.917405    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7"
Jul 28 01:35:30.923425 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:30.923389    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj"
Jul 28 01:35:50.592918 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:35:50.592843    2578 scope.go:117] "RemoveContainer" containerID="9832685f743c81dbb679c2e0c300a01f7e7876cf3dd848f4413de1a203c69285"
Jul 28 01:36:50.627301 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:36:50.627271    2578 scope.go:117] "RemoveContainer" containerID="1fc20ea03e4f2b932792ee3f9f5a55b08664596edd22271b53095884f10a40e8"
Jul 28 01:40:23.117666 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:23.117582    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1a2510d6-6hgqt"]
Jul 28 01:40:23.121061 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:23.121045    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1a2510d6-6hgqt"
Jul 28 01:40:23.123962 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:23.123938    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8ph8w\""
Jul 28 01:40:23.136322 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:23.136294    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1a2510d6-6hgqt"]
Jul 28 01:40:23.230780 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:23.230739    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb241d44-f8a3-4010-8471-b3d61bbacdf4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-1a2510d6-6hgqt\" (UID: \"fb241d44-f8a3-4010-8471-b3d61bbacdf4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1a2510d6-6hgqt"
Jul 28 01:40:23.331983 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:23.331945    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb241d44-f8a3-4010-8471-b3d61bbacdf4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-1a2510d6-6hgqt\" (UID: \"fb241d44-f8a3-4010-8471-b3d61bbacdf4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1a2510d6-6hgqt"
Jul 28 01:40:23.332603 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:23.332586    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb241d44-f8a3-4010-8471-b3d61bbacdf4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-1a2510d6-6hgqt\" (UID: \"fb241d44-f8a3-4010-8471-b3d61bbacdf4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1a2510d6-6hgqt"
Jul 28 01:40:23.431275 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:23.431157    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1a2510d6-6hgqt"
Jul 28 01:40:23.563416 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:23.563387    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1a2510d6-6hgqt"]
Jul 28 01:40:23.564433 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:40:23.564408    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb241d44_f8a3_4010_8471_b3d61bbacdf4.slice/crio-39f173b0bc9dddf1dca611fab0b149a66e395bcbb3611dbbbc9c8db79e7075e5 WatchSource:0}: Error finding container 39f173b0bc9dddf1dca611fab0b149a66e395bcbb3611dbbbc9c8db79e7075e5: Status 404 returned error can't find the container with id 39f173b0bc9dddf1dca611fab0b149a66e395bcbb3611dbbbc9c8db79e7075e5
Jul 28 01:40:24.507498 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:24.507463    2578 generic.go:356] "Generic (PLEG): container finished" podID="fb241d44-f8a3-4010-8471-b3d61bbacdf4" containerID="a372dbb1cd16f20182b01f524ca2656936270f8d09fa0d6d4879fa9977b69833" exitCode=0
Jul 28 01:40:24.507871 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:24.507506    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1a2510d6-6hgqt" event={"ID":"fb241d44-f8a3-4010-8471-b3d61bbacdf4","Type":"ContainerDied","Data":"a372dbb1cd16f20182b01f524ca2656936270f8d09fa0d6d4879fa9977b69833"}
Jul 28 01:40:24.507871 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:24.507549    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1a2510d6-6hgqt" event={"ID":"fb241d44-f8a3-4010-8471-b3d61bbacdf4","Type":"ContainerStarted","Data":"39f173b0bc9dddf1dca611fab0b149a66e395bcbb3611dbbbc9c8db79e7075e5"}
Jul 28 01:40:25.653946 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:25.653920    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1a2510d6-6hgqt"
Jul 28 01:40:25.757568 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:25.757531    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb241d44-f8a3-4010-8471-b3d61bbacdf4-maas-api-service-ca\") pod \"fb241d44-f8a3-4010-8471-b3d61bbacdf4\" (UID: \"fb241d44-f8a3-4010-8471-b3d61bbacdf4\") "
Jul 28 01:40:25.757876 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:25.757850    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb241d44-f8a3-4010-8471-b3d61bbacdf4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fb241d44-f8a3-4010-8471-b3d61bbacdf4" (UID: "fb241d44-f8a3-4010-8471-b3d61bbacdf4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:40:25.858472 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:25.858438    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb241d44-f8a3-4010-8471-b3d61bbacdf4-maas-api-service-ca\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:40:26.519588 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:26.519553    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1a2510d6-6hgqt" event={"ID":"fb241d44-f8a3-4010-8471-b3d61bbacdf4","Type":"ContainerDied","Data":"39f173b0bc9dddf1dca611fab0b149a66e395bcbb3611dbbbc9c8db79e7075e5"}
Jul 28 01:40:26.519758 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:26.519604    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="39f173b0bc9dddf1dca611fab0b149a66e395bcbb3611dbbbc9c8db79e7075e5"
Jul 28 01:40:26.519758 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:26.519568    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1a2510d6-6hgqt"
Jul 28 01:40:33.581870 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:33.581784    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m"]
Jul 28 01:40:33.582403 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:33.582021    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m" podUID="62c11f6f-b2c9-4639-8ff9-a62d63260a09" containerName="maas-api" containerID="cri-o://0f6875ad57f83ae49ba3b811da7f825811138099d7443ea6bc84bcbafab7ad41" gracePeriod=30
Jul 28 01:40:33.640978 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:33.640938    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7"]
Jul 28 01:40:33.641261 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:33.641203    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7" podUID="ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad" containerName="payload-processing" containerID="cri-o://a82c490ba8190857116a3a1ffb88286e6871cf116570fe57b28764489151c47a" gracePeriod=30
Jul 28 01:40:33.644515 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:33.644489    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj"]
Jul 28 01:40:33.644710 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:33.644686    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj" podUID="97d1856f-3cf1-41ab-b2b4-b529176d0a0b" containerName="payload-pre-processing" containerID="cri-o://05b5bc2b2204c81e88fb44cbc6c07c31835bf532468190bd0ccaf451944d164b" gracePeriod=30
Jul 28 01:40:33.925180 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:33.925148    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m"
Jul 28 01:40:33.946004 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:33.945974    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj"
Jul 28 01:40:33.948693 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:33.948670    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7"
Jul 28 01:40:34.034747 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.034714    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62c11f6f-b2c9-4639-8ff9-a62d63260a09-maas-api-tls\") pod \"62c11f6f-b2c9-4639-8ff9-a62d63260a09\" (UID: \"62c11f6f-b2c9-4639-8ff9-a62d63260a09\") "
Jul 28 01:40:34.034747 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.034752    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad-plugins-config-volume\") pod \"ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad\" (UID: \"ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad\") "
Jul 28 01:40:34.034994 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.034794    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d96r9\" (UniqueName: \"kubernetes.io/projected/ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad-kube-api-access-d96r9\") pod \"ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad\" (UID: \"ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad\") "
Jul 28 01:40:34.034994 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.034843    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97d1856f-3cf1-41ab-b2b4-b529176d0a0b-plugins-config-volume\") pod \"97d1856f-3cf1-41ab-b2b4-b529176d0a0b\" (UID: \"97d1856f-3cf1-41ab-b2b4-b529176d0a0b\") "
Jul 28 01:40:34.034994 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.034959    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62c11f6f-b2c9-4639-8ff9-a62d63260a09-openshift-service-ca\") pod \"62c11f6f-b2c9-4639-8ff9-a62d63260a09\" (UID: \"62c11f6f-b2c9-4639-8ff9-a62d63260a09\") "
Jul 28 01:40:34.035155 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.034997    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mhm8\" (UniqueName: \"kubernetes.io/projected/97d1856f-3cf1-41ab-b2b4-b529176d0a0b-kube-api-access-6mhm8\") pod \"97d1856f-3cf1-41ab-b2b4-b529176d0a0b\" (UID: \"97d1856f-3cf1-41ab-b2b4-b529176d0a0b\") "
Jul 28 01:40:34.035155 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.035033    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54d4p\" (UniqueName: \"kubernetes.io/projected/62c11f6f-b2c9-4639-8ff9-a62d63260a09-kube-api-access-54d4p\") pod \"62c11f6f-b2c9-4639-8ff9-a62d63260a09\" (UID: \"62c11f6f-b2c9-4639-8ff9-a62d63260a09\") "
Jul 28 01:40:34.035314 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.035279    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad" (UID: "ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:40:34.035314 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.035297    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97d1856f-3cf1-41ab-b2b4-b529176d0a0b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "97d1856f-3cf1-41ab-b2b4-b529176d0a0b" (UID: "97d1856f-3cf1-41ab-b2b4-b529176d0a0b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:40:34.035422 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.035308    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62c11f6f-b2c9-4639-8ff9-a62d63260a09-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "62c11f6f-b2c9-4639-8ff9-a62d63260a09" (UID: "62c11f6f-b2c9-4639-8ff9-a62d63260a09"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:40:34.035479 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.035455    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62c11f6f-b2c9-4639-8ff9-a62d63260a09-openshift-service-ca\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:40:34.035479 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.035472    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad-plugins-config-volume\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:40:34.035586 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.035487    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97d1856f-3cf1-41ab-b2b4-b529176d0a0b-plugins-config-volume\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:40:34.037031 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.037004    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62c11f6f-b2c9-4639-8ff9-a62d63260a09-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "62c11f6f-b2c9-4639-8ff9-a62d63260a09" (UID: "62c11f6f-b2c9-4639-8ff9-a62d63260a09"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:40:34.037146 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.037009    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad-kube-api-access-d96r9" (OuterVolumeSpecName: "kube-api-access-d96r9") pod "ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad" (UID: "ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad"). InnerVolumeSpecName "kube-api-access-d96r9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:40:34.037146 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.037051    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97d1856f-3cf1-41ab-b2b4-b529176d0a0b-kube-api-access-6mhm8" (OuterVolumeSpecName: "kube-api-access-6mhm8") pod "97d1856f-3cf1-41ab-b2b4-b529176d0a0b" (UID: "97d1856f-3cf1-41ab-b2b4-b529176d0a0b"). InnerVolumeSpecName "kube-api-access-6mhm8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:40:34.037265 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.037142    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62c11f6f-b2c9-4639-8ff9-a62d63260a09-kube-api-access-54d4p" (OuterVolumeSpecName: "kube-api-access-54d4p") pod "62c11f6f-b2c9-4639-8ff9-a62d63260a09" (UID: "62c11f6f-b2c9-4639-8ff9-a62d63260a09"). InnerVolumeSpecName "kube-api-access-54d4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:40:34.136956 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.136866    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mhm8\" (UniqueName: \"kubernetes.io/projected/97d1856f-3cf1-41ab-b2b4-b529176d0a0b-kube-api-access-6mhm8\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:40:34.136956 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.136897    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54d4p\" (UniqueName: \"kubernetes.io/projected/62c11f6f-b2c9-4639-8ff9-a62d63260a09-kube-api-access-54d4p\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:40:34.136956 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.136907    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62c11f6f-b2c9-4639-8ff9-a62d63260a09-maas-api-tls\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:40:34.136956 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.136917    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d96r9\" (UniqueName: \"kubernetes.io/projected/ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad-kube-api-access-d96r9\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:40:34.568542 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.568505    2578 generic.go:356] "Generic (PLEG): container finished" podID="97d1856f-3cf1-41ab-b2b4-b529176d0a0b" containerID="05b5bc2b2204c81e88fb44cbc6c07c31835bf532468190bd0ccaf451944d164b" exitCode=0
Jul 28 01:40:34.568734 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.568553    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj"
Jul 28 01:40:34.568734 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.568599    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj" event={"ID":"97d1856f-3cf1-41ab-b2b4-b529176d0a0b","Type":"ContainerDied","Data":"05b5bc2b2204c81e88fb44cbc6c07c31835bf532468190bd0ccaf451944d164b"}
Jul 28 01:40:34.568734 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.568668    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj" event={"ID":"97d1856f-3cf1-41ab-b2b4-b529176d0a0b","Type":"ContainerDied","Data":"d347b7b8549bea89795c7b154b644de13bf9ca2133b0597ee741cc0a1bd943b6"}
Jul 28 01:40:34.568734 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.568702    2578 scope.go:117] "RemoveContainer" containerID="05b5bc2b2204c81e88fb44cbc6c07c31835bf532468190bd0ccaf451944d164b"
Jul 28 01:40:34.570314 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.570236    2578 generic.go:356] "Generic (PLEG): container finished" podID="ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad" containerID="a82c490ba8190857116a3a1ffb88286e6871cf116570fe57b28764489151c47a" exitCode=0
Jul 28 01:40:34.570314 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.570275    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7" event={"ID":"ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad","Type":"ContainerDied","Data":"a82c490ba8190857116a3a1ffb88286e6871cf116570fe57b28764489151c47a"}
Jul 28 01:40:34.570314 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.570295    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7"
Jul 28 01:40:34.570645 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.570578    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7" event={"ID":"ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad","Type":"ContainerDied","Data":"53feb1a036c8be8ecc577114e4975db7af9e96886276be624da48b73908b979b"}
Jul 28 01:40:34.572129 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.572110    2578 generic.go:356] "Generic (PLEG): container finished" podID="62c11f6f-b2c9-4639-8ff9-a62d63260a09" containerID="0f6875ad57f83ae49ba3b811da7f825811138099d7443ea6bc84bcbafab7ad41" exitCode=0
Jul 28 01:40:34.572272 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.572140    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m" event={"ID":"62c11f6f-b2c9-4639-8ff9-a62d63260a09","Type":"ContainerDied","Data":"0f6875ad57f83ae49ba3b811da7f825811138099d7443ea6bc84bcbafab7ad41"}
Jul 28 01:40:34.572272 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.572159    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m"
Jul 28 01:40:34.572272 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.572173    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m" event={"ID":"62c11f6f-b2c9-4639-8ff9-a62d63260a09","Type":"ContainerDied","Data":"67aa6a4adedc56afa051cdbbe9547ef7efaf004b349f36532fa46f77740b6c9e"}
Jul 28 01:40:34.580428 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.580402    2578 scope.go:117] "RemoveContainer" containerID="05b5bc2b2204c81e88fb44cbc6c07c31835bf532468190bd0ccaf451944d164b"
Jul 28 01:40:34.580730 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:40:34.580708    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05b5bc2b2204c81e88fb44cbc6c07c31835bf532468190bd0ccaf451944d164b\": container with ID starting with 05b5bc2b2204c81e88fb44cbc6c07c31835bf532468190bd0ccaf451944d164b not found: ID does not exist" containerID="05b5bc2b2204c81e88fb44cbc6c07c31835bf532468190bd0ccaf451944d164b"
Jul 28 01:40:34.580829 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.580740    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05b5bc2b2204c81e88fb44cbc6c07c31835bf532468190bd0ccaf451944d164b"} err="failed to get container status \"05b5bc2b2204c81e88fb44cbc6c07c31835bf532468190bd0ccaf451944d164b\": rpc error: code = NotFound desc = could not find container \"05b5bc2b2204c81e88fb44cbc6c07c31835bf532468190bd0ccaf451944d164b\": container with ID starting with 05b5bc2b2204c81e88fb44cbc6c07c31835bf532468190bd0ccaf451944d164b not found: ID does not exist"
Jul 28 01:40:34.580829 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.580764    2578 scope.go:117] "RemoveContainer" containerID="a82c490ba8190857116a3a1ffb88286e6871cf116570fe57b28764489151c47a"
Jul 28 01:40:34.590456 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.590326    2578 scope.go:117] "RemoveContainer" containerID="a82c490ba8190857116a3a1ffb88286e6871cf116570fe57b28764489151c47a"
Jul 28 01:40:34.590872 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:40:34.590851    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a82c490ba8190857116a3a1ffb88286e6871cf116570fe57b28764489151c47a\": container with ID starting with a82c490ba8190857116a3a1ffb88286e6871cf116570fe57b28764489151c47a not found: ID does not exist" containerID="a82c490ba8190857116a3a1ffb88286e6871cf116570fe57b28764489151c47a"
Jul 28 01:40:34.590940 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.590886    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a82c490ba8190857116a3a1ffb88286e6871cf116570fe57b28764489151c47a"} err="failed to get container status \"a82c490ba8190857116a3a1ffb88286e6871cf116570fe57b28764489151c47a\": rpc error: code = NotFound desc = could not find container \"a82c490ba8190857116a3a1ffb88286e6871cf116570fe57b28764489151c47a\": container with ID starting with a82c490ba8190857116a3a1ffb88286e6871cf116570fe57b28764489151c47a not found: ID does not exist"
Jul 28 01:40:34.590940 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.590911    2578 scope.go:117] "RemoveContainer" containerID="0f6875ad57f83ae49ba3b811da7f825811138099d7443ea6bc84bcbafab7ad41"
Jul 28 01:40:34.599627 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.599191    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7"]
Jul 28 01:40:34.604719 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.604695    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-1a2510d6-5998cc4fdb-whhf7"]
Jul 28 01:40:34.609992 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.609970    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj"]
Jul 28 01:40:34.614590 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.614564    2578 scope.go:117] "RemoveContainer" containerID="0f6875ad57f83ae49ba3b811da7f825811138099d7443ea6bc84bcbafab7ad41"
Jul 28 01:40:34.614907 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:40:34.614885    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f6875ad57f83ae49ba3b811da7f825811138099d7443ea6bc84bcbafab7ad41\": container with ID starting with 0f6875ad57f83ae49ba3b811da7f825811138099d7443ea6bc84bcbafab7ad41 not found: ID does not exist" containerID="0f6875ad57f83ae49ba3b811da7f825811138099d7443ea6bc84bcbafab7ad41"
Jul 28 01:40:34.614976 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.614921    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f6875ad57f83ae49ba3b811da7f825811138099d7443ea6bc84bcbafab7ad41"} err="failed to get container status \"0f6875ad57f83ae49ba3b811da7f825811138099d7443ea6bc84bcbafab7ad41\": rpc error: code = NotFound desc = could not find container \"0f6875ad57f83ae49ba3b811da7f825811138099d7443ea6bc84bcbafab7ad41\": container with ID starting with 0f6875ad57f83ae49ba3b811da7f825811138099d7443ea6bc84bcbafab7ad41 not found: ID does not exist"
Jul 28 01:40:34.615183 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.615148    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1a2510d6-7dc9f557b4-dwjxj"]
Jul 28 01:40:34.625364 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.625338    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m"]
Jul 28 01:40:34.628965 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:34.628943    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m"]
Jul 28 01:40:36.404055 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:36.404018    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62c11f6f-b2c9-4639-8ff9-a62d63260a09" path="/var/lib/kubelet/pods/62c11f6f-b2c9-4639-8ff9-a62d63260a09/volumes"
Jul 28 01:40:36.404471 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:36.404410    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97d1856f-3cf1-41ab-b2b4-b529176d0a0b" path="/var/lib/kubelet/pods/97d1856f-3cf1-41ab-b2b4-b529176d0a0b/volumes"
Jul 28 01:40:36.404730 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:36.404716    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad" path="/var/lib/kubelet/pods/ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad/volumes"
Jul 28 01:40:36.909232 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:36.909165    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1a2510d6-664d4fb7db-ght5m" podUID="62c11f6f-b2c9-4639-8ff9-a62d63260a09" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.49:8443/health\": context deadline exceeded"
Jul 28 01:40:38.200331 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:38.200295    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1a2510d6-6hgqt"]
Jul 28 01:40:38.202756 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:38.202735    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1a2510d6-6hgqt"]
Jul 28 01:40:38.402045 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:40:38.402006    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb241d44-f8a3-4010-8471-b3d61bbacdf4" path="/var/lib/kubelet/pods/fb241d44-f8a3-4010-8471-b3d61bbacdf4/volumes"
Jul 28 01:41:50.691189 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:41:50.691114    2578 scope.go:117] "RemoveContainer" containerID="d3e780648e6916632228a4a21e49bc4a7d2df2e4ab0c9d93ed1ddaa68859c9de"
Jul 28 01:42:33.253345 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.253312    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj"]
Jul 28 01:42:33.253817 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.253721    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb241d44-f8a3-4010-8471-b3d61bbacdf4" containerName="revoke-keys"
Jul 28 01:42:33.253817 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.253732    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb241d44-f8a3-4010-8471-b3d61bbacdf4" containerName="revoke-keys"
Jul 28 01:42:33.253817 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.253743    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62c11f6f-b2c9-4639-8ff9-a62d63260a09" containerName="maas-api"
Jul 28 01:42:33.253817 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.253749    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="62c11f6f-b2c9-4639-8ff9-a62d63260a09" containerName="maas-api"
Jul 28 01:42:33.253817 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.253758    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad" containerName="payload-processing"
Jul 28 01:42:33.253817 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.253765    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad" containerName="payload-processing"
Jul 28 01:42:33.253817 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.253785    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97d1856f-3cf1-41ab-b2b4-b529176d0a0b" containerName="payload-pre-processing"
Jul 28 01:42:33.253817 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.253791    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="97d1856f-3cf1-41ab-b2b4-b529176d0a0b" containerName="payload-pre-processing"
Jul 28 01:42:33.254068 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.253849    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef16c973-4dd4-4ead-aa7f-9ee1dd34e3ad" containerName="payload-processing"
Jul 28 01:42:33.254068 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.253857    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="62c11f6f-b2c9-4639-8ff9-a62d63260a09" containerName="maas-api"
Jul 28 01:42:33.254068 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.253865    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="97d1856f-3cf1-41ab-b2b4-b529176d0a0b" containerName="payload-pre-processing"
Jul 28 01:42:33.254068 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.253872    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb241d44-f8a3-4010-8471-b3d61bbacdf4" containerName="revoke-keys"
Jul 28 01:42:33.257656 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.257637    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj"
Jul 28 01:42:33.263884 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.263850    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-c69ac9dc-serving-cert\""
Jul 28 01:42:33.263884 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.263879    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 01:42:33.264287 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.264269    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 01:42:33.264366 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.264269    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lrljm\""
Jul 28 01:42:33.283747 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.283721    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj"]
Jul 28 01:42:33.387352 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.387315    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f57r9\" (UniqueName: \"kubernetes.io/projected/241e93eb-0aec-446a-8af3-ece1cb2665db-kube-api-access-f57r9\") pod \"maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj\" (UID: \"241e93eb-0aec-446a-8af3-ece1cb2665db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj"
Jul 28 01:42:33.387569 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.387455    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/241e93eb-0aec-446a-8af3-ece1cb2665db-maas-api-tls\") pod \"maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj\" (UID: \"241e93eb-0aec-446a-8af3-ece1cb2665db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj"
Jul 28 01:42:33.387569 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.387529    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/241e93eb-0aec-446a-8af3-ece1cb2665db-openshift-service-ca\") pod \"maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj\" (UID: \"241e93eb-0aec-446a-8af3-ece1cb2665db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj"
Jul 28 01:42:33.488825 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.488786    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f57r9\" (UniqueName: \"kubernetes.io/projected/241e93eb-0aec-446a-8af3-ece1cb2665db-kube-api-access-f57r9\") pod \"maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj\" (UID: \"241e93eb-0aec-446a-8af3-ece1cb2665db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj"
Jul 28 01:42:33.488825 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.488831    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/241e93eb-0aec-446a-8af3-ece1cb2665db-maas-api-tls\") pod \"maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj\" (UID: \"241e93eb-0aec-446a-8af3-ece1cb2665db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj"
Jul 28 01:42:33.489072 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.488951    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/241e93eb-0aec-446a-8af3-ece1cb2665db-openshift-service-ca\") pod \"maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj\" (UID: \"241e93eb-0aec-446a-8af3-ece1cb2665db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj"
Jul 28 01:42:33.489602 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.489578    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/241e93eb-0aec-446a-8af3-ece1cb2665db-openshift-service-ca\") pod \"maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj\" (UID: \"241e93eb-0aec-446a-8af3-ece1cb2665db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj"
Jul 28 01:42:33.491207 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.491189    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/241e93eb-0aec-446a-8af3-ece1cb2665db-maas-api-tls\") pod \"maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj\" (UID: \"241e93eb-0aec-446a-8af3-ece1cb2665db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj"
Jul 28 01:42:33.503527 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.503473    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f57r9\" (UniqueName: \"kubernetes.io/projected/241e93eb-0aec-446a-8af3-ece1cb2665db-kube-api-access-f57r9\") pod \"maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj\" (UID: \"241e93eb-0aec-446a-8af3-ece1cb2665db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj"
Jul 28 01:42:33.570247 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.570205    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj"
Jul 28 01:42:33.710781 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.710754    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj"]
Jul 28 01:42:33.712755 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:42:33.712734    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod241e93eb_0aec_446a_8af3_ece1cb2665db.slice/crio-30d01cf02df28bffb00fa0ec080b4ad33233347b84401fc1fcfcd2f27139eec1 WatchSource:0}: Error finding container 30d01cf02df28bffb00fa0ec080b4ad33233347b84401fc1fcfcd2f27139eec1: Status 404 returned error can't find the container with id 30d01cf02df28bffb00fa0ec080b4ad33233347b84401fc1fcfcd2f27139eec1
Jul 28 01:42:33.714487 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:33.714468    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 01:42:34.185102 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:34.185070    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj" event={"ID":"241e93eb-0aec-446a-8af3-ece1cb2665db","Type":"ContainerStarted","Data":"30d01cf02df28bffb00fa0ec080b4ad33233347b84401fc1fcfcd2f27139eec1"}
Jul 28 01:42:34.261206 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:34.261170    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj"]
Jul 28 01:42:34.264704 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:34.264680    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj"
Jul 28 01:42:34.267277 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:34.267251    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-c69ac9dc\""
Jul 28 01:42:34.267277 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:34.267266    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-c69ac9dc-dockercfg-hrqjf\""
Jul 28 01:42:34.276019 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:34.275994    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj"]
Jul 28 01:42:34.295840 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:34.295810    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06e96eb1-d4a1-46fe-ba72-b29f35858ca4-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj\" (UID: \"06e96eb1-d4a1-46fe-ba72-b29f35858ca4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj"
Jul 28 01:42:34.296030 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:34.295858    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlx85\" (UniqueName: \"kubernetes.io/projected/06e96eb1-d4a1-46fe-ba72-b29f35858ca4-kube-api-access-qlx85\") pod \"payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj\" (UID: \"06e96eb1-d4a1-46fe-ba72-b29f35858ca4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj"
Jul 28 01:42:34.338482 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:34.338448    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd"]
Jul 28 01:42:34.341980 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:34.341963    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd"
Jul 28 01:42:34.354829 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:34.354800    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd"]
Jul 28 01:42:34.396980 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:34.396950    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9ebd816-f89c-4411-aeed-66544fbf7fc9-plugins-config-volume\") pod \"payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd\" (UID: \"c9ebd816-f89c-4411-aeed-66544fbf7fc9\") " pod="openshift-ingress/payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd"
Jul 28 01:42:34.397177 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:34.397020    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhxll\" (UniqueName: \"kubernetes.io/projected/c9ebd816-f89c-4411-aeed-66544fbf7fc9-kube-api-access-vhxll\") pod \"payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd\" (UID: \"c9ebd816-f89c-4411-aeed-66544fbf7fc9\") " pod="openshift-ingress/payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd"
Jul 28 01:42:34.397177 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:34.397051    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06e96eb1-d4a1-46fe-ba72-b29f35858ca4-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj\" (UID: \"06e96eb1-d4a1-46fe-ba72-b29f35858ca4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj"
Jul 28 01:42:34.397177 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:34.397084    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlx85\" (UniqueName: \"kubernetes.io/projected/06e96eb1-d4a1-46fe-ba72-b29f35858ca4-kube-api-access-qlx85\") pod \"payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj\" (UID: \"06e96eb1-d4a1-46fe-ba72-b29f35858ca4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj"
Jul 28 01:42:34.397937 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:34.397913    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06e96eb1-d4a1-46fe-ba72-b29f35858ca4-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj\" (UID: \"06e96eb1-d4a1-46fe-ba72-b29f35858ca4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj"
Jul 28 01:42:34.447717 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:34.447640    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlx85\" (UniqueName: \"kubernetes.io/projected/06e96eb1-d4a1-46fe-ba72-b29f35858ca4-kube-api-access-qlx85\") pod \"payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj\" (UID: \"06e96eb1-d4a1-46fe-ba72-b29f35858ca4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj"
Jul 28 01:42:34.497800 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:34.497757    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9ebd816-f89c-4411-aeed-66544fbf7fc9-plugins-config-volume\") pod \"payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd\" (UID: \"c9ebd816-f89c-4411-aeed-66544fbf7fc9\") " pod="openshift-ingress/payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd"
Jul 28 01:42:34.498047 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:34.497891    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhxll\" (UniqueName: \"kubernetes.io/projected/c9ebd816-f89c-4411-aeed-66544fbf7fc9-kube-api-access-vhxll\") pod \"payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd\" (UID: \"c9ebd816-f89c-4411-aeed-66544fbf7fc9\") " pod="openshift-ingress/payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd"
Jul 28 01:42:34.498583 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:34.498564    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9ebd816-f89c-4411-aeed-66544fbf7fc9-plugins-config-volume\") pod \"payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd\" (UID: \"c9ebd816-f89c-4411-aeed-66544fbf7fc9\") " pod="openshift-ingress/payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd"
Jul 28 01:42:34.512713 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:34.512691    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhxll\" (UniqueName: \"kubernetes.io/projected/c9ebd816-f89c-4411-aeed-66544fbf7fc9-kube-api-access-vhxll\") pod \"payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd\" (UID: \"c9ebd816-f89c-4411-aeed-66544fbf7fc9\") " pod="openshift-ingress/payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd"
Jul 28 01:42:34.577330 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:34.577287    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj"
Jul 28 01:42:34.653437 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:34.653402    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd"
Jul 28 01:42:34.727361 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:34.727330    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj"]
Jul 28 01:42:34.727907 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:42:34.727877    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06e96eb1_d4a1_46fe_ba72_b29f35858ca4.slice/crio-11f24e2065db932f435deb166f52dfad769a3993f612ec88463dd3194394176b WatchSource:0}: Error finding container 11f24e2065db932f435deb166f52dfad769a3993f612ec88463dd3194394176b: Status 404 returned error can't find the container with id 11f24e2065db932f435deb166f52dfad769a3993f612ec88463dd3194394176b
Jul 28 01:42:34.831500 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:34.831474    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd"]
Jul 28 01:42:34.832515 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:42:34.832487    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9ebd816_f89c_4411_aeed_66544fbf7fc9.slice/crio-abadd11634deb3bde79bf368d71310804e1642617d17b9b1aca58e7a34274f7e WatchSource:0}: Error finding container abadd11634deb3bde79bf368d71310804e1642617d17b9b1aca58e7a34274f7e: Status 404 returned error can't find the container with id abadd11634deb3bde79bf368d71310804e1642617d17b9b1aca58e7a34274f7e
Jul 28 01:42:35.192164 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:35.192119    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj" event={"ID":"241e93eb-0aec-446a-8af3-ece1cb2665db","Type":"ContainerStarted","Data":"afb0cd580a16399ab8ae8bf6e040a216e8b9233a74cf4037b266d9c9dd5902ad"}
Jul 28 01:42:35.192390 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:35.192237    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj"
Jul 28 01:42:35.193744 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:35.193720    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd" event={"ID":"c9ebd816-f89c-4411-aeed-66544fbf7fc9","Type":"ContainerStarted","Data":"34bf58cad839a82cd94377dc2e08dbbd655fdcdd050687b059a02433cf836af2"}
Jul 28 01:42:35.193868 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:35.193761    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd" event={"ID":"c9ebd816-f89c-4411-aeed-66544fbf7fc9","Type":"ContainerStarted","Data":"abadd11634deb3bde79bf368d71310804e1642617d17b9b1aca58e7a34274f7e"}
Jul 28 01:42:35.193926 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:35.193864    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd"
Jul 28 01:42:35.195160 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:35.195140    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj" event={"ID":"06e96eb1-d4a1-46fe-ba72-b29f35858ca4","Type":"ContainerStarted","Data":"8fc661471590f45cff2f4630b2308c1675445eb6d7633ef759a47034da690b50"}
Jul 28 01:42:35.195281 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:35.195185    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj" event={"ID":"06e96eb1-d4a1-46fe-ba72-b29f35858ca4","Type":"ContainerStarted","Data":"11f24e2065db932f435deb166f52dfad769a3993f612ec88463dd3194394176b"}
Jul 28 01:42:35.195281 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:35.195243    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj"
Jul 28 01:42:35.210588 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:35.210539    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj" podStartSLOduration=1.767878686 podStartE2EDuration="2.210525217s" totalImagesPullingTime="442.646531ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:42:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:42:35.208900965 +0000 UTC m=+2925.373709089" watchObservedRunningTime="2026-07-28 01:42:35.210525217 +0000 UTC m=+2925.375333324"
Jul 28 01:42:35.226986 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:35.226929    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd" podStartSLOduration=1.226913296 podStartE2EDuration="1.226913296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:42:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:42:35.225066729 +0000 UTC m=+2925.389874865" watchObservedRunningTime="2026-07-28 01:42:35.226913296 +0000 UTC m=+2925.391721442"
Jul 28 01:42:35.247127 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:35.247075    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj" podStartSLOduration=1.247058114 podStartE2EDuration="1.247058114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:42:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:42:35.245812514 +0000 UTC m=+2925.410620635" watchObservedRunningTime="2026-07-28 01:42:35.247058114 +0000 UTC m=+2925.411866238"
Jul 28 01:42:41.207327 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:41.207295    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj"
Jul 28 01:42:44.832353 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:44.832315    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c69ac9dc-n2zbn"]
Jul 28 01:42:44.837072 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:44.837052    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c69ac9dc-n2zbn"
Jul 28 01:42:44.840136 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:44.840109    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8ph8w\""
Jul 28 01:42:44.844640 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:44.844607    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c69ac9dc-n2zbn"]
Jul 28 01:42:44.889568 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:44.889533    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/465b3bf5-e359-450d-a86b-ec597a6dd290-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-c69ac9dc-n2zbn\" (UID: \"465b3bf5-e359-450d-a86b-ec597a6dd290\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c69ac9dc-n2zbn"
Jul 28 01:42:44.991062 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:44.991022    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/465b3bf5-e359-450d-a86b-ec597a6dd290-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-c69ac9dc-n2zbn\" (UID: \"465b3bf5-e359-450d-a86b-ec597a6dd290\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c69ac9dc-n2zbn"
Jul 28 01:42:44.991739 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:44.991713    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/465b3bf5-e359-450d-a86b-ec597a6dd290-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-c69ac9dc-n2zbn\" (UID: \"465b3bf5-e359-450d-a86b-ec597a6dd290\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c69ac9dc-n2zbn"
Jul 28 01:42:45.151387 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:45.151360    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c69ac9dc-n2zbn"
Jul 28 01:42:45.294843 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:45.294818    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c69ac9dc-n2zbn"]
Jul 28 01:42:45.297715 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:42:45.297687    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod465b3bf5_e359_450d_a86b_ec597a6dd290.slice/crio-237b389b98b88499b298f60ddf83720a5f0bb062cf92667f68d212879950ebb1 WatchSource:0}: Error finding container 237b389b98b88499b298f60ddf83720a5f0bb062cf92667f68d212879950ebb1: Status 404 returned error can't find the container with id 237b389b98b88499b298f60ddf83720a5f0bb062cf92667f68d212879950ebb1
Jul 28 01:42:46.203332 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:46.203301    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd"
Jul 28 01:42:46.203723 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:46.203576    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj"
Jul 28 01:42:46.257055 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:46.256970    2578 generic.go:356] "Generic (PLEG): container finished" podID="465b3bf5-e359-450d-a86b-ec597a6dd290" containerID="a8c75e3d38aa7f6fd0a63daafc716f9dc1ead20737e24d7c0cc87a6ef579dc43" exitCode=0
Jul 28 01:42:46.257247 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:46.257107    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c69ac9dc-n2zbn" event={"ID":"465b3bf5-e359-450d-a86b-ec597a6dd290","Type":"ContainerDied","Data":"a8c75e3d38aa7f6fd0a63daafc716f9dc1ead20737e24d7c0cc87a6ef579dc43"}
Jul 28 01:42:46.257247 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:46.257175    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c69ac9dc-n2zbn" event={"ID":"465b3bf5-e359-450d-a86b-ec597a6dd290","Type":"ContainerStarted","Data":"237b389b98b88499b298f60ddf83720a5f0bb062cf92667f68d212879950ebb1"}
Jul 28 01:42:47.399117 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:47.399087    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c69ac9dc-n2zbn"
Jul 28 01:42:47.513700 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:47.513609    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/465b3bf5-e359-450d-a86b-ec597a6dd290-maas-api-service-ca\") pod \"465b3bf5-e359-450d-a86b-ec597a6dd290\" (UID: \"465b3bf5-e359-450d-a86b-ec597a6dd290\") "
Jul 28 01:42:47.513949 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:47.513922    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/465b3bf5-e359-450d-a86b-ec597a6dd290-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "465b3bf5-e359-450d-a86b-ec597a6dd290" (UID: "465b3bf5-e359-450d-a86b-ec597a6dd290"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:42:47.514086 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:47.514069    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/465b3bf5-e359-450d-a86b-ec597a6dd290-maas-api-service-ca\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:42:48.268841 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:48.268809    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c69ac9dc-n2zbn" event={"ID":"465b3bf5-e359-450d-a86b-ec597a6dd290","Type":"ContainerDied","Data":"237b389b98b88499b298f60ddf83720a5f0bb062cf92667f68d212879950ebb1"}
Jul 28 01:42:48.269025 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:48.268863    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="237b389b98b88499b298f60ddf83720a5f0bb062cf92667f68d212879950ebb1"
Jul 28 01:42:48.269025 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:48.268820    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c69ac9dc-n2zbn"
Jul 28 01:42:54.888030 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:54.887989    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj"]
Jul 28 01:42:54.888529 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:54.888339    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj" podUID="241e93eb-0aec-446a-8af3-ece1cb2665db" containerName="maas-api" containerID="cri-o://afb0cd580a16399ab8ae8bf6e040a216e8b9233a74cf4037b266d9c9dd5902ad" gracePeriod=30
Jul 28 01:42:54.944377 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:54.944340    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd"]
Jul 28 01:42:54.944692 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:54.944657    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd" podUID="c9ebd816-f89c-4411-aeed-66544fbf7fc9" containerName="payload-processing" containerID="cri-o://34bf58cad839a82cd94377dc2e08dbbd655fdcdd050687b059a02433cf836af2" gracePeriod=30
Jul 28 01:42:54.952480 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:54.952349    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj"]
Jul 28 01:42:54.953265 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:54.952799    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj" podUID="06e96eb1-d4a1-46fe-ba72-b29f35858ca4" containerName="payload-pre-processing" containerID="cri-o://8fc661471590f45cff2f4630b2308c1675445eb6d7633ef759a47034da690b50" gracePeriod=30
Jul 28 01:42:55.221078 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.221050    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj"
Jul 28 01:42:55.238794 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.238511    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd"
Jul 28 01:42:55.247933 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.247906    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj"
Jul 28 01:42:55.284480 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.284450    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/241e93eb-0aec-446a-8af3-ece1cb2665db-maas-api-tls\") pod \"241e93eb-0aec-446a-8af3-ece1cb2665db\" (UID: \"241e93eb-0aec-446a-8af3-ece1cb2665db\") "
Jul 28 01:42:55.284658 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.284495    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/241e93eb-0aec-446a-8af3-ece1cb2665db-openshift-service-ca\") pod \"241e93eb-0aec-446a-8af3-ece1cb2665db\" (UID: \"241e93eb-0aec-446a-8af3-ece1cb2665db\") "
Jul 28 01:42:55.284658 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.284524    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f57r9\" (UniqueName: \"kubernetes.io/projected/241e93eb-0aec-446a-8af3-ece1cb2665db-kube-api-access-f57r9\") pod \"241e93eb-0aec-446a-8af3-ece1cb2665db\" (UID: \"241e93eb-0aec-446a-8af3-ece1cb2665db\") "
Jul 28 01:42:55.284658 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.284543    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9ebd816-f89c-4411-aeed-66544fbf7fc9-plugins-config-volume\") pod \"c9ebd816-f89c-4411-aeed-66544fbf7fc9\" (UID: \"c9ebd816-f89c-4411-aeed-66544fbf7fc9\") "
Jul 28 01:42:55.284865 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.284665    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhxll\" (UniqueName: \"kubernetes.io/projected/c9ebd816-f89c-4411-aeed-66544fbf7fc9-kube-api-access-vhxll\") pod \"c9ebd816-f89c-4411-aeed-66544fbf7fc9\" (UID: \"c9ebd816-f89c-4411-aeed-66544fbf7fc9\") "
Jul 28 01:42:55.284927 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.284876    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/241e93eb-0aec-446a-8af3-ece1cb2665db-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "241e93eb-0aec-446a-8af3-ece1cb2665db" (UID: "241e93eb-0aec-446a-8af3-ece1cb2665db"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:42:55.284927 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.284897    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9ebd816-f89c-4411-aeed-66544fbf7fc9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c9ebd816-f89c-4411-aeed-66544fbf7fc9" (UID: "c9ebd816-f89c-4411-aeed-66544fbf7fc9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:42:55.285168 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.285132    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/241e93eb-0aec-446a-8af3-ece1cb2665db-openshift-service-ca\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:42:55.285168 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.285168    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9ebd816-f89c-4411-aeed-66544fbf7fc9-plugins-config-volume\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:42:55.286724 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.286697    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/241e93eb-0aec-446a-8af3-ece1cb2665db-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "241e93eb-0aec-446a-8af3-ece1cb2665db" (UID: "241e93eb-0aec-446a-8af3-ece1cb2665db"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:42:55.286816 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.286696    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9ebd816-f89c-4411-aeed-66544fbf7fc9-kube-api-access-vhxll" (OuterVolumeSpecName: "kube-api-access-vhxll") pod "c9ebd816-f89c-4411-aeed-66544fbf7fc9" (UID: "c9ebd816-f89c-4411-aeed-66544fbf7fc9"). InnerVolumeSpecName "kube-api-access-vhxll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:42:55.286816 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.286751    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/241e93eb-0aec-446a-8af3-ece1cb2665db-kube-api-access-f57r9" (OuterVolumeSpecName: "kube-api-access-f57r9") pod "241e93eb-0aec-446a-8af3-ece1cb2665db" (UID: "241e93eb-0aec-446a-8af3-ece1cb2665db"). InnerVolumeSpecName "kube-api-access-f57r9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:42:55.312769 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.312733    2578 generic.go:356] "Generic (PLEG): container finished" podID="c9ebd816-f89c-4411-aeed-66544fbf7fc9" containerID="34bf58cad839a82cd94377dc2e08dbbd655fdcdd050687b059a02433cf836af2" exitCode=0
Jul 28 01:42:55.312956 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.312778    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd"
Jul 28 01:42:55.312956 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.312826    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd" event={"ID":"c9ebd816-f89c-4411-aeed-66544fbf7fc9","Type":"ContainerDied","Data":"34bf58cad839a82cd94377dc2e08dbbd655fdcdd050687b059a02433cf836af2"}
Jul 28 01:42:55.312956 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.312886    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd" event={"ID":"c9ebd816-f89c-4411-aeed-66544fbf7fc9","Type":"ContainerDied","Data":"abadd11634deb3bde79bf368d71310804e1642617d17b9b1aca58e7a34274f7e"}
Jul 28 01:42:55.312956 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.312917    2578 scope.go:117] "RemoveContainer" containerID="34bf58cad839a82cd94377dc2e08dbbd655fdcdd050687b059a02433cf836af2"
Jul 28 01:42:55.314164 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.314139    2578 generic.go:356] "Generic (PLEG): container finished" podID="06e96eb1-d4a1-46fe-ba72-b29f35858ca4" containerID="8fc661471590f45cff2f4630b2308c1675445eb6d7633ef759a47034da690b50" exitCode=0
Jul 28 01:42:55.314282 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.314185    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj"
Jul 28 01:42:55.314282 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.314200    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj" event={"ID":"06e96eb1-d4a1-46fe-ba72-b29f35858ca4","Type":"ContainerDied","Data":"8fc661471590f45cff2f4630b2308c1675445eb6d7633ef759a47034da690b50"}
Jul 28 01:42:55.314400 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.314289    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj" event={"ID":"06e96eb1-d4a1-46fe-ba72-b29f35858ca4","Type":"ContainerDied","Data":"11f24e2065db932f435deb166f52dfad769a3993f612ec88463dd3194394176b"}
Jul 28 01:42:55.315607 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.315584    2578 generic.go:356] "Generic (PLEG): container finished" podID="241e93eb-0aec-446a-8af3-ece1cb2665db" containerID="afb0cd580a16399ab8ae8bf6e040a216e8b9233a74cf4037b266d9c9dd5902ad" exitCode=0
Jul 28 01:42:55.315735 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.315618    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj" event={"ID":"241e93eb-0aec-446a-8af3-ece1cb2665db","Type":"ContainerDied","Data":"afb0cd580a16399ab8ae8bf6e040a216e8b9233a74cf4037b266d9c9dd5902ad"}
Jul 28 01:42:55.315735 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.315626    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj"
Jul 28 01:42:55.315735 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.315654    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj" event={"ID":"241e93eb-0aec-446a-8af3-ece1cb2665db","Type":"ContainerDied","Data":"30d01cf02df28bffb00fa0ec080b4ad33233347b84401fc1fcfcd2f27139eec1"}
Jul 28 01:42:55.322584 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.322560    2578 scope.go:117] "RemoveContainer" containerID="34bf58cad839a82cd94377dc2e08dbbd655fdcdd050687b059a02433cf836af2"
Jul 28 01:42:55.322873 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:42:55.322850    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34bf58cad839a82cd94377dc2e08dbbd655fdcdd050687b059a02433cf836af2\": container with ID starting with 34bf58cad839a82cd94377dc2e08dbbd655fdcdd050687b059a02433cf836af2 not found: ID does not exist" containerID="34bf58cad839a82cd94377dc2e08dbbd655fdcdd050687b059a02433cf836af2"
Jul 28 01:42:55.322965 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.322884    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34bf58cad839a82cd94377dc2e08dbbd655fdcdd050687b059a02433cf836af2"} err="failed to get container status \"34bf58cad839a82cd94377dc2e08dbbd655fdcdd050687b059a02433cf836af2\": rpc error: code = NotFound desc = could not find container \"34bf58cad839a82cd94377dc2e08dbbd655fdcdd050687b059a02433cf836af2\": container with ID starting with 34bf58cad839a82cd94377dc2e08dbbd655fdcdd050687b059a02433cf836af2 not found: ID does not exist"
Jul 28 01:42:55.322965 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.322906    2578 scope.go:117] "RemoveContainer" containerID="8fc661471590f45cff2f4630b2308c1675445eb6d7633ef759a47034da690b50"
Jul 28 01:42:55.331361 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.331343    2578 scope.go:117] "RemoveContainer" containerID="8fc661471590f45cff2f4630b2308c1675445eb6d7633ef759a47034da690b50"
Jul 28 01:42:55.331610 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:42:55.331591    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fc661471590f45cff2f4630b2308c1675445eb6d7633ef759a47034da690b50\": container with ID starting with 8fc661471590f45cff2f4630b2308c1675445eb6d7633ef759a47034da690b50 not found: ID does not exist" containerID="8fc661471590f45cff2f4630b2308c1675445eb6d7633ef759a47034da690b50"
Jul 28 01:42:55.331682 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.331622    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fc661471590f45cff2f4630b2308c1675445eb6d7633ef759a47034da690b50"} err="failed to get container status \"8fc661471590f45cff2f4630b2308c1675445eb6d7633ef759a47034da690b50\": rpc error: code = NotFound desc = could not find container \"8fc661471590f45cff2f4630b2308c1675445eb6d7633ef759a47034da690b50\": container with ID starting with 8fc661471590f45cff2f4630b2308c1675445eb6d7633ef759a47034da690b50 not found: ID does not exist"
Jul 28 01:42:55.331682 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.331646    2578 scope.go:117] "RemoveContainer" containerID="afb0cd580a16399ab8ae8bf6e040a216e8b9233a74cf4037b266d9c9dd5902ad"
Jul 28 01:42:55.336712 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.336641    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd"]
Jul 28 01:42:55.340048 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.340027    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c69ac9dc-6b8f45bc7d-b8tdd"]
Jul 28 01:42:55.341305 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.341287    2578 scope.go:117] "RemoveContainer" containerID="afb0cd580a16399ab8ae8bf6e040a216e8b9233a74cf4037b266d9c9dd5902ad"
Jul 28 01:42:55.341586 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:42:55.341565    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afb0cd580a16399ab8ae8bf6e040a216e8b9233a74cf4037b266d9c9dd5902ad\": container with ID starting with afb0cd580a16399ab8ae8bf6e040a216e8b9233a74cf4037b266d9c9dd5902ad not found: ID does not exist" containerID="afb0cd580a16399ab8ae8bf6e040a216e8b9233a74cf4037b266d9c9dd5902ad"
Jul 28 01:42:55.341649 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.341591    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afb0cd580a16399ab8ae8bf6e040a216e8b9233a74cf4037b266d9c9dd5902ad"} err="failed to get container status \"afb0cd580a16399ab8ae8bf6e040a216e8b9233a74cf4037b266d9c9dd5902ad\": rpc error: code = NotFound desc = could not find container \"afb0cd580a16399ab8ae8bf6e040a216e8b9233a74cf4037b266d9c9dd5902ad\": container with ID starting with afb0cd580a16399ab8ae8bf6e040a216e8b9233a74cf4037b266d9c9dd5902ad not found: ID does not exist"
Jul 28 01:42:55.350616 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.350586    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj"]
Jul 28 01:42:55.354374 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.354346    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c69ac9dc-5d56c88d6b-8x2mj"]
Jul 28 01:42:55.385877 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.385848    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qlx85\" (UniqueName: \"kubernetes.io/projected/06e96eb1-d4a1-46fe-ba72-b29f35858ca4-kube-api-access-qlx85\") pod \"06e96eb1-d4a1-46fe-ba72-b29f35858ca4\" (UID: \"06e96eb1-d4a1-46fe-ba72-b29f35858ca4\") "
Jul 28 01:42:55.386059 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.385888    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06e96eb1-d4a1-46fe-ba72-b29f35858ca4-plugins-config-volume\") pod \"06e96eb1-d4a1-46fe-ba72-b29f35858ca4\" (UID: \"06e96eb1-d4a1-46fe-ba72-b29f35858ca4\") "
Jul 28 01:42:55.386101 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.386061    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhxll\" (UniqueName: \"kubernetes.io/projected/c9ebd816-f89c-4411-aeed-66544fbf7fc9-kube-api-access-vhxll\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:42:55.386101 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.386073    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/241e93eb-0aec-446a-8af3-ece1cb2665db-maas-api-tls\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:42:55.386101 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.386088    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f57r9\" (UniqueName: \"kubernetes.io/projected/241e93eb-0aec-446a-8af3-ece1cb2665db-kube-api-access-f57r9\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:42:55.386351 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.386324    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06e96eb1-d4a1-46fe-ba72-b29f35858ca4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "06e96eb1-d4a1-46fe-ba72-b29f35858ca4" (UID: "06e96eb1-d4a1-46fe-ba72-b29f35858ca4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:42:55.388043 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.388019    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06e96eb1-d4a1-46fe-ba72-b29f35858ca4-kube-api-access-qlx85" (OuterVolumeSpecName: "kube-api-access-qlx85") pod "06e96eb1-d4a1-46fe-ba72-b29f35858ca4" (UID: "06e96eb1-d4a1-46fe-ba72-b29f35858ca4"). InnerVolumeSpecName "kube-api-access-qlx85". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:42:55.486797 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.486703    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlx85\" (UniqueName: \"kubernetes.io/projected/06e96eb1-d4a1-46fe-ba72-b29f35858ca4-kube-api-access-qlx85\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:42:55.486797 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.486737    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06e96eb1-d4a1-46fe-ba72-b29f35858ca4-plugins-config-volume\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:42:55.636309 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.636279    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj"]
Jul 28 01:42:55.640228 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:55.640184    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c69ac9dc-ccfb694b8-lsphj"]
Jul 28 01:42:56.402490 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:56.402460    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06e96eb1-d4a1-46fe-ba72-b29f35858ca4" path="/var/lib/kubelet/pods/06e96eb1-d4a1-46fe-ba72-b29f35858ca4/volumes"
Jul 28 01:42:56.402858 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:56.402851    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="241e93eb-0aec-446a-8af3-ece1cb2665db" path="/var/lib/kubelet/pods/241e93eb-0aec-446a-8af3-ece1cb2665db/volumes"
Jul 28 01:42:56.403197 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:56.403183    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9ebd816-f89c-4411-aeed-66544fbf7fc9" path="/var/lib/kubelet/pods/c9ebd816-f89c-4411-aeed-66544fbf7fc9/volumes"
Jul 28 01:42:59.907520 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:59.907480    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c69ac9dc-n2zbn"]
Jul 28 01:42:59.910359 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:42:59.910336    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c69ac9dc-n2zbn"]
Jul 28 01:43:00.402035 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:00.401999    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="465b3bf5-e359-450d-a86b-ec597a6dd290" path="/var/lib/kubelet/pods/465b3bf5-e359-450d-a86b-ec597a6dd290/volumes"
Jul 28 01:43:29.248906 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.248823    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb"]
Jul 28 01:43:29.249492 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.249442    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06e96eb1-d4a1-46fe-ba72-b29f35858ca4" containerName="payload-pre-processing"
Jul 28 01:43:29.249492 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.249462    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="06e96eb1-d4a1-46fe-ba72-b29f35858ca4" containerName="payload-pre-processing"
Jul 28 01:43:29.249492 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.249487    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="465b3bf5-e359-450d-a86b-ec597a6dd290" containerName="revoke-keys"
Jul 28 01:43:29.249675 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.249509    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="465b3bf5-e359-450d-a86b-ec597a6dd290" containerName="revoke-keys"
Jul 28 01:43:29.249675 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.249523    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9ebd816-f89c-4411-aeed-66544fbf7fc9" containerName="payload-processing"
Jul 28 01:43:29.249675 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.249534    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9ebd816-f89c-4411-aeed-66544fbf7fc9" containerName="payload-processing"
Jul 28 01:43:29.249675 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.249562    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="241e93eb-0aec-446a-8af3-ece1cb2665db" containerName="maas-api"
Jul 28 01:43:29.249675 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.249572    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="241e93eb-0aec-446a-8af3-ece1cb2665db" containerName="maas-api"
Jul 28 01:43:29.249920 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.249699    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9ebd816-f89c-4411-aeed-66544fbf7fc9" containerName="payload-processing"
Jul 28 01:43:29.249920 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.249713    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="06e96eb1-d4a1-46fe-ba72-b29f35858ca4" containerName="payload-pre-processing"
Jul 28 01:43:29.249920 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.249723    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="241e93eb-0aec-446a-8af3-ece1cb2665db" containerName="maas-api"
Jul 28 01:43:29.249920 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.249733    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="465b3bf5-e359-450d-a86b-ec597a6dd290" containerName="revoke-keys"
Jul 28 01:43:29.252796 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.252775    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb"
Jul 28 01:43:29.255707 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.255677    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-ec1b3fa4-serving-cert\""
Jul 28 01:43:29.255845 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.255694    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lrljm\""
Jul 28 01:43:29.255845 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.255750    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 01:43:29.255845 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.255763    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 01:43:29.263694 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.263627    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb"]
Jul 28 01:43:29.305989 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.305946    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f42c837a-aec6-4024-8bc6-929022d90a8a-maas-api-tls\") pod \"maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb\" (UID: \"f42c837a-aec6-4024-8bc6-929022d90a8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb"
Jul 28 01:43:29.306150 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.305998    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9hf6\" (UniqueName: \"kubernetes.io/projected/f42c837a-aec6-4024-8bc6-929022d90a8a-kube-api-access-h9hf6\") pod \"maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb\" (UID: \"f42c837a-aec6-4024-8bc6-929022d90a8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb"
Jul 28 01:43:29.306150 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.306080    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f42c837a-aec6-4024-8bc6-929022d90a8a-openshift-service-ca\") pod \"maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb\" (UID: \"f42c837a-aec6-4024-8bc6-929022d90a8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb"
Jul 28 01:43:29.407423 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.407384    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f42c837a-aec6-4024-8bc6-929022d90a8a-openshift-service-ca\") pod \"maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb\" (UID: \"f42c837a-aec6-4024-8bc6-929022d90a8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb"
Jul 28 01:43:29.407605 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.407481    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f42c837a-aec6-4024-8bc6-929022d90a8a-maas-api-tls\") pod \"maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb\" (UID: \"f42c837a-aec6-4024-8bc6-929022d90a8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb"
Jul 28 01:43:29.407605 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.407510    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9hf6\" (UniqueName: \"kubernetes.io/projected/f42c837a-aec6-4024-8bc6-929022d90a8a-kube-api-access-h9hf6\") pod \"maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb\" (UID: \"f42c837a-aec6-4024-8bc6-929022d90a8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb"
Jul 28 01:43:29.407688 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:43:29.407621    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-serving-cert: secret "maas-api-e2e-mt-ec1b3fa4-serving-cert" not found
Jul 28 01:43:29.407723 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:43:29.407694    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f42c837a-aec6-4024-8bc6-929022d90a8a-maas-api-tls podName:f42c837a-aec6-4024-8bc6-929022d90a8a nodeName:}" failed. No retries permitted until 2026-07-28 01:43:29.907678443 +0000 UTC m=+2980.072486528 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f42c837a-aec6-4024-8bc6-929022d90a8a-maas-api-tls") pod "maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb" (UID: "f42c837a-aec6-4024-8bc6-929022d90a8a") : secret "maas-api-e2e-mt-ec1b3fa4-serving-cert" not found
Jul 28 01:43:29.408092 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.408072    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f42c837a-aec6-4024-8bc6-929022d90a8a-openshift-service-ca\") pod \"maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb\" (UID: \"f42c837a-aec6-4024-8bc6-929022d90a8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb"
Jul 28 01:43:29.416347 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.416314    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9hf6\" (UniqueName: \"kubernetes.io/projected/f42c837a-aec6-4024-8bc6-929022d90a8a-kube-api-access-h9hf6\") pod \"maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb\" (UID: \"f42c837a-aec6-4024-8bc6-929022d90a8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb"
Jul 28 01:43:29.912231 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.912178    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f42c837a-aec6-4024-8bc6-929022d90a8a-maas-api-tls\") pod \"maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb\" (UID: \"f42c837a-aec6-4024-8bc6-929022d90a8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb"
Jul 28 01:43:29.914709 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.914678    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f42c837a-aec6-4024-8bc6-929022d90a8a-maas-api-tls\") pod \"maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb\" (UID: \"f42c837a-aec6-4024-8bc6-929022d90a8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb"
Jul 28 01:43:29.955005 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.954973    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb"]
Jul 28 01:43:29.958753 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.958734    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb"
Jul 28 01:43:29.961752 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.961735    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-ec1b3fa4-dockercfg-mt649\""
Jul 28 01:43:29.962638 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.962618    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-ec1b3fa4\""
Jul 28 01:43:29.971195 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:29.968752    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb"]
Jul 28 01:43:30.000191 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:30.000159    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg"]
Jul 28 01:43:30.003570 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:30.003553    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg"
Jul 28 01:43:30.012782 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:30.012758    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bf0962e-f32b-46bc-958f-12c3efce0460-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb\" (UID: \"2bf0962e-f32b-46bc-958f-12c3efce0460\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb"
Jul 28 01:43:30.012911 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:30.012877    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5njg\" (UniqueName: \"kubernetes.io/projected/2bf0962e-f32b-46bc-958f-12c3efce0460-kube-api-access-j5njg\") pod \"payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb\" (UID: \"2bf0962e-f32b-46bc-958f-12c3efce0460\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb"
Jul 28 01:43:30.019097 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:30.019041    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg"]
Jul 28 01:43:30.114145 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:30.114110    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5njg\" (UniqueName: \"kubernetes.io/projected/2bf0962e-f32b-46bc-958f-12c3efce0460-kube-api-access-j5njg\") pod \"payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb\" (UID: \"2bf0962e-f32b-46bc-958f-12c3efce0460\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb"
Jul 28 01:43:30.114145 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:30.114152    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmgqb\" (UniqueName: \"kubernetes.io/projected/90ea085d-beff-4801-a0f6-69514b6c2515-kube-api-access-jmgqb\") pod \"payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg\" (UID: \"90ea085d-beff-4801-a0f6-69514b6c2515\") " pod="openshift-ingress/payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg"
Jul 28 01:43:30.114393 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:30.114195    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bf0962e-f32b-46bc-958f-12c3efce0460-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb\" (UID: \"2bf0962e-f32b-46bc-958f-12c3efce0460\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb"
Jul 28 01:43:30.114393 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:30.114236    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90ea085d-beff-4801-a0f6-69514b6c2515-plugins-config-volume\") pod \"payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg\" (UID: \"90ea085d-beff-4801-a0f6-69514b6c2515\") " pod="openshift-ingress/payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg"
Jul 28 01:43:30.114887 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:30.114868    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bf0962e-f32b-46bc-958f-12c3efce0460-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb\" (UID: \"2bf0962e-f32b-46bc-958f-12c3efce0460\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb"
Jul 28 01:43:30.123143 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:30.123113    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5njg\" (UniqueName: \"kubernetes.io/projected/2bf0962e-f32b-46bc-958f-12c3efce0460-kube-api-access-j5njg\") pod \"payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb\" (UID: \"2bf0962e-f32b-46bc-958f-12c3efce0460\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb"
Jul 28 01:43:30.166240 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:30.166132    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb"
Jul 28 01:43:30.215607 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:30.215575    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmgqb\" (UniqueName: \"kubernetes.io/projected/90ea085d-beff-4801-a0f6-69514b6c2515-kube-api-access-jmgqb\") pod \"payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg\" (UID: \"90ea085d-beff-4801-a0f6-69514b6c2515\") " pod="openshift-ingress/payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg"
Jul 28 01:43:30.215779 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:30.215664    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90ea085d-beff-4801-a0f6-69514b6c2515-plugins-config-volume\") pod \"payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg\" (UID: \"90ea085d-beff-4801-a0f6-69514b6c2515\") " pod="openshift-ingress/payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg"
Jul 28 01:43:30.216628 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:30.216604    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90ea085d-beff-4801-a0f6-69514b6c2515-plugins-config-volume\") pod \"payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg\" (UID: \"90ea085d-beff-4801-a0f6-69514b6c2515\") " pod="openshift-ingress/payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg"
Jul 28 01:43:30.232564 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:30.232505    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmgqb\" (UniqueName: \"kubernetes.io/projected/90ea085d-beff-4801-a0f6-69514b6c2515-kube-api-access-jmgqb\") pod \"payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg\" (UID: \"90ea085d-beff-4801-a0f6-69514b6c2515\") " pod="openshift-ingress/payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg"
Jul 28 01:43:30.273031 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:30.272997    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb"
Jul 28 01:43:30.304492 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:30.304466    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb"]
Jul 28 01:43:30.306256 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:43:30.306196    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf42c837a_aec6_4024_8bc6_929022d90a8a.slice/crio-0d09e66985076d52c45a4f53ca1df661da7d7feb6c15c4900cf7257bbcbe9324 WatchSource:0}: Error finding container 0d09e66985076d52c45a4f53ca1df661da7d7feb6c15c4900cf7257bbcbe9324: Status 404 returned error can't find the container with id 0d09e66985076d52c45a4f53ca1df661da7d7feb6c15c4900cf7257bbcbe9324
Jul 28 01:43:30.314166 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:30.314142    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg"
Jul 28 01:43:30.456409 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:30.456297    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb"]
Jul 28 01:43:30.457249 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:43:30.457185    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bf0962e_f32b_46bc_958f_12c3efce0460.slice/crio-a29d6ec2ff72a8981886349cc1cd5042116c5b9478ddc39d78b44073bbe3f6f1 WatchSource:0}: Error finding container a29d6ec2ff72a8981886349cc1cd5042116c5b9478ddc39d78b44073bbe3f6f1: Status 404 returned error can't find the container with id a29d6ec2ff72a8981886349cc1cd5042116c5b9478ddc39d78b44073bbe3f6f1
Jul 28 01:43:30.501459 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:30.501391    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg"]
Jul 28 01:43:30.503696 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:43:30.503666    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90ea085d_beff_4801_a0f6_69514b6c2515.slice/crio-5be6fa6b5ac3f3c5ef9a8be444ab906ccf833427723f2bca6567dcc3918c0291 WatchSource:0}: Error finding container 5be6fa6b5ac3f3c5ef9a8be444ab906ccf833427723f2bca6567dcc3918c0291: Status 404 returned error can't find the container with id 5be6fa6b5ac3f3c5ef9a8be444ab906ccf833427723f2bca6567dcc3918c0291
Jul 28 01:43:30.515791 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:30.515762    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb" event={"ID":"f42c837a-aec6-4024-8bc6-929022d90a8a","Type":"ContainerStarted","Data":"0d09e66985076d52c45a4f53ca1df661da7d7feb6c15c4900cf7257bbcbe9324"}
Jul 28 01:43:30.517015 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:30.516993    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg" event={"ID":"90ea085d-beff-4801-a0f6-69514b6c2515","Type":"ContainerStarted","Data":"5be6fa6b5ac3f3c5ef9a8be444ab906ccf833427723f2bca6567dcc3918c0291"}
Jul 28 01:43:30.518204 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:30.518185    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb" event={"ID":"2bf0962e-f32b-46bc-958f-12c3efce0460","Type":"ContainerStarted","Data":"a29d6ec2ff72a8981886349cc1cd5042116c5b9478ddc39d78b44073bbe3f6f1"}
Jul 28 01:43:31.525104 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:31.525067    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb" event={"ID":"f42c837a-aec6-4024-8bc6-929022d90a8a","Type":"ContainerStarted","Data":"6f96bdd86b13e9075ddfd94755b21b31df146e7bd11e03d7869ef87ee9d4e69b"}
Jul 28 01:43:31.525612 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:31.525174    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb"
Jul 28 01:43:31.526686 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:31.526658    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg" event={"ID":"90ea085d-beff-4801-a0f6-69514b6c2515","Type":"ContainerStarted","Data":"e892726a8998ae87bcf08fe5b850cb267498ab11907edc56e16d9c852351da39"}
Jul 28 01:43:31.526783 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:31.526713    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg"
Jul 28 01:43:31.528006 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:31.527983    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb" event={"ID":"2bf0962e-f32b-46bc-958f-12c3efce0460","Type":"ContainerStarted","Data":"54aaa233270d2458aa4048e064b87cfad561e11730bba88bbe3e9ab22668d956"}
Jul 28 01:43:31.528105 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:31.528025    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb"
Jul 28 01:43:31.545339 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:31.545292    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb" podStartSLOduration=2.248372878 podStartE2EDuration="2.545277301s" totalImagesPullingTime="296.904423ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:43:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:43:31.543561853 +0000 UTC m=+2981.708369961" watchObservedRunningTime="2026-07-28 01:43:31.545277301 +0000 UTC m=+2981.710085407"
Jul 28 01:43:31.563614 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:31.563567    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb" podStartSLOduration=2.563555098 podStartE2EDuration="2.563555098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:43:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:43:31.561038786 +0000 UTC m=+2981.725846894" watchObservedRunningTime="2026-07-28 01:43:31.563555098 +0000 UTC m=+2981.728363205"
Jul 28 01:43:31.580070 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:31.580026    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg" podStartSLOduration=2.580012445 podStartE2EDuration="2.580012445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:43:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:43:31.577022365 +0000 UTC m=+2981.741830473" watchObservedRunningTime="2026-07-28 01:43:31.580012445 +0000 UTC m=+2981.744820552"
Jul 28 01:43:37.539998 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:37.539967    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb"
Jul 28 01:43:42.535813 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:42.535786    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg"
Jul 28 01:43:42.536301 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:42.535852    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb"
Jul 28 01:43:45.447946 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:45.447909    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"]
Jul 28 01:43:45.453610 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:45.453584    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"
Jul 28 01:43:45.457957 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:45.457925    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ec1b3fa4\"/\"default-dockercfg-gr6gp\""
Jul 28 01:43:45.458392 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:45.458364    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ec1b3fa4\"/\"e2e-lifecycle-model-ec1b3fa4-kserve-self-signed-certs\""
Jul 28 01:43:45.458716 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:45.458696    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ec1b3fa4\"/\"openshift-service-ca.crt\""
Jul 28 01:43:45.458944 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:45.458907    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ec1b3fa4\"/\"kube-root-ca.crt\""
Jul 28 01:43:45.461621 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:45.461594    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"]
Jul 28 01:43:45.572348 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:45.572313    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e32f26a-b6e1-449c-87fd-d15a2769f298-home\") pod \"e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r\" (UID: \"3e32f26a-b6e1-449c-87fd-d15a2769f298\") " pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"
Jul 28 01:43:45.572577 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:45.572377    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e32f26a-b6e1-449c-87fd-d15a2769f298-model-cache\") pod \"e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r\" (UID: \"3e32f26a-b6e1-449c-87fd-d15a2769f298\") " pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"
Jul 28 01:43:45.572577 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:45.572487    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e32f26a-b6e1-449c-87fd-d15a2769f298-tls-certs\") pod \"e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r\" (UID: \"3e32f26a-b6e1-449c-87fd-d15a2769f298\") " pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"
Jul 28 01:43:45.572577 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:45.572534    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmk5k\" (UniqueName: \"kubernetes.io/projected/3e32f26a-b6e1-449c-87fd-d15a2769f298-kube-api-access-nmk5k\") pod \"e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r\" (UID: \"3e32f26a-b6e1-449c-87fd-d15a2769f298\") " pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"
Jul 28 01:43:45.572729 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:45.572606    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e32f26a-b6e1-449c-87fd-d15a2769f298-tmp-dir\") pod \"e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r\" (UID: \"3e32f26a-b6e1-449c-87fd-d15a2769f298\") " pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"
Jul 28 01:43:45.572729 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:45.572679    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e32f26a-b6e1-449c-87fd-d15a2769f298-dshm\") pod \"e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r\" (UID: \"3e32f26a-b6e1-449c-87fd-d15a2769f298\") " pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"
Jul 28 01:43:45.674084 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:45.674046    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e32f26a-b6e1-449c-87fd-d15a2769f298-dshm\") pod \"e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r\" (UID: \"3e32f26a-b6e1-449c-87fd-d15a2769f298\") " pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"
Jul 28 01:43:45.674084 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:45.674086    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e32f26a-b6e1-449c-87fd-d15a2769f298-home\") pod \"e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r\" (UID: \"3e32f26a-b6e1-449c-87fd-d15a2769f298\") " pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"
Jul 28 01:43:45.674374 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:45.674252    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e32f26a-b6e1-449c-87fd-d15a2769f298-model-cache\") pod \"e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r\" (UID: \"3e32f26a-b6e1-449c-87fd-d15a2769f298\") " pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"
Jul 28 01:43:45.674374 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:45.674311    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e32f26a-b6e1-449c-87fd-d15a2769f298-tls-certs\") pod \"e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r\" (UID: \"3e32f26a-b6e1-449c-87fd-d15a2769f298\") " pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"
Jul 28 01:43:45.674374 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:45.674342    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmk5k\" (UniqueName: \"kubernetes.io/projected/3e32f26a-b6e1-449c-87fd-d15a2769f298-kube-api-access-nmk5k\") pod \"e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r\" (UID: \"3e32f26a-b6e1-449c-87fd-d15a2769f298\") " pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"
Jul 28 01:43:45.674546 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:45.674380    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e32f26a-b6e1-449c-87fd-d15a2769f298-tmp-dir\") pod \"e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r\" (UID: \"3e32f26a-b6e1-449c-87fd-d15a2769f298\") " pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"
Jul 28 01:43:45.674546 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:45.674521    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e32f26a-b6e1-449c-87fd-d15a2769f298-home\") pod \"e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r\" (UID: \"3e32f26a-b6e1-449c-87fd-d15a2769f298\") " pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"
Jul 28 01:43:45.674667 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:45.674583    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e32f26a-b6e1-449c-87fd-d15a2769f298-model-cache\") pod \"e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r\" (UID: \"3e32f26a-b6e1-449c-87fd-d15a2769f298\") " pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"
Jul 28 01:43:45.674729 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:45.674707    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e32f26a-b6e1-449c-87fd-d15a2769f298-tmp-dir\") pod \"e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r\" (UID: \"3e32f26a-b6e1-449c-87fd-d15a2769f298\") " pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"
Jul 28 01:43:45.676572 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:45.676543    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e32f26a-b6e1-449c-87fd-d15a2769f298-dshm\") pod \"e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r\" (UID: \"3e32f26a-b6e1-449c-87fd-d15a2769f298\") " pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"
Jul 28 01:43:45.676833 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:45.676815    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e32f26a-b6e1-449c-87fd-d15a2769f298-tls-certs\") pod \"e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r\" (UID: \"3e32f26a-b6e1-449c-87fd-d15a2769f298\") " pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"
Jul 28 01:43:45.682474 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:45.682450    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmk5k\" (UniqueName: \"kubernetes.io/projected/3e32f26a-b6e1-449c-87fd-d15a2769f298-kube-api-access-nmk5k\") pod \"e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r\" (UID: \"3e32f26a-b6e1-449c-87fd-d15a2769f298\") " pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"
Jul 28 01:43:45.766791 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:45.766700    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"
Jul 28 01:43:45.907032 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:45.907000    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"]
Jul 28 01:43:45.907624 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:43:45.907594    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e32f26a_b6e1_449c_87fd_d15a2769f298.slice/crio-8e4e75f21d95a2393d22945e69101da4673b35ceb378126ace129e5800aed9f4 WatchSource:0}: Error finding container 8e4e75f21d95a2393d22945e69101da4673b35ceb378126ace129e5800aed9f4: Status 404 returned error can't find the container with id 8e4e75f21d95a2393d22945e69101da4673b35ceb378126ace129e5800aed9f4
Jul 28 01:43:46.622678 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:46.622639    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r" event={"ID":"3e32f26a-b6e1-449c-87fd-d15a2769f298","Type":"ContainerStarted","Data":"db4f871c48a8223625327af4f34b5a6296f4fd4fe05a1deab9293b4429ee3ebd"}
Jul 28 01:43:46.623063 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:46.622686    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r" event={"ID":"3e32f26a-b6e1-449c-87fd-d15a2769f298","Type":"ContainerStarted","Data":"8e4e75f21d95a2393d22945e69101da4673b35ceb378126ace129e5800aed9f4"}
Jul 28 01:43:46.644500 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:46.644440    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r" podStartSLOduration=1.644423358 podStartE2EDuration="1.644423358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:43:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:43:46.641308172 +0000 UTC m=+2996.806116280" watchObservedRunningTime="2026-07-28 01:43:46.644423358 +0000 UTC m=+2996.809231465"
Jul 28 01:43:55.767354 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:55.767312    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"
Jul 28 01:43:55.767813 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:55.767400    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"
Jul 28 01:43:55.779986 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:55.779952    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"
Jul 28 01:43:56.693077 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:43:56.693047    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"
Jul 28 01:44:28.604400 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:28.604369    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb"]
Jul 28 01:44:28.605009 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:28.604583    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb" podUID="f42c837a-aec6-4024-8bc6-929022d90a8a" containerName="maas-api" containerID="cri-o://6f96bdd86b13e9075ddfd94755b21b31df146e7bd11e03d7869ef87ee9d4e69b" gracePeriod=30
Jul 28 01:44:28.661578 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:28.661542    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg"]
Jul 28 01:44:28.662047 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:28.662015    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg" podUID="90ea085d-beff-4801-a0f6-69514b6c2515" containerName="payload-processing" containerID="cri-o://e892726a8998ae87bcf08fe5b850cb267498ab11907edc56e16d9c852351da39" gracePeriod=30
Jul 28 01:44:28.684306 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:28.684271    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb"]
Jul 28 01:44:28.685114 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:28.685078    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb" podUID="2bf0962e-f32b-46bc-958f-12c3efce0460" containerName="payload-pre-processing" containerID="cri-o://54aaa233270d2458aa4048e064b87cfad561e11730bba88bbe3e9ab22668d956" gracePeriod=30
Jul 28 01:44:28.862818 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:28.862673    2578 generic.go:356] "Generic (PLEG): container finished" podID="f42c837a-aec6-4024-8bc6-929022d90a8a" containerID="6f96bdd86b13e9075ddfd94755b21b31df146e7bd11e03d7869ef87ee9d4e69b" exitCode=0
Jul 28 01:44:28.862818 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:28.862789    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb" event={"ID":"f42c837a-aec6-4024-8bc6-929022d90a8a","Type":"ContainerDied","Data":"6f96bdd86b13e9075ddfd94755b21b31df146e7bd11e03d7869ef87ee9d4e69b"}
Jul 28 01:44:28.865542 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:28.865369    2578 generic.go:356] "Generic (PLEG): container finished" podID="90ea085d-beff-4801-a0f6-69514b6c2515" containerID="e892726a8998ae87bcf08fe5b850cb267498ab11907edc56e16d9c852351da39" exitCode=0
Jul 28 01:44:28.865542 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:28.865469    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg" event={"ID":"90ea085d-beff-4801-a0f6-69514b6c2515","Type":"ContainerDied","Data":"e892726a8998ae87bcf08fe5b850cb267498ab11907edc56e16d9c852351da39"}
Jul 28 01:44:28.867395 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:28.867351    2578 generic.go:356] "Generic (PLEG): container finished" podID="2bf0962e-f32b-46bc-958f-12c3efce0460" containerID="54aaa233270d2458aa4048e064b87cfad561e11730bba88bbe3e9ab22668d956" exitCode=0
Jul 28 01:44:28.867554 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:28.867437    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb" event={"ID":"2bf0962e-f32b-46bc-958f-12c3efce0460","Type":"ContainerDied","Data":"54aaa233270d2458aa4048e064b87cfad561e11730bba88bbe3e9ab22668d956"}
Jul 28 01:44:28.954681 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:28.954645    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb"
Jul 28 01:44:28.986525 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:28.986499    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb"
Jul 28 01:44:28.989827 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:28.989804    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg"
Jul 28 01:44:29.005441 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.005408    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f42c837a-aec6-4024-8bc6-929022d90a8a-openshift-service-ca\") pod \"f42c837a-aec6-4024-8bc6-929022d90a8a\" (UID: \"f42c837a-aec6-4024-8bc6-929022d90a8a\") "
Jul 28 01:44:29.005647 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.005471    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90ea085d-beff-4801-a0f6-69514b6c2515-plugins-config-volume\") pod \"90ea085d-beff-4801-a0f6-69514b6c2515\" (UID: \"90ea085d-beff-4801-a0f6-69514b6c2515\") "
Jul 28 01:44:29.005647 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.005507    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9hf6\" (UniqueName: \"kubernetes.io/projected/f42c837a-aec6-4024-8bc6-929022d90a8a-kube-api-access-h9hf6\") pod \"f42c837a-aec6-4024-8bc6-929022d90a8a\" (UID: \"f42c837a-aec6-4024-8bc6-929022d90a8a\") "
Jul 28 01:44:29.005647 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.005533    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j5njg\" (UniqueName: \"kubernetes.io/projected/2bf0962e-f32b-46bc-958f-12c3efce0460-kube-api-access-j5njg\") pod \"2bf0962e-f32b-46bc-958f-12c3efce0460\" (UID: \"2bf0962e-f32b-46bc-958f-12c3efce0460\") "
Jul 28 01:44:29.005647 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.005577    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bf0962e-f32b-46bc-958f-12c3efce0460-plugins-config-volume\") pod \"2bf0962e-f32b-46bc-958f-12c3efce0460\" (UID: \"2bf0962e-f32b-46bc-958f-12c3efce0460\") "
Jul 28 01:44:29.005647 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.005612    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f42c837a-aec6-4024-8bc6-929022d90a8a-maas-api-tls\") pod \"f42c837a-aec6-4024-8bc6-929022d90a8a\" (UID: \"f42c837a-aec6-4024-8bc6-929022d90a8a\") "
Jul 28 01:44:29.005943 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.005668    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmgqb\" (UniqueName: \"kubernetes.io/projected/90ea085d-beff-4801-a0f6-69514b6c2515-kube-api-access-jmgqb\") pod \"90ea085d-beff-4801-a0f6-69514b6c2515\" (UID: \"90ea085d-beff-4801-a0f6-69514b6c2515\") "
Jul 28 01:44:29.006428 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.006385    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f42c837a-aec6-4024-8bc6-929022d90a8a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f42c837a-aec6-4024-8bc6-929022d90a8a" (UID: "f42c837a-aec6-4024-8bc6-929022d90a8a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:44:29.006545 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.006421    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bf0962e-f32b-46bc-958f-12c3efce0460-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2bf0962e-f32b-46bc-958f-12c3efce0460" (UID: "2bf0962e-f32b-46bc-958f-12c3efce0460"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:44:29.007521 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.006850    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90ea085d-beff-4801-a0f6-69514b6c2515-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "90ea085d-beff-4801-a0f6-69514b6c2515" (UID: "90ea085d-beff-4801-a0f6-69514b6c2515"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:44:29.008834 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.008808    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bf0962e-f32b-46bc-958f-12c3efce0460-kube-api-access-j5njg" (OuterVolumeSpecName: "kube-api-access-j5njg") pod "2bf0962e-f32b-46bc-958f-12c3efce0460" (UID: "2bf0962e-f32b-46bc-958f-12c3efce0460"). InnerVolumeSpecName "kube-api-access-j5njg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:44:29.008991 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.008975    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90ea085d-beff-4801-a0f6-69514b6c2515-kube-api-access-jmgqb" (OuterVolumeSpecName: "kube-api-access-jmgqb") pod "90ea085d-beff-4801-a0f6-69514b6c2515" (UID: "90ea085d-beff-4801-a0f6-69514b6c2515"). InnerVolumeSpecName "kube-api-access-jmgqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:44:29.009180 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.009155    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f42c837a-aec6-4024-8bc6-929022d90a8a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f42c837a-aec6-4024-8bc6-929022d90a8a" (UID: "f42c837a-aec6-4024-8bc6-929022d90a8a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:44:29.009754 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.009727    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f42c837a-aec6-4024-8bc6-929022d90a8a-kube-api-access-h9hf6" (OuterVolumeSpecName: "kube-api-access-h9hf6") pod "f42c837a-aec6-4024-8bc6-929022d90a8a" (UID: "f42c837a-aec6-4024-8bc6-929022d90a8a"). InnerVolumeSpecName "kube-api-access-h9hf6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:44:29.106865 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.106827    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f42c837a-aec6-4024-8bc6-929022d90a8a-openshift-service-ca\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:44:29.106865 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.106856    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90ea085d-beff-4801-a0f6-69514b6c2515-plugins-config-volume\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:44:29.106865 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.106865    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9hf6\" (UniqueName: \"kubernetes.io/projected/f42c837a-aec6-4024-8bc6-929022d90a8a-kube-api-access-h9hf6\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:44:29.107101 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.106897    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5njg\" (UniqueName: \"kubernetes.io/projected/2bf0962e-f32b-46bc-958f-12c3efce0460-kube-api-access-j5njg\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:44:29.107101 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.106906    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bf0962e-f32b-46bc-958f-12c3efce0460-plugins-config-volume\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:44:29.107101 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.106915    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f42c837a-aec6-4024-8bc6-929022d90a8a-maas-api-tls\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:44:29.107101 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.106924    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmgqb\" (UniqueName: \"kubernetes.io/projected/90ea085d-beff-4801-a0f6-69514b6c2515-kube-api-access-jmgqb\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:44:29.874498 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.874412    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb"
Jul 28 01:44:29.874498 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.874443    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb" event={"ID":"f42c837a-aec6-4024-8bc6-929022d90a8a","Type":"ContainerDied","Data":"0d09e66985076d52c45a4f53ca1df661da7d7feb6c15c4900cf7257bbcbe9324"}
Jul 28 01:44:29.874986 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.874514    2578 scope.go:117] "RemoveContainer" containerID="6f96bdd86b13e9075ddfd94755b21b31df146e7bd11e03d7869ef87ee9d4e69b"
Jul 28 01:44:29.875987 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.875962    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg"
Jul 28 01:44:29.876124 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.876026    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg" event={"ID":"90ea085d-beff-4801-a0f6-69514b6c2515","Type":"ContainerDied","Data":"5be6fa6b5ac3f3c5ef9a8be444ab906ccf833427723f2bca6567dcc3918c0291"}
Jul 28 01:44:29.877701 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.877684    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb"
Jul 28 01:44:29.877833 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.877713    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb" event={"ID":"2bf0962e-f32b-46bc-958f-12c3efce0460","Type":"ContainerDied","Data":"a29d6ec2ff72a8981886349cc1cd5042116c5b9478ddc39d78b44073bbe3f6f1"}
Jul 28 01:44:29.888824 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.888802    2578 scope.go:117] "RemoveContainer" containerID="e892726a8998ae87bcf08fe5b850cb267498ab11907edc56e16d9c852351da39"
Jul 28 01:44:29.898954 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.898909    2578 scope.go:117] "RemoveContainer" containerID="54aaa233270d2458aa4048e064b87cfad561e11730bba88bbe3e9ab22668d956"
Jul 28 01:44:29.900489 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.900396    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb"]
Jul 28 01:44:29.903284 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.903263    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ec1b3fa4-77c9c75fc9-4pfqb"]
Jul 28 01:44:29.912738 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.912714    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg"]
Jul 28 01:44:29.916658 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.916633    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ec1b3fa4-67f58b667d-xqpzg"]
Jul 28 01:44:29.927254 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.927204    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb"]
Jul 28 01:44:29.930728 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:29.930706    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ec1b3fa4-5d76895856-m27qb"]
Jul 28 01:44:30.403001 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:30.402971    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bf0962e-f32b-46bc-958f-12c3efce0460" path="/var/lib/kubelet/pods/2bf0962e-f32b-46bc-958f-12c3efce0460/volumes"
Jul 28 01:44:30.403374 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:30.403360    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90ea085d-beff-4801-a0f6-69514b6c2515" path="/var/lib/kubelet/pods/90ea085d-beff-4801-a0f6-69514b6c2515/volumes"
Jul 28 01:44:30.403732 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:30.403718    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f42c837a-aec6-4024-8bc6-929022d90a8a" path="/var/lib/kubelet/pods/f42c837a-aec6-4024-8bc6-929022d90a8a/volumes"
Jul 28 01:44:41.504666 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:41.504628    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"]
Jul 28 01:44:56.523897 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:56.523837    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r" podUID="3e32f26a-b6e1-449c-87fd-d15a2769f298" containerName="main" containerID="cri-o://db4f871c48a8223625327af4f34b5a6296f4fd4fe05a1deab9293b4429ee3ebd" gracePeriod=45
Jul 28 01:44:56.682340 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:56.682291    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r" podUID="3e32f26a-b6e1-449c-87fd-d15a2769f298" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": dial tcp 10.134.0.60:8000: connect: connection refused"
Jul 28 01:44:56.774020 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:56.773956    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"
Jul 28 01:44:56.880473 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:56.880442    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e32f26a-b6e1-449c-87fd-d15a2769f298-home\") pod \"3e32f26a-b6e1-449c-87fd-d15a2769f298\" (UID: \"3e32f26a-b6e1-449c-87fd-d15a2769f298\") "
Jul 28 01:44:56.880473 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:56.880479    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmk5k\" (UniqueName: \"kubernetes.io/projected/3e32f26a-b6e1-449c-87fd-d15a2769f298-kube-api-access-nmk5k\") pod \"3e32f26a-b6e1-449c-87fd-d15a2769f298\" (UID: \"3e32f26a-b6e1-449c-87fd-d15a2769f298\") "
Jul 28 01:44:56.880692 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:56.880564    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e32f26a-b6e1-449c-87fd-d15a2769f298-tls-certs\") pod \"3e32f26a-b6e1-449c-87fd-d15a2769f298\" (UID: \"3e32f26a-b6e1-449c-87fd-d15a2769f298\") "
Jul 28 01:44:56.880692 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:56.880613    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e32f26a-b6e1-449c-87fd-d15a2769f298-model-cache\") pod \"3e32f26a-b6e1-449c-87fd-d15a2769f298\" (UID: \"3e32f26a-b6e1-449c-87fd-d15a2769f298\") "
Jul 28 01:44:56.880692 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:56.880644    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e32f26a-b6e1-449c-87fd-d15a2769f298-tmp-dir\") pod \"3e32f26a-b6e1-449c-87fd-d15a2769f298\" (UID: \"3e32f26a-b6e1-449c-87fd-d15a2769f298\") "
Jul 28 01:44:56.880852 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:56.880708    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e32f26a-b6e1-449c-87fd-d15a2769f298-home" (OuterVolumeSpecName: "home") pod "3e32f26a-b6e1-449c-87fd-d15a2769f298" (UID: "3e32f26a-b6e1-449c-87fd-d15a2769f298"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:44:56.880852 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:56.880733    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e32f26a-b6e1-449c-87fd-d15a2769f298-dshm\") pod \"3e32f26a-b6e1-449c-87fd-d15a2769f298\" (UID: \"3e32f26a-b6e1-449c-87fd-d15a2769f298\") "
Jul 28 01:44:56.880954 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:56.880913    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e32f26a-b6e1-449c-87fd-d15a2769f298-model-cache" (OuterVolumeSpecName: "model-cache") pod "3e32f26a-b6e1-449c-87fd-d15a2769f298" (UID: "3e32f26a-b6e1-449c-87fd-d15a2769f298"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:44:56.880954 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:56.880925    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e32f26a-b6e1-449c-87fd-d15a2769f298-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3e32f26a-b6e1-449c-87fd-d15a2769f298" (UID: "3e32f26a-b6e1-449c-87fd-d15a2769f298"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:44:56.881095 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:56.881072    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e32f26a-b6e1-449c-87fd-d15a2769f298-model-cache\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:44:56.881095 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:56.881098    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e32f26a-b6e1-449c-87fd-d15a2769f298-tmp-dir\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:44:56.881267 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:56.881111    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e32f26a-b6e1-449c-87fd-d15a2769f298-home\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:44:56.882869 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:56.882846    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e32f26a-b6e1-449c-87fd-d15a2769f298-dshm" (OuterVolumeSpecName: "dshm") pod "3e32f26a-b6e1-449c-87fd-d15a2769f298" (UID: "3e32f26a-b6e1-449c-87fd-d15a2769f298"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:44:56.882869 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:56.882852    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e32f26a-b6e1-449c-87fd-d15a2769f298-kube-api-access-nmk5k" (OuterVolumeSpecName: "kube-api-access-nmk5k") pod "3e32f26a-b6e1-449c-87fd-d15a2769f298" (UID: "3e32f26a-b6e1-449c-87fd-d15a2769f298"). InnerVolumeSpecName "kube-api-access-nmk5k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:44:56.883035 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:56.882883    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e32f26a-b6e1-449c-87fd-d15a2769f298-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3e32f26a-b6e1-449c-87fd-d15a2769f298" (UID: "3e32f26a-b6e1-449c-87fd-d15a2769f298"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:44:56.982532 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:56.982496    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e32f26a-b6e1-449c-87fd-d15a2769f298-dshm\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:44:56.982532 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:56.982523    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmk5k\" (UniqueName: \"kubernetes.io/projected/3e32f26a-b6e1-449c-87fd-d15a2769f298-kube-api-access-nmk5k\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:44:56.982532 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:56.982533    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e32f26a-b6e1-449c-87fd-d15a2769f298-tls-certs\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:44:57.019205 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:57.019168    2578 generic.go:356] "Generic (PLEG): container finished" podID="3e32f26a-b6e1-449c-87fd-d15a2769f298" containerID="db4f871c48a8223625327af4f34b5a6296f4fd4fe05a1deab9293b4429ee3ebd" exitCode=0
Jul 28 01:44:57.019205 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:57.019207    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"
Jul 28 01:44:57.019407 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:57.019244    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r" event={"ID":"3e32f26a-b6e1-449c-87fd-d15a2769f298","Type":"ContainerDied","Data":"db4f871c48a8223625327af4f34b5a6296f4fd4fe05a1deab9293b4429ee3ebd"}
Jul 28 01:44:57.019407 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:57.019287    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r" event={"ID":"3e32f26a-b6e1-449c-87fd-d15a2769f298","Type":"ContainerDied","Data":"8e4e75f21d95a2393d22945e69101da4673b35ceb378126ace129e5800aed9f4"}
Jul 28 01:44:57.019407 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:57.019305    2578 scope.go:117] "RemoveContainer" containerID="db4f871c48a8223625327af4f34b5a6296f4fd4fe05a1deab9293b4429ee3ebd"
Jul 28 01:44:57.028462 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:57.028449    2578 scope.go:117] "RemoveContainer" containerID="db4f871c48a8223625327af4f34b5a6296f4fd4fe05a1deab9293b4429ee3ebd"
Jul 28 01:44:57.028709 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:44:57.028691    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db4f871c48a8223625327af4f34b5a6296f4fd4fe05a1deab9293b4429ee3ebd\": container with ID starting with db4f871c48a8223625327af4f34b5a6296f4fd4fe05a1deab9293b4429ee3ebd not found: ID does not exist" containerID="db4f871c48a8223625327af4f34b5a6296f4fd4fe05a1deab9293b4429ee3ebd"
Jul 28 01:44:57.028774 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:57.028717    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db4f871c48a8223625327af4f34b5a6296f4fd4fe05a1deab9293b4429ee3ebd"} err="failed to get container status \"db4f871c48a8223625327af4f34b5a6296f4fd4fe05a1deab9293b4429ee3ebd\": rpc error: code = NotFound desc = could not find container \"db4f871c48a8223625327af4f34b5a6296f4fd4fe05a1deab9293b4429ee3ebd\": container with ID starting with db4f871c48a8223625327af4f34b5a6296f4fd4fe05a1deab9293b4429ee3ebd not found: ID does not exist"
Jul 28 01:44:57.043452 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:57.043430    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"]
Jul 28 01:44:57.049857 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:57.049832    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-ec1b3fa4/e2e-lifecycle-model-ec1b3fa4-kserve-548bfd4fff-qk97r"]
Jul 28 01:44:58.402735 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:44:58.402704    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e32f26a-b6e1-449c-87fd-d15a2769f298" path="/var/lib/kubelet/pods/3e32f26a-b6e1-449c-87fd-d15a2769f298/volumes"
Jul 28 01:45:00.202497 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:45:00.199547    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753385-h76mj"]
Jul 28 01:45:00.202497 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:45:00.200379    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bf0962e-f32b-46bc-958f-12c3efce0460" containerName="payload-pre-processing"
Jul 28 01:45:00.202497 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:45:00.200409    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bf0962e-f32b-46bc-958f-12c3efce0460" containerName="payload-pre-processing"
Jul 28 01:45:00.202497 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:45:00.200448    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90ea085d-beff-4801-a0f6-69514b6c2515" containerName="payload-processing"
Jul 28 01:45:00.202497 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:45:00.200456    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="90ea085d-beff-4801-a0f6-69514b6c2515" containerName="payload-processing"
Jul 28 01:45:00.202497 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:45:00.200502    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f42c837a-aec6-4024-8bc6-929022d90a8a" containerName="maas-api"
Jul 28 01:45:00.202497 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:45:00.200511    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f42c837a-aec6-4024-8bc6-929022d90a8a" containerName="maas-api"
Jul 28 01:45:00.202497 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:45:00.200530    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e32f26a-b6e1-449c-87fd-d15a2769f298" containerName="main"
Jul 28 01:45:00.202497 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:45:00.200537    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e32f26a-b6e1-449c-87fd-d15a2769f298" containerName="main"
Jul 28 01:45:00.202497 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:45:00.200696    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f42c837a-aec6-4024-8bc6-929022d90a8a" containerName="maas-api"
Jul 28 01:45:00.202497 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:45:00.200713    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e32f26a-b6e1-449c-87fd-d15a2769f298" containerName="main"
Jul 28 01:45:00.202497 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:45:00.200728    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="90ea085d-beff-4801-a0f6-69514b6c2515" containerName="payload-processing"
Jul 28 01:45:00.202497 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:45:00.200739    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bf0962e-f32b-46bc-958f-12c3efce0460" containerName="payload-pre-processing"
Jul 28 01:45:00.204758 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:45:00.204732    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753385-h76mj"
Jul 28 01:45:00.207800 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:45:00.207776    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8ph8w\""
Jul 28 01:45:00.215942 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:45:00.215916    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753385-h76mj"
Jul 28 01:45:00.217049 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:45:00.217028    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753385-h76mj"]
Jul 28 01:45:00.357811 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:45:00.357784    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753385-h76mj"]
Jul 28 01:45:00.359530 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:45:00.359507    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda11c4d37_aa38_4789_90f2_2978495cfecd.slice/crio-8648f1c9a2807681c7a6d204280d266b3310b2be7b2513e414642dc60891bfd8 WatchSource:0}: Error finding container 8648f1c9a2807681c7a6d204280d266b3310b2be7b2513e414642dc60891bfd8: Status 404 returned error can't find the container with id 8648f1c9a2807681c7a6d204280d266b3310b2be7b2513e414642dc60891bfd8
Jul 28 01:45:01.042282 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:45:01.042252    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753385-h76mj" event={"ID":"a11c4d37-aa38-4789-90f2-2978495cfecd","Type":"ContainerStarted","Data":"4831c151f46d765d10aaba3a6f913feead180aba092a997dfc2494805e67eb9a"}
Jul 28 01:45:01.042513 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:45:01.042301    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753385-h76mj" event={"ID":"a11c4d37-aa38-4789-90f2-2978495cfecd","Type":"ContainerStarted","Data":"8648f1c9a2807681c7a6d204280d266b3310b2be7b2513e414642dc60891bfd8"}
Jul 28 01:45:01.063226 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:45:01.063155    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753385-h76mj" podStartSLOduration=1.063136332 podStartE2EDuration="1.063136332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:45:01.061374025 +0000 UTC m=+3071.226182132" watchObservedRunningTime="2026-07-28 01:45:01.063136332 +0000 UTC m=+3071.227944460"
Jul 28 01:45:02.048222 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:45:02.048169    2578 generic.go:356] "Generic (PLEG): container finished" podID="a11c4d37-aa38-4789-90f2-2978495cfecd" containerID="4831c151f46d765d10aaba3a6f913feead180aba092a997dfc2494805e67eb9a" exitCode=0
Jul 28 01:45:02.048607 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:45:02.048246    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753385-h76mj" event={"ID":"a11c4d37-aa38-4789-90f2-2978495cfecd","Type":"ContainerDied","Data":"4831c151f46d765d10aaba3a6f913feead180aba092a997dfc2494805e67eb9a"}
Jul 28 01:45:03.183745 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:45:03.183722    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753385-h76mj"
Jul 28 01:45:04.060224 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:45:04.060180    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753385-h76mj"
Jul 28 01:45:04.060408 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:45:04.060242    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753385-h76mj" event={"ID":"a11c4d37-aa38-4789-90f2-2978495cfecd","Type":"ContainerDied","Data":"8648f1c9a2807681c7a6d204280d266b3310b2be7b2513e414642dc60891bfd8"}
Jul 28 01:45:04.060408 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:45:04.060296    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8648f1c9a2807681c7a6d204280d266b3310b2be7b2513e414642dc60891bfd8"
Jul 28 01:46:23.152614 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:23.152530    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n"]
Jul 28 01:46:23.153124 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:23.153107    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a11c4d37-aa38-4789-90f2-2978495cfecd" containerName="cleanup"
Jul 28 01:46:23.153181 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:23.153128    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a11c4d37-aa38-4789-90f2-2978495cfecd" containerName="cleanup"
Jul 28 01:46:23.153287 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:23.153274    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a11c4d37-aa38-4789-90f2-2978495cfecd" containerName="cleanup"
Jul 28 01:46:23.156737 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:23.156717    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n"
Jul 28 01:46:23.159674 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:23.159649    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 01:46:23.159869 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:23.159725    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-2cd0f2-serving-cert\""
Jul 28 01:46:23.160725 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:23.160704    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lrljm\""
Jul 28 01:46:23.160832 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:23.160726    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 01:46:23.172832 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:23.172803    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n"]
Jul 28 01:46:23.220614 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:23.220572    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57e57ec2-65c8-4073-a229-74741e1a1030-maas-api-tls\") pod \"maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n\" (UID: \"57e57ec2-65c8-4073-a229-74741e1a1030\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n"
Jul 28 01:46:23.220799 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:23.220622    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57e57ec2-65c8-4073-a229-74741e1a1030-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n\" (UID: \"57e57ec2-65c8-4073-a229-74741e1a1030\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n"
Jul 28 01:46:23.220799 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:23.220675    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8xdr\" (UniqueName: \"kubernetes.io/projected/57e57ec2-65c8-4073-a229-74741e1a1030-kube-api-access-v8xdr\") pod \"maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n\" (UID: \"57e57ec2-65c8-4073-a229-74741e1a1030\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n"
Jul 28 01:46:23.321701 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:23.321659    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57e57ec2-65c8-4073-a229-74741e1a1030-maas-api-tls\") pod \"maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n\" (UID: \"57e57ec2-65c8-4073-a229-74741e1a1030\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n"
Jul 28 01:46:23.321701 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:23.321705    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57e57ec2-65c8-4073-a229-74741e1a1030-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n\" (UID: \"57e57ec2-65c8-4073-a229-74741e1a1030\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n"
Jul 28 01:46:23.321921 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:23.321796    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8xdr\" (UniqueName: \"kubernetes.io/projected/57e57ec2-65c8-4073-a229-74741e1a1030-kube-api-access-v8xdr\") pod \"maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n\" (UID: \"57e57ec2-65c8-4073-a229-74741e1a1030\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n"
Jul 28 01:46:23.321921 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:46:23.321821    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-serving-cert: secret "maas-api-e2e-inf-a-2cd0f2-serving-cert" not found
Jul 28 01:46:23.321921 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:46:23.321910    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57e57ec2-65c8-4073-a229-74741e1a1030-maas-api-tls podName:57e57ec2-65c8-4073-a229-74741e1a1030 nodeName:}" failed. No retries permitted until 2026-07-28 01:46:23.821886931 +0000 UTC m=+3153.986695037 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/57e57ec2-65c8-4073-a229-74741e1a1030-maas-api-tls") pod "maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n" (UID: "57e57ec2-65c8-4073-a229-74741e1a1030") : secret "maas-api-e2e-inf-a-2cd0f2-serving-cert" not found
Jul 28 01:46:23.322504 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:23.322485    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57e57ec2-65c8-4073-a229-74741e1a1030-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n\" (UID: \"57e57ec2-65c8-4073-a229-74741e1a1030\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n"
Jul 28 01:46:23.333885 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:23.333855    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8xdr\" (UniqueName: \"kubernetes.io/projected/57e57ec2-65c8-4073-a229-74741e1a1030-kube-api-access-v8xdr\") pod \"maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n\" (UID: \"57e57ec2-65c8-4073-a229-74741e1a1030\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n"
Jul 28 01:46:23.826526 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:23.826486    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57e57ec2-65c8-4073-a229-74741e1a1030-maas-api-tls\") pod \"maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n\" (UID: \"57e57ec2-65c8-4073-a229-74741e1a1030\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n"
Jul 28 01:46:23.829316 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:23.829290    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57e57ec2-65c8-4073-a229-74741e1a1030-maas-api-tls\") pod \"maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n\" (UID: \"57e57ec2-65c8-4073-a229-74741e1a1030\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n"
Jul 28 01:46:23.908326 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:23.908291    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc"]
Jul 28 01:46:23.911975 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:23.911957    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc"
Jul 28 01:46:23.914544 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:23.914522    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-2cd0f2-dockercfg-ngkxc\""
Jul 28 01:46:23.914896 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:23.914881    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-2cd0f2\""
Jul 28 01:46:23.917262 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:23.917231    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679"]
Jul 28 01:46:23.921464 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:23.921441    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc"]
Jul 28 01:46:23.921590 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:23.921540    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679"
Jul 28 01:46:23.930570 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:23.930537    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679"]
Jul 28 01:46:24.028408 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:24.028368    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft2bd\" (UniqueName: \"kubernetes.io/projected/34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6-kube-api-access-ft2bd\") pod \"payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679\" (UID: \"34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6\") " pod="openshift-ingress/payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679"
Jul 28 01:46:24.028602 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:24.028474    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679\" (UID: \"34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6\") " pod="openshift-ingress/payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679"
Jul 28 01:46:24.028602 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:24.028531    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6b500ee-e3b5-409c-967d-c4426ba9dfa2-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc\" (UID: \"a6b500ee-e3b5-409c-967d-c4426ba9dfa2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc"
Jul 28 01:46:24.028602 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:24.028587    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgvks\" (UniqueName: \"kubernetes.io/projected/a6b500ee-e3b5-409c-967d-c4426ba9dfa2-kube-api-access-xgvks\") pod \"payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc\" (UID: \"a6b500ee-e3b5-409c-967d-c4426ba9dfa2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc"
Jul 28 01:46:24.071433 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:24.071386    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n"
Jul 28 01:46:24.129290 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:24.129252    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft2bd\" (UniqueName: \"kubernetes.io/projected/34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6-kube-api-access-ft2bd\") pod \"payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679\" (UID: \"34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6\") " pod="openshift-ingress/payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679"
Jul 28 01:46:24.129437 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:24.129355    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679\" (UID: \"34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6\") " pod="openshift-ingress/payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679"
Jul 28 01:46:24.129437 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:24.129389    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6b500ee-e3b5-409c-967d-c4426ba9dfa2-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc\" (UID: \"a6b500ee-e3b5-409c-967d-c4426ba9dfa2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc"
Jul 28 01:46:24.129552 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:24.129452    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgvks\" (UniqueName: \"kubernetes.io/projected/a6b500ee-e3b5-409c-967d-c4426ba9dfa2-kube-api-access-xgvks\") pod \"payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc\" (UID: \"a6b500ee-e3b5-409c-967d-c4426ba9dfa2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc"
Jul 28 01:46:24.130287 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:24.130264    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6b500ee-e3b5-409c-967d-c4426ba9dfa2-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc\" (UID: \"a6b500ee-e3b5-409c-967d-c4426ba9dfa2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc"
Jul 28 01:46:24.130396 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:24.130302    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679\" (UID: \"34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6\") " pod="openshift-ingress/payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679"
Jul 28 01:46:24.139388 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:24.139302    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgvks\" (UniqueName: \"kubernetes.io/projected/a6b500ee-e3b5-409c-967d-c4426ba9dfa2-kube-api-access-xgvks\") pod \"payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc\" (UID: \"a6b500ee-e3b5-409c-967d-c4426ba9dfa2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc"
Jul 28 01:46:24.139388 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:24.139302    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft2bd\" (UniqueName: \"kubernetes.io/projected/34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6-kube-api-access-ft2bd\") pod \"payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679\" (UID: \"34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6\") " pod="openshift-ingress/payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679"
Jul 28 01:46:24.217591 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:24.217563    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n"]
Jul 28 01:46:24.218713 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:46:24.218687    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57e57ec2_65c8_4073_a229_74741e1a1030.slice/crio-1a1208326d0875dcf892bd06b4046a33913c5a230010f558f39079055ebb4307 WatchSource:0}: Error finding container 1a1208326d0875dcf892bd06b4046a33913c5a230010f558f39079055ebb4307: Status 404 returned error can't find the container with id 1a1208326d0875dcf892bd06b4046a33913c5a230010f558f39079055ebb4307
Jul 28 01:46:24.226252 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:24.226227    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc"
Jul 28 01:46:24.236655 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:24.236626    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679"
Jul 28 01:46:24.374067 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:24.374002    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc"]
Jul 28 01:46:24.378641 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:46:24.378614    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6b500ee_e3b5_409c_967d_c4426ba9dfa2.slice/crio-da1d042f24eea921e5e1d832b85622a29b0806232fdba3a8f610411f8c4dadf4 WatchSource:0}: Error finding container da1d042f24eea921e5e1d832b85622a29b0806232fdba3a8f610411f8c4dadf4: Status 404 returned error can't find the container with id da1d042f24eea921e5e1d832b85622a29b0806232fdba3a8f610411f8c4dadf4
Jul 28 01:46:24.400840 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:46:24.400815    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34a4cd1e_9790_4c59_bd6e_ddc427a8f2e6.slice/crio-e39db1fbd32c67124c6f955d0dde09ca68fb1394fdcc2daff984e9c5c9a2c9e7 WatchSource:0}: Error finding container e39db1fbd32c67124c6f955d0dde09ca68fb1394fdcc2daff984e9c5c9a2c9e7: Status 404 returned error can't find the container with id e39db1fbd32c67124c6f955d0dde09ca68fb1394fdcc2daff984e9c5c9a2c9e7
Jul 28 01:46:24.404823 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:24.404800    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679"]
Jul 28 01:46:24.481888 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:24.481816    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n" event={"ID":"57e57ec2-65c8-4073-a229-74741e1a1030","Type":"ContainerStarted","Data":"1a1208326d0875dcf892bd06b4046a33913c5a230010f558f39079055ebb4307"}
Jul 28 01:46:24.483382 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:24.483348    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679" event={"ID":"34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6","Type":"ContainerStarted","Data":"2e0ec2fc5d88d6aaeb46c57d206c1d2adc12ae45e9a0d3d7435e4da35fd781bc"}
Jul 28 01:46:24.483543 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:24.483410    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679"
Jul 28 01:46:24.483543 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:24.483434    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679" event={"ID":"34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6","Type":"ContainerStarted","Data":"e39db1fbd32c67124c6f955d0dde09ca68fb1394fdcc2daff984e9c5c9a2c9e7"}
Jul 28 01:46:24.485035 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:24.485014    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc" event={"ID":"a6b500ee-e3b5-409c-967d-c4426ba9dfa2","Type":"ContainerStarted","Data":"85b6cbe7c3ccf06de22d89773b1730cd46a1393d4f9a6ee0b6889c412e0370ab"}
Jul 28 01:46:24.485179 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:24.485051    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc" event={"ID":"a6b500ee-e3b5-409c-967d-c4426ba9dfa2","Type":"ContainerStarted","Data":"da1d042f24eea921e5e1d832b85622a29b0806232fdba3a8f610411f8c4dadf4"}
Jul 28 01:46:24.485179 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:24.485074    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc"
Jul 28 01:46:24.500405 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:24.500360    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679" podStartSLOduration=1.5003450809999999 podStartE2EDuration="1.500345081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:46:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:46:24.499185076 +0000 UTC m=+3154.663993177" watchObservedRunningTime="2026-07-28 01:46:24.500345081 +0000 UTC m=+3154.665153189"
Jul 28 01:46:24.515440 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:24.515391    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc" podStartSLOduration=1.515376133 podStartE2EDuration="1.515376133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:46:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:46:24.514648719 +0000 UTC m=+3154.679456838" watchObservedRunningTime="2026-07-28 01:46:24.515376133 +0000 UTC m=+3154.680184240"
Jul 28 01:46:25.493542 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:25.493446    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n" event={"ID":"57e57ec2-65c8-4073-a229-74741e1a1030","Type":"ContainerStarted","Data":"c9e0f34d05447e7a14b74ae01c4466b3355e3f57bebd02c793f131b3fd504d3d"}
Jul 28 01:46:25.494005 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:25.493964    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n"
Jul 28 01:46:25.512322 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:25.512267    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n" podStartSLOduration=2.012190929 podStartE2EDuration="2.512249465s" totalImagesPullingTime="500.058536ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:46:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:46:25.509396537 +0000 UTC m=+3155.674204678" watchObservedRunningTime="2026-07-28 01:46:25.512249465 +0000 UTC m=+3155.677057565"
Jul 28 01:46:31.111077 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:31.111045    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g"]
Jul 28 01:46:31.114824 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:31.114801    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g"
Jul 28 01:46:31.117771 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:31.117752    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-f67c19-serving-cert\""
Jul 28 01:46:31.135609 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:31.135587    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g"]
Jul 28 01:46:31.195996 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:31.195962    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdr2r\" (UniqueName: \"kubernetes.io/projected/8b335b18-c9fa-4693-9e38-e1a1686c0315-kube-api-access-kdr2r\") pod \"maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g\" (UID: \"8b335b18-c9fa-4693-9e38-e1a1686c0315\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g"
Jul 28 01:46:31.196185 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:31.196029    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b335b18-c9fa-4693-9e38-e1a1686c0315-maas-api-tls\") pod \"maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g\" (UID: \"8b335b18-c9fa-4693-9e38-e1a1686c0315\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g"
Jul 28 01:46:31.196185 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:31.196074    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b335b18-c9fa-4693-9e38-e1a1686c0315-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g\" (UID: \"8b335b18-c9fa-4693-9e38-e1a1686c0315\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g"
Jul 28 01:46:31.297640 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:31.297608    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b335b18-c9fa-4693-9e38-e1a1686c0315-maas-api-tls\") pod \"maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g\" (UID: \"8b335b18-c9fa-4693-9e38-e1a1686c0315\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g"
Jul 28 01:46:31.297818 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:31.297649    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b335b18-c9fa-4693-9e38-e1a1686c0315-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g\" (UID: \"8b335b18-c9fa-4693-9e38-e1a1686c0315\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g"
Jul 28 01:46:31.297818 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:31.297713    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdr2r\" (UniqueName: \"kubernetes.io/projected/8b335b18-c9fa-4693-9e38-e1a1686c0315-kube-api-access-kdr2r\") pod \"maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g\" (UID: \"8b335b18-c9fa-4693-9e38-e1a1686c0315\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g"
Jul 28 01:46:31.298442 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:31.298408    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b335b18-c9fa-4693-9e38-e1a1686c0315-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g\" (UID: \"8b335b18-c9fa-4693-9e38-e1a1686c0315\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g"
Jul 28 01:46:31.300076 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:31.300057    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b335b18-c9fa-4693-9e38-e1a1686c0315-maas-api-tls\") pod \"maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g\" (UID: \"8b335b18-c9fa-4693-9e38-e1a1686c0315\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g"
Jul 28 01:46:31.310799 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:31.310770    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdr2r\" (UniqueName: \"kubernetes.io/projected/8b335b18-c9fa-4693-9e38-e1a1686c0315-kube-api-access-kdr2r\") pod \"maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g\" (UID: \"8b335b18-c9fa-4693-9e38-e1a1686c0315\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g"
Jul 28 01:46:31.428416 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:31.428335    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g"
Jul 28 01:46:31.569416 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:31.569384    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g"]
Jul 28 01:46:31.570297 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:46:31.570268    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b335b18_c9fa_4693_9e38_e1a1686c0315.slice/crio-a4f38cd068c963ceb4d20d5d872fbb51fbae9a93059455225d5bd43b5243d41e WatchSource:0}: Error finding container a4f38cd068c963ceb4d20d5d872fbb51fbae9a93059455225d5bd43b5243d41e: Status 404 returned error can't find the container with id a4f38cd068c963ceb4d20d5d872fbb51fbae9a93059455225d5bd43b5243d41e
Jul 28 01:46:31.949738 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:31.949714    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x"]
Jul 28 01:46:31.953049 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:31.953031    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x"
Jul 28 01:46:31.955685 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:31.955660    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-f67c19\""
Jul 28 01:46:31.955838 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:31.955819    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-f67c19-dockercfg-tw5jf\""
Jul 28 01:46:31.965662 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:31.965619    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x"]
Jul 28 01:46:32.006045 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:32.006012    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-586pn\" (UniqueName: \"kubernetes.io/projected/069ae680-92f6-4d0e-82ef-2933da3bd757-kube-api-access-586pn\") pod \"payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x\" (UID: \"069ae680-92f6-4d0e-82ef-2933da3bd757\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x"
Jul 28 01:46:32.006237 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:32.006134    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/069ae680-92f6-4d0e-82ef-2933da3bd757-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x\" (UID: \"069ae680-92f6-4d0e-82ef-2933da3bd757\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x"
Jul 28 01:46:32.014729 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:32.014705    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m"]
Jul 28 01:46:32.018255 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:32.018237    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m"
Jul 28 01:46:32.029620 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:32.029595    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m"]
Jul 28 01:46:32.107625 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:32.107582    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0e797f2-0eae-489c-997b-6a7cda092b82-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m\" (UID: \"b0e797f2-0eae-489c-997b-6a7cda092b82\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m"
Jul 28 01:46:32.107805 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:32.107708    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-586pn\" (UniqueName: \"kubernetes.io/projected/069ae680-92f6-4d0e-82ef-2933da3bd757-kube-api-access-586pn\") pod \"payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x\" (UID: \"069ae680-92f6-4d0e-82ef-2933da3bd757\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x"
Jul 28 01:46:32.107805 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:32.107774    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwjq8\" (UniqueName: \"kubernetes.io/projected/b0e797f2-0eae-489c-997b-6a7cda092b82-kube-api-access-qwjq8\") pod \"payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m\" (UID: \"b0e797f2-0eae-489c-997b-6a7cda092b82\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m"
Jul 28 01:46:32.107884 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:32.107833    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/069ae680-92f6-4d0e-82ef-2933da3bd757-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x\" (UID: \"069ae680-92f6-4d0e-82ef-2933da3bd757\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x"
Jul 28 01:46:32.108594 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:32.108576    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/069ae680-92f6-4d0e-82ef-2933da3bd757-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x\" (UID: \"069ae680-92f6-4d0e-82ef-2933da3bd757\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x"
Jul 28 01:46:32.116719 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:32.116691    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-586pn\" (UniqueName: \"kubernetes.io/projected/069ae680-92f6-4d0e-82ef-2933da3bd757-kube-api-access-586pn\") pod \"payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x\" (UID: \"069ae680-92f6-4d0e-82ef-2933da3bd757\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x"
Jul 28 01:46:32.209531 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:32.209434    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwjq8\" (UniqueName: \"kubernetes.io/projected/b0e797f2-0eae-489c-997b-6a7cda092b82-kube-api-access-qwjq8\") pod \"payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m\" (UID: \"b0e797f2-0eae-489c-997b-6a7cda092b82\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m"
Jul 28 01:46:32.209693 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:32.209547    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0e797f2-0eae-489c-997b-6a7cda092b82-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m\" (UID: \"b0e797f2-0eae-489c-997b-6a7cda092b82\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m"
Jul 28 01:46:32.210280 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:32.210260    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0e797f2-0eae-489c-997b-6a7cda092b82-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m\" (UID: \"b0e797f2-0eae-489c-997b-6a7cda092b82\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m"
Jul 28 01:46:32.220586 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:32.220552    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwjq8\" (UniqueName: \"kubernetes.io/projected/b0e797f2-0eae-489c-997b-6a7cda092b82-kube-api-access-qwjq8\") pod \"payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m\" (UID: \"b0e797f2-0eae-489c-997b-6a7cda092b82\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m"
Jul 28 01:46:32.267054 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:32.267022    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x"
Jul 28 01:46:32.328778 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:32.328746    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m"
Jul 28 01:46:32.436471 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:46:32.436421    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod069ae680_92f6_4d0e_82ef_2933da3bd757.slice/crio-7e5c4b12637e2e429ec333cc33790e422dd9c770b7cb7a3a562fb4e079f85b6a WatchSource:0}: Error finding container 7e5c4b12637e2e429ec333cc33790e422dd9c770b7cb7a3a562fb4e079f85b6a: Status 404 returned error can't find the container with id 7e5c4b12637e2e429ec333cc33790e422dd9c770b7cb7a3a562fb4e079f85b6a
Jul 28 01:46:32.437902 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:32.437858    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x"]
Jul 28 01:46:32.499933 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:32.499904    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m"]
Jul 28 01:46:32.501204 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:46:32.501177    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0e797f2_0eae_489c_997b_6a7cda092b82.slice/crio-c2b2b2c39ee323bbab9c8f126a0d2e5f05d4443a44f13e45f0dd48dc6ee8e181 WatchSource:0}: Error finding container c2b2b2c39ee323bbab9c8f126a0d2e5f05d4443a44f13e45f0dd48dc6ee8e181: Status 404 returned error can't find the container with id c2b2b2c39ee323bbab9c8f126a0d2e5f05d4443a44f13e45f0dd48dc6ee8e181
Jul 28 01:46:32.508804 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:32.508785    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n"
Jul 28 01:46:32.535131 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:32.535104    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x" event={"ID":"069ae680-92f6-4d0e-82ef-2933da3bd757","Type":"ContainerStarted","Data":"7e5c4b12637e2e429ec333cc33790e422dd9c770b7cb7a3a562fb4e079f85b6a"}
Jul 28 01:46:32.536490 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:32.536463    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m" event={"ID":"b0e797f2-0eae-489c-997b-6a7cda092b82","Type":"ContainerStarted","Data":"c2b2b2c39ee323bbab9c8f126a0d2e5f05d4443a44f13e45f0dd48dc6ee8e181"}
Jul 28 01:46:32.538092 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:32.538064    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g" event={"ID":"8b335b18-c9fa-4693-9e38-e1a1686c0315","Type":"ContainerStarted","Data":"9ef1194aba9edb60bcd55d124f2a0fd1d2b48cbc4c976190d3685923f7424b24"}
Jul 28 01:46:32.538194 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:32.538117    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g" event={"ID":"8b335b18-c9fa-4693-9e38-e1a1686c0315","Type":"ContainerStarted","Data":"a4f38cd068c963ceb4d20d5d872fbb51fbae9a93059455225d5bd43b5243d41e"}
Jul 28 01:46:32.538194 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:32.538147    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g"
Jul 28 01:46:33.546370 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:33.546335    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x" event={"ID":"069ae680-92f6-4d0e-82ef-2933da3bd757","Type":"ContainerStarted","Data":"1a344d4415631fe4c4f6e15e036c63a1e68f55d45d1efdace11034f7d8f025ca"}
Jul 28 01:46:33.546827 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:33.546465    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x"
Jul 28 01:46:33.548013 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:33.547987    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m" event={"ID":"b0e797f2-0eae-489c-997b-6a7cda092b82","Type":"ContainerStarted","Data":"3090685f7dc1360d6169e637a12628d2d0753f2eb64966a402eac4bfb6b8bde0"}
Jul 28 01:46:33.548120 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:33.548107    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m"
Jul 28 01:46:33.563782 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:33.563711    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x" podStartSLOduration=2.563690413 podStartE2EDuration="2.563690413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:46:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:46:33.561163712 +0000 UTC m=+3163.725971821" watchObservedRunningTime="2026-07-28 01:46:33.563690413 +0000 UTC m=+3163.728498521"
Jul 28 01:46:33.565959 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:33.565918    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g" podStartSLOduration=2.243043371 podStartE2EDuration="2.565905445s" totalImagesPullingTime="322.862074ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:46:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:46:32.567774719 +0000 UTC m=+3162.732582827" watchObservedRunningTime="2026-07-28 01:46:33.565905445 +0000 UTC m=+3163.730713589"
Jul 28 01:46:33.578462 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:33.578413    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m" podStartSLOduration=2.5783965699999998 podStartE2EDuration="2.57839657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:46:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:46:33.577239093 +0000 UTC m=+3163.742047201" watchObservedRunningTime="2026-07-28 01:46:33.57839657 +0000 UTC m=+3163.743204680"
Jul 28 01:46:35.495512 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:35.495466    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc" podUID="a6b500ee-e3b5-409c-967d-c4426ba9dfa2" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.63:9004: connect: connection refused"
Jul 28 01:46:35.495910 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:35.495703    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679"
Jul 28 01:46:38.553849 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:38.553818    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g"
Jul 28 01:46:44.555889 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:44.555858    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x"
Jul 28 01:46:44.556303 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:44.555930    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m"
Jul 28 01:46:45.495089 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:45.495058    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc"
Jul 28 01:46:50.779449 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:46:50.779416    2578 scope.go:117] "RemoveContainer" containerID="a372dbb1cd16f20182b01f524ca2656936270f8d09fa0d6d4879fa9977b69833"
Jul 28 01:47:15.120882 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.120848    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"]
Jul 28 01:47:15.124303 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.124282    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"
Jul 28 01:47:15.127009 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.126984    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-f67c19\"/\"kube-root-ca.crt\""
Jul 28 01:47:15.127145 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.126984    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-f67c19\"/\"openshift-service-ca.crt\""
Jul 28 01:47:15.128070 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.128046    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-f67c19\"/\"default-dockercfg-sgpr2\""
Jul 28 01:47:15.128329 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.128289    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-f67c19\"/\"test-model-f67c19-kserve-self-signed-certs\""
Jul 28 01:47:15.135928 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.135904    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"]
Jul 28 01:47:15.230786 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.230740    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vzn9\" (UniqueName: \"kubernetes.io/projected/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-kube-api-access-2vzn9\") pod \"test-model-f67c19-kserve-7cf79664fc-mr6xp\" (UID: \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\") " pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"
Jul 28 01:47:15.230786 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.230790    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-tls-certs\") pod \"test-model-f67c19-kserve-7cf79664fc-mr6xp\" (UID: \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\") " pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"
Jul 28 01:47:15.231002 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.230816    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-dshm\") pod \"test-model-f67c19-kserve-7cf79664fc-mr6xp\" (UID: \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\") " pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"
Jul 28 01:47:15.231002 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.230921    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-home\") pod \"test-model-f67c19-kserve-7cf79664fc-mr6xp\" (UID: \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\") " pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"
Jul 28 01:47:15.231002 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.230958    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-tmp-dir\") pod \"test-model-f67c19-kserve-7cf79664fc-mr6xp\" (UID: \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\") " pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"
Jul 28 01:47:15.231002 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.230988    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-model-cache\") pod \"test-model-f67c19-kserve-7cf79664fc-mr6xp\" (UID: \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\") " pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"
Jul 28 01:47:15.332083 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.332045    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vzn9\" (UniqueName: \"kubernetes.io/projected/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-kube-api-access-2vzn9\") pod \"test-model-f67c19-kserve-7cf79664fc-mr6xp\" (UID: \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\") " pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"
Jul 28 01:47:15.332083 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.332084    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-tls-certs\") pod \"test-model-f67c19-kserve-7cf79664fc-mr6xp\" (UID: \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\") " pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"
Jul 28 01:47:15.332390 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.332110    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-dshm\") pod \"test-model-f67c19-kserve-7cf79664fc-mr6xp\" (UID: \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\") " pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"
Jul 28 01:47:15.332390 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.332164    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-home\") pod \"test-model-f67c19-kserve-7cf79664fc-mr6xp\" (UID: \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\") " pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"
Jul 28 01:47:15.332390 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.332187    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-tmp-dir\") pod \"test-model-f67c19-kserve-7cf79664fc-mr6xp\" (UID: \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\") " pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"
Jul 28 01:47:15.332390 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.332242    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-model-cache\") pod \"test-model-f67c19-kserve-7cf79664fc-mr6xp\" (UID: \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\") " pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"
Jul 28 01:47:15.332635 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.332615    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-home\") pod \"test-model-f67c19-kserve-7cf79664fc-mr6xp\" (UID: \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\") " pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"
Jul 28 01:47:15.332715 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.332693    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-tmp-dir\") pod \"test-model-f67c19-kserve-7cf79664fc-mr6xp\" (UID: \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\") " pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"
Jul 28 01:47:15.332773 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.332721    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-model-cache\") pod \"test-model-f67c19-kserve-7cf79664fc-mr6xp\" (UID: \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\") " pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"
Jul 28 01:47:15.334415 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.334395    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-dshm\") pod \"test-model-f67c19-kserve-7cf79664fc-mr6xp\" (UID: \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\") " pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"
Jul 28 01:47:15.334787 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.334767    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-tls-certs\") pod \"test-model-f67c19-kserve-7cf79664fc-mr6xp\" (UID: \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\") " pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"
Jul 28 01:47:15.340323 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.340300    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vzn9\" (UniqueName: \"kubernetes.io/projected/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-kube-api-access-2vzn9\") pod \"test-model-f67c19-kserve-7cf79664fc-mr6xp\" (UID: \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\") " pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"
Jul 28 01:47:15.437671 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.437586    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"
Jul 28 01:47:15.578258 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.578228    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"]
Jul 28 01:47:15.580184 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:47:15.580146    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac8bc5e8_bb27_4e57_9a10_fc35e2753605.slice/crio-a3677d4e4ec2493a7ce0f5ae8e347965c47698982fe3f4e1a5b23f87a74281a1 WatchSource:0}: Error finding container a3677d4e4ec2493a7ce0f5ae8e347965c47698982fe3f4e1a5b23f87a74281a1: Status 404 returned error can't find the container with id a3677d4e4ec2493a7ce0f5ae8e347965c47698982fe3f4e1a5b23f87a74281a1
Jul 28 01:47:15.794376 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.794274    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp" event={"ID":"ac8bc5e8-bb27-4e57-9a10-fc35e2753605","Type":"ContainerStarted","Data":"5f9e8b7657b1868d4749bcc127ab5de56cac1d410f34aa3a237b91e67091ecc0"}
Jul 28 01:47:15.794376 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.794323    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp" event={"ID":"ac8bc5e8-bb27-4e57-9a10-fc35e2753605","Type":"ContainerStarted","Data":"a3677d4e4ec2493a7ce0f5ae8e347965c47698982fe3f4e1a5b23f87a74281a1"}
Jul 28 01:47:15.814357 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:15.814298    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp" podStartSLOduration=0.814281222 podStartE2EDuration="814.281222ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:47:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:47:15.811274011 +0000 UTC m=+3205.976082115" watchObservedRunningTime="2026-07-28 01:47:15.814281222 +0000 UTC m=+3205.979089328"
Jul 28 01:47:25.438084 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:25.438050    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"
Jul 28 01:47:25.438084 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:25.438089    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"
Jul 28 01:47:25.451441 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:25.451417    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"
Jul 28 01:47:25.865079 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:47:25.865054    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"
Jul 28 01:48:01.096837 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:01.096803    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"]
Jul 28 01:48:11.785684 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:11.785648    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n"]
Jul 28 01:48:11.786128 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:11.785866    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n" podUID="57e57ec2-65c8-4073-a229-74741e1a1030" containerName="maas-api" containerID="cri-o://c9e0f34d05447e7a14b74ae01c4466b3355e3f57bebd02c793f131b3fd504d3d" gracePeriod=30
Jul 28 01:48:11.841456 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:11.841420    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679"]
Jul 28 01:48:11.841669 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:11.841637    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679" podUID="34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6" containerName="payload-processing" containerID="cri-o://2e0ec2fc5d88d6aaeb46c57d206c1d2adc12ae45e9a0d3d7435e4da35fd781bc" gracePeriod=30
Jul 28 01:48:11.853458 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:11.853424    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc"]
Jul 28 01:48:11.853714 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:11.853686    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc" podUID="a6b500ee-e3b5-409c-967d-c4426ba9dfa2" containerName="payload-pre-processing" containerID="cri-o://85b6cbe7c3ccf06de22d89773b1730cd46a1393d4f9a6ee0b6889c412e0370ab" gracePeriod=30
Jul 28 01:48:12.106488 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.106461    2578 generic.go:356] "Generic (PLEG): container finished" podID="57e57ec2-65c8-4073-a229-74741e1a1030" containerID="c9e0f34d05447e7a14b74ae01c4466b3355e3f57bebd02c793f131b3fd504d3d" exitCode=0
Jul 28 01:48:12.106607 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.106533    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n" event={"ID":"57e57ec2-65c8-4073-a229-74741e1a1030","Type":"ContainerDied","Data":"c9e0f34d05447e7a14b74ae01c4466b3355e3f57bebd02c793f131b3fd504d3d"}
Jul 28 01:48:12.108066 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.108042    2578 generic.go:356] "Generic (PLEG): container finished" podID="34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6" containerID="2e0ec2fc5d88d6aaeb46c57d206c1d2adc12ae45e9a0d3d7435e4da35fd781bc" exitCode=0
Jul 28 01:48:12.108168 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.108117    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679" event={"ID":"34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6","Type":"ContainerDied","Data":"2e0ec2fc5d88d6aaeb46c57d206c1d2adc12ae45e9a0d3d7435e4da35fd781bc"}
Jul 28 01:48:12.109960 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.109938    2578 generic.go:356] "Generic (PLEG): container finished" podID="a6b500ee-e3b5-409c-967d-c4426ba9dfa2" containerID="85b6cbe7c3ccf06de22d89773b1730cd46a1393d4f9a6ee0b6889c412e0370ab" exitCode=0
Jul 28 01:48:12.110086 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.109976    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc" event={"ID":"a6b500ee-e3b5-409c-967d-c4426ba9dfa2","Type":"ContainerDied","Data":"85b6cbe7c3ccf06de22d89773b1730cd46a1393d4f9a6ee0b6889c412e0370ab"}
Jul 28 01:48:12.143653 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.143486    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679"
Jul 28 01:48:12.146710 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.146685    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc"
Jul 28 01:48:12.207101 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.207067    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ft2bd\" (UniqueName: \"kubernetes.io/projected/34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6-kube-api-access-ft2bd\") pod \"34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6\" (UID: \"34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6\") "
Jul 28 01:48:12.207298 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.207159    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6-plugins-config-volume\") pod \"34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6\" (UID: \"34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6\") "
Jul 28 01:48:12.207298 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.207226    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6b500ee-e3b5-409c-967d-c4426ba9dfa2-plugins-config-volume\") pod \"a6b500ee-e3b5-409c-967d-c4426ba9dfa2\" (UID: \"a6b500ee-e3b5-409c-967d-c4426ba9dfa2\") "
Jul 28 01:48:12.207298 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.207267    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgvks\" (UniqueName: \"kubernetes.io/projected/a6b500ee-e3b5-409c-967d-c4426ba9dfa2-kube-api-access-xgvks\") pod \"a6b500ee-e3b5-409c-967d-c4426ba9dfa2\" (UID: \"a6b500ee-e3b5-409c-967d-c4426ba9dfa2\") "
Jul 28 01:48:12.207807 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.207773    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6b500ee-e3b5-409c-967d-c4426ba9dfa2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a6b500ee-e3b5-409c-967d-c4426ba9dfa2" (UID: "a6b500ee-e3b5-409c-967d-c4426ba9dfa2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:48:12.207969 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.207860    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6" (UID: "34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:48:12.210502 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.210470    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6-kube-api-access-ft2bd" (OuterVolumeSpecName: "kube-api-access-ft2bd") pod "34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6" (UID: "34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6"). InnerVolumeSpecName "kube-api-access-ft2bd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:48:12.210624 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.210587    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6b500ee-e3b5-409c-967d-c4426ba9dfa2-kube-api-access-xgvks" (OuterVolumeSpecName: "kube-api-access-xgvks") pod "a6b500ee-e3b5-409c-967d-c4426ba9dfa2" (UID: "a6b500ee-e3b5-409c-967d-c4426ba9dfa2"). InnerVolumeSpecName "kube-api-access-xgvks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:48:12.229862 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.229840    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n"
Jul 28 01:48:12.308940 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.308894    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57e57ec2-65c8-4073-a229-74741e1a1030-maas-api-tls\") pod \"57e57ec2-65c8-4073-a229-74741e1a1030\" (UID: \"57e57ec2-65c8-4073-a229-74741e1a1030\") "
Jul 28 01:48:12.308940 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.308946    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8xdr\" (UniqueName: \"kubernetes.io/projected/57e57ec2-65c8-4073-a229-74741e1a1030-kube-api-access-v8xdr\") pod \"57e57ec2-65c8-4073-a229-74741e1a1030\" (UID: \"57e57ec2-65c8-4073-a229-74741e1a1030\") "
Jul 28 01:48:12.309289 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.309016    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57e57ec2-65c8-4073-a229-74741e1a1030-openshift-service-ca\") pod \"57e57ec2-65c8-4073-a229-74741e1a1030\" (UID: \"57e57ec2-65c8-4073-a229-74741e1a1030\") "
Jul 28 01:48:12.309289 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.309207    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ft2bd\" (UniqueName: \"kubernetes.io/projected/34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6-kube-api-access-ft2bd\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:48:12.309289 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.309247    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6-plugins-config-volume\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:48:12.309289 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.309258    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6b500ee-e3b5-409c-967d-c4426ba9dfa2-plugins-config-volume\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:48:12.309289 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.309267    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgvks\" (UniqueName: \"kubernetes.io/projected/a6b500ee-e3b5-409c-967d-c4426ba9dfa2-kube-api-access-xgvks\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:48:12.309577 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.309498    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57e57ec2-65c8-4073-a229-74741e1a1030-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "57e57ec2-65c8-4073-a229-74741e1a1030" (UID: "57e57ec2-65c8-4073-a229-74741e1a1030"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:48:12.311046 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.311019    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57e57ec2-65c8-4073-a229-74741e1a1030-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "57e57ec2-65c8-4073-a229-74741e1a1030" (UID: "57e57ec2-65c8-4073-a229-74741e1a1030"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:48:12.311330 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.311309    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57e57ec2-65c8-4073-a229-74741e1a1030-kube-api-access-v8xdr" (OuterVolumeSpecName: "kube-api-access-v8xdr") pod "57e57ec2-65c8-4073-a229-74741e1a1030" (UID: "57e57ec2-65c8-4073-a229-74741e1a1030"). InnerVolumeSpecName "kube-api-access-v8xdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:48:12.409828 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.409795    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57e57ec2-65c8-4073-a229-74741e1a1030-openshift-service-ca\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:48:12.409828 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.409823    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57e57ec2-65c8-4073-a229-74741e1a1030-maas-api-tls\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:48:12.410067 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:12.409841    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8xdr\" (UniqueName: \"kubernetes.io/projected/57e57ec2-65c8-4073-a229-74741e1a1030-kube-api-access-v8xdr\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:48:13.116860 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:13.116769    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc"
Jul 28 01:48:13.116860 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:13.116784    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc" event={"ID":"a6b500ee-e3b5-409c-967d-c4426ba9dfa2","Type":"ContainerDied","Data":"da1d042f24eea921e5e1d832b85622a29b0806232fdba3a8f610411f8c4dadf4"}
Jul 28 01:48:13.116860 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:13.116851    2578 scope.go:117] "RemoveContainer" containerID="85b6cbe7c3ccf06de22d89773b1730cd46a1393d4f9a6ee0b6889c412e0370ab"
Jul 28 01:48:13.118292 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:13.118270    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n"
Jul 28 01:48:13.118292 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:13.118283    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n" event={"ID":"57e57ec2-65c8-4073-a229-74741e1a1030","Type":"ContainerDied","Data":"1a1208326d0875dcf892bd06b4046a33913c5a230010f558f39079055ebb4307"}
Jul 28 01:48:13.120057 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:13.120031    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679" event={"ID":"34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6","Type":"ContainerDied","Data":"e39db1fbd32c67124c6f955d0dde09ca68fb1394fdcc2daff984e9c5c9a2c9e7"}
Jul 28 01:48:13.120158 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:13.120042    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679"
Jul 28 01:48:13.127095 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:13.127080    2578 scope.go:117] "RemoveContainer" containerID="c9e0f34d05447e7a14b74ae01c4466b3355e3f57bebd02c793f131b3fd504d3d"
Jul 28 01:48:13.134604 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:13.134590    2578 scope.go:117] "RemoveContainer" containerID="2e0ec2fc5d88d6aaeb46c57d206c1d2adc12ae45e9a0d3d7435e4da35fd781bc"
Jul 28 01:48:13.141685 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:13.141657    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679"]
Jul 28 01:48:13.144730 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:13.144711    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-2cd0f2-7bdc79f768-gx679"]
Jul 28 01:48:13.154382 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:13.154358    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n"]
Jul 28 01:48:13.158380 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:13.158355    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2cd0f2-6bbb8d75f-4f95n"]
Jul 28 01:48:13.169172 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:13.169145    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc"]
Jul 28 01:48:13.176806 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:13.176782    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-2cd0f2-7644b5c596-m77kc"]
Jul 28 01:48:14.401946 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:14.401914    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6" path="/var/lib/kubelet/pods/34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6/volumes"
Jul 28 01:48:14.402328 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:14.402306    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57e57ec2-65c8-4073-a229-74741e1a1030" path="/var/lib/kubelet/pods/57e57ec2-65c8-4073-a229-74741e1a1030/volumes"
Jul 28 01:48:14.402624 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:14.402611    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6b500ee-e3b5-409c-967d-c4426ba9dfa2" path="/var/lib/kubelet/pods/a6b500ee-e3b5-409c-967d-c4426ba9dfa2/volumes"
Jul 28 01:48:16.116079 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:16.116010    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp" podUID="ac8bc5e8-bb27-4e57-9a10-fc35e2753605" containerName="main" containerID="cri-o://5f9e8b7657b1868d4749bcc127ab5de56cac1d410f34aa3a237b91e67091ecc0" gracePeriod=45
Jul 28 01:48:16.371329 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:16.371272    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"
Jul 28 01:48:16.449525 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:16.449490    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-home\") pod \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\" (UID: \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\") "
Jul 28 01:48:16.449700 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:16.449533    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-tmp-dir\") pod \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\" (UID: \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\") "
Jul 28 01:48:16.449700 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:16.449610    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-tls-certs\") pod \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\" (UID: \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\") "
Jul 28 01:48:16.449700 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:16.449636    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-model-cache\") pod \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\" (UID: \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\") "
Jul 28 01:48:16.449700 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:16.449681    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vzn9\" (UniqueName: \"kubernetes.io/projected/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-kube-api-access-2vzn9\") pod \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\" (UID: \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\") "
Jul 28 01:48:16.449891 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:16.449756    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-dshm\") pod \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\" (UID: \"ac8bc5e8-bb27-4e57-9a10-fc35e2753605\") "
Jul 28 01:48:16.449891 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:16.449810    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-home" (OuterVolumeSpecName: "home") pod "ac8bc5e8-bb27-4e57-9a10-fc35e2753605" (UID: "ac8bc5e8-bb27-4e57-9a10-fc35e2753605"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:48:16.449891 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:16.449837    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ac8bc5e8-bb27-4e57-9a10-fc35e2753605" (UID: "ac8bc5e8-bb27-4e57-9a10-fc35e2753605"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:48:16.450056 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:16.449929    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-model-cache" (OuterVolumeSpecName: "model-cache") pod "ac8bc5e8-bb27-4e57-9a10-fc35e2753605" (UID: "ac8bc5e8-bb27-4e57-9a10-fc35e2753605"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:48:16.450200 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:16.450179    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-home\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:48:16.450304 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:16.450205    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-tmp-dir\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:48:16.450304 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:16.450249    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-model-cache\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:48:16.451909 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:16.451874    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-dshm" (OuterVolumeSpecName: "dshm") pod "ac8bc5e8-bb27-4e57-9a10-fc35e2753605" (UID: "ac8bc5e8-bb27-4e57-9a10-fc35e2753605"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:48:16.452015 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:16.451989    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ac8bc5e8-bb27-4e57-9a10-fc35e2753605" (UID: "ac8bc5e8-bb27-4e57-9a10-fc35e2753605"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:48:16.452059 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:16.452020    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-kube-api-access-2vzn9" (OuterVolumeSpecName: "kube-api-access-2vzn9") pod "ac8bc5e8-bb27-4e57-9a10-fc35e2753605" (UID: "ac8bc5e8-bb27-4e57-9a10-fc35e2753605"). InnerVolumeSpecName "kube-api-access-2vzn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:48:16.551575 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:16.551543    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-tls-certs\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:48:16.551575 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:16.551572    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vzn9\" (UniqueName: \"kubernetes.io/projected/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-kube-api-access-2vzn9\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:48:16.551575 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:16.551582    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac8bc5e8-bb27-4e57-9a10-fc35e2753605-dshm\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:48:17.144366 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:17.144332    2578 generic.go:356] "Generic (PLEG): container finished" podID="ac8bc5e8-bb27-4e57-9a10-fc35e2753605" containerID="5f9e8b7657b1868d4749bcc127ab5de56cac1d410f34aa3a237b91e67091ecc0" exitCode=0
Jul 28 01:48:17.144816 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:17.144371    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"
Jul 28 01:48:17.144816 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:17.144372    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp" event={"ID":"ac8bc5e8-bb27-4e57-9a10-fc35e2753605","Type":"ContainerDied","Data":"5f9e8b7657b1868d4749bcc127ab5de56cac1d410f34aa3a237b91e67091ecc0"}
Jul 28 01:48:17.144816 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:17.144507    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp" event={"ID":"ac8bc5e8-bb27-4e57-9a10-fc35e2753605","Type":"ContainerDied","Data":"a3677d4e4ec2493a7ce0f5ae8e347965c47698982fe3f4e1a5b23f87a74281a1"}
Jul 28 01:48:17.144816 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:17.144541    2578 scope.go:117] "RemoveContainer" containerID="5f9e8b7657b1868d4749bcc127ab5de56cac1d410f34aa3a237b91e67091ecc0"
Jul 28 01:48:17.153788 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:17.153768    2578 scope.go:117] "RemoveContainer" containerID="5f9e8b7657b1868d4749bcc127ab5de56cac1d410f34aa3a237b91e67091ecc0"
Jul 28 01:48:17.154057 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:48:17.154040    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f9e8b7657b1868d4749bcc127ab5de56cac1d410f34aa3a237b91e67091ecc0\": container with ID starting with 5f9e8b7657b1868d4749bcc127ab5de56cac1d410f34aa3a237b91e67091ecc0 not found: ID does not exist" containerID="5f9e8b7657b1868d4749bcc127ab5de56cac1d410f34aa3a237b91e67091ecc0"
Jul 28 01:48:17.154128 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:17.154068    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f9e8b7657b1868d4749bcc127ab5de56cac1d410f34aa3a237b91e67091ecc0"} err="failed to get container status \"5f9e8b7657b1868d4749bcc127ab5de56cac1d410f34aa3a237b91e67091ecc0\": rpc error: code = NotFound desc = could not find container \"5f9e8b7657b1868d4749bcc127ab5de56cac1d410f34aa3a237b91e67091ecc0\": container with ID starting with 5f9e8b7657b1868d4749bcc127ab5de56cac1d410f34aa3a237b91e67091ecc0 not found: ID does not exist"
Jul 28 01:48:17.167779 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:17.167741    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"]
Jul 28 01:48:17.171000 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:17.170973    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-f67c19/test-model-f67c19-kserve-7cf79664fc-mr6xp"]
Jul 28 01:48:18.402963 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:18.402926    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac8bc5e8-bb27-4e57-9a10-fc35e2753605" path="/var/lib/kubelet/pods/ac8bc5e8-bb27-4e57-9a10-fc35e2753605/volumes"
Jul 28 01:48:33.849858 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:33.849826    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g"]
Jul 28 01:48:33.850412 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:33.850048    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g" podUID="8b335b18-c9fa-4693-9e38-e1a1686c0315" containerName="maas-api" containerID="cri-o://9ef1194aba9edb60bcd55d124f2a0fd1d2b48cbc4c976190d3685923f7424b24" gracePeriod=30
Jul 28 01:48:33.910404 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:33.910364    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x"]
Jul 28 01:48:33.910705 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:33.910674    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x" podUID="069ae680-92f6-4d0e-82ef-2933da3bd757" containerName="payload-processing" containerID="cri-o://1a344d4415631fe4c4f6e15e036c63a1e68f55d45d1efdace11034f7d8f025ca" gracePeriod=30
Jul 28 01:48:33.919418 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:33.919385    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m"]
Jul 28 01:48:33.919721 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:33.919672    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m" podUID="b0e797f2-0eae-489c-997b-6a7cda092b82" containerName="payload-pre-processing" containerID="cri-o://3090685f7dc1360d6169e637a12628d2d0753f2eb64966a402eac4bfb6b8bde0" gracePeriod=30
Jul 28 01:48:34.182956 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.182928    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g"
Jul 28 01:48:34.206357 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.206302    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x"
Jul 28 01:48:34.214536 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.214509    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m"
Jul 28 01:48:34.242197 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.242160    2578 generic.go:356] "Generic (PLEG): container finished" podID="069ae680-92f6-4d0e-82ef-2933da3bd757" containerID="1a344d4415631fe4c4f6e15e036c63a1e68f55d45d1efdace11034f7d8f025ca" exitCode=0
Jul 28 01:48:34.242197 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.242230    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x"
Jul 28 01:48:34.242509 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.242262    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x" event={"ID":"069ae680-92f6-4d0e-82ef-2933da3bd757","Type":"ContainerDied","Data":"1a344d4415631fe4c4f6e15e036c63a1e68f55d45d1efdace11034f7d8f025ca"}
Jul 28 01:48:34.242509 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.242326    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x" event={"ID":"069ae680-92f6-4d0e-82ef-2933da3bd757","Type":"ContainerDied","Data":"7e5c4b12637e2e429ec333cc33790e422dd9c770b7cb7a3a562fb4e079f85b6a"}
Jul 28 01:48:34.242509 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.242359    2578 scope.go:117] "RemoveContainer" containerID="1a344d4415631fe4c4f6e15e036c63a1e68f55d45d1efdace11034f7d8f025ca"
Jul 28 01:48:34.243708 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.243687    2578 generic.go:356] "Generic (PLEG): container finished" podID="b0e797f2-0eae-489c-997b-6a7cda092b82" containerID="3090685f7dc1360d6169e637a12628d2d0753f2eb64966a402eac4bfb6b8bde0" exitCode=0
Jul 28 01:48:34.243787 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.243734    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m"
Jul 28 01:48:34.243787 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.243771    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m" event={"ID":"b0e797f2-0eae-489c-997b-6a7cda092b82","Type":"ContainerDied","Data":"3090685f7dc1360d6169e637a12628d2d0753f2eb64966a402eac4bfb6b8bde0"}
Jul 28 01:48:34.243905 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.243807    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m" event={"ID":"b0e797f2-0eae-489c-997b-6a7cda092b82","Type":"ContainerDied","Data":"c2b2b2c39ee323bbab9c8f126a0d2e5f05d4443a44f13e45f0dd48dc6ee8e181"}
Jul 28 01:48:34.245121 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.245097    2578 generic.go:356] "Generic (PLEG): container finished" podID="8b335b18-c9fa-4693-9e38-e1a1686c0315" containerID="9ef1194aba9edb60bcd55d124f2a0fd1d2b48cbc4c976190d3685923f7424b24" exitCode=0
Jul 28 01:48:34.245265 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.245149    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g"
Jul 28 01:48:34.245265 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.245184    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g" event={"ID":"8b335b18-c9fa-4693-9e38-e1a1686c0315","Type":"ContainerDied","Data":"9ef1194aba9edb60bcd55d124f2a0fd1d2b48cbc4c976190d3685923f7424b24"}
Jul 28 01:48:34.245265 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.245245    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g" event={"ID":"8b335b18-c9fa-4693-9e38-e1a1686c0315","Type":"ContainerDied","Data":"a4f38cd068c963ceb4d20d5d872fbb51fbae9a93059455225d5bd43b5243d41e"}
Jul 28 01:48:34.253876 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.253856    2578 scope.go:117] "RemoveContainer" containerID="1a344d4415631fe4c4f6e15e036c63a1e68f55d45d1efdace11034f7d8f025ca"
Jul 28 01:48:34.254153 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:48:34.254135    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a344d4415631fe4c4f6e15e036c63a1e68f55d45d1efdace11034f7d8f025ca\": container with ID starting with 1a344d4415631fe4c4f6e15e036c63a1e68f55d45d1efdace11034f7d8f025ca not found: ID does not exist" containerID="1a344d4415631fe4c4f6e15e036c63a1e68f55d45d1efdace11034f7d8f025ca"
Jul 28 01:48:34.254203 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.254162    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a344d4415631fe4c4f6e15e036c63a1e68f55d45d1efdace11034f7d8f025ca"} err="failed to get container status \"1a344d4415631fe4c4f6e15e036c63a1e68f55d45d1efdace11034f7d8f025ca\": rpc error: code = NotFound desc = could not find container \"1a344d4415631fe4c4f6e15e036c63a1e68f55d45d1efdace11034f7d8f025ca\": container with ID starting with 1a344d4415631fe4c4f6e15e036c63a1e68f55d45d1efdace11034f7d8f025ca not found: ID does not exist"
Jul 28 01:48:34.254203 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.254179    2578 scope.go:117] "RemoveContainer" containerID="3090685f7dc1360d6169e637a12628d2d0753f2eb64966a402eac4bfb6b8bde0"
Jul 28 01:48:34.262900 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.262880    2578 scope.go:117] "RemoveContainer" containerID="3090685f7dc1360d6169e637a12628d2d0753f2eb64966a402eac4bfb6b8bde0"
Jul 28 01:48:34.263242 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:48:34.263202    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3090685f7dc1360d6169e637a12628d2d0753f2eb64966a402eac4bfb6b8bde0\": container with ID starting with 3090685f7dc1360d6169e637a12628d2d0753f2eb64966a402eac4bfb6b8bde0 not found: ID does not exist" containerID="3090685f7dc1360d6169e637a12628d2d0753f2eb64966a402eac4bfb6b8bde0"
Jul 28 01:48:34.263338 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.263254    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3090685f7dc1360d6169e637a12628d2d0753f2eb64966a402eac4bfb6b8bde0"} err="failed to get container status \"3090685f7dc1360d6169e637a12628d2d0753f2eb64966a402eac4bfb6b8bde0\": rpc error: code = NotFound desc = could not find container \"3090685f7dc1360d6169e637a12628d2d0753f2eb64966a402eac4bfb6b8bde0\": container with ID starting with 3090685f7dc1360d6169e637a12628d2d0753f2eb64966a402eac4bfb6b8bde0 not found: ID does not exist"
Jul 28 01:48:34.263338 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.263278    2578 scope.go:117] "RemoveContainer" containerID="9ef1194aba9edb60bcd55d124f2a0fd1d2b48cbc4c976190d3685923f7424b24"
Jul 28 01:48:34.271950 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.271932    2578 scope.go:117] "RemoveContainer" containerID="9ef1194aba9edb60bcd55d124f2a0fd1d2b48cbc4c976190d3685923f7424b24"
Jul 28 01:48:34.272253 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:48:34.272231    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ef1194aba9edb60bcd55d124f2a0fd1d2b48cbc4c976190d3685923f7424b24\": container with ID starting with 9ef1194aba9edb60bcd55d124f2a0fd1d2b48cbc4c976190d3685923f7424b24 not found: ID does not exist" containerID="9ef1194aba9edb60bcd55d124f2a0fd1d2b48cbc4c976190d3685923f7424b24"
Jul 28 01:48:34.272339 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.272265    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef1194aba9edb60bcd55d124f2a0fd1d2b48cbc4c976190d3685923f7424b24"} err="failed to get container status \"9ef1194aba9edb60bcd55d124f2a0fd1d2b48cbc4c976190d3685923f7424b24\": rpc error: code = NotFound desc = could not find container \"9ef1194aba9edb60bcd55d124f2a0fd1d2b48cbc4c976190d3685923f7424b24\": container with ID starting with 9ef1194aba9edb60bcd55d124f2a0fd1d2b48cbc4c976190d3685923f7424b24 not found: ID does not exist"
Jul 28 01:48:34.328630 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.328597    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwjq8\" (UniqueName: \"kubernetes.io/projected/b0e797f2-0eae-489c-997b-6a7cda092b82-kube-api-access-qwjq8\") pod \"b0e797f2-0eae-489c-997b-6a7cda092b82\" (UID: \"b0e797f2-0eae-489c-997b-6a7cda092b82\") "
Jul 28 01:48:34.328829 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.328651    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b335b18-c9fa-4693-9e38-e1a1686c0315-openshift-service-ca\") pod \"8b335b18-c9fa-4693-9e38-e1a1686c0315\" (UID: \"8b335b18-c9fa-4693-9e38-e1a1686c0315\") "
Jul 28 01:48:34.328829 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.328683    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdr2r\" (UniqueName: \"kubernetes.io/projected/8b335b18-c9fa-4693-9e38-e1a1686c0315-kube-api-access-kdr2r\") pod \"8b335b18-c9fa-4693-9e38-e1a1686c0315\" (UID: \"8b335b18-c9fa-4693-9e38-e1a1686c0315\") "
Jul 28 01:48:34.328829 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.328700    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b335b18-c9fa-4693-9e38-e1a1686c0315-maas-api-tls\") pod \"8b335b18-c9fa-4693-9e38-e1a1686c0315\" (UID: \"8b335b18-c9fa-4693-9e38-e1a1686c0315\") "
Jul 28 01:48:34.328829 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.328752    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/069ae680-92f6-4d0e-82ef-2933da3bd757-plugins-config-volume\") pod \"069ae680-92f6-4d0e-82ef-2933da3bd757\" (UID: \"069ae680-92f6-4d0e-82ef-2933da3bd757\") "
Jul 28 01:48:34.328829 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.328798    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0e797f2-0eae-489c-997b-6a7cda092b82-plugins-config-volume\") pod \"b0e797f2-0eae-489c-997b-6a7cda092b82\" (UID: \"b0e797f2-0eae-489c-997b-6a7cda092b82\") "
Jul 28 01:48:34.328829 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.328827    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-586pn\" (UniqueName: \"kubernetes.io/projected/069ae680-92f6-4d0e-82ef-2933da3bd757-kube-api-access-586pn\") pod \"069ae680-92f6-4d0e-82ef-2933da3bd757\" (UID: \"069ae680-92f6-4d0e-82ef-2933da3bd757\") "
Jul 28 01:48:34.329198 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.329165    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b335b18-c9fa-4693-9e38-e1a1686c0315-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8b335b18-c9fa-4693-9e38-e1a1686c0315" (UID: "8b335b18-c9fa-4693-9e38-e1a1686c0315"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:48:34.329320 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.329293    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/069ae680-92f6-4d0e-82ef-2933da3bd757-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "069ae680-92f6-4d0e-82ef-2933da3bd757" (UID: "069ae680-92f6-4d0e-82ef-2933da3bd757"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:48:34.329364 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.329308    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0e797f2-0eae-489c-997b-6a7cda092b82-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b0e797f2-0eae-489c-997b-6a7cda092b82" (UID: "b0e797f2-0eae-489c-997b-6a7cda092b82"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:48:34.330852 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.330832    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b335b18-c9fa-4693-9e38-e1a1686c0315-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8b335b18-c9fa-4693-9e38-e1a1686c0315" (UID: "8b335b18-c9fa-4693-9e38-e1a1686c0315"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:48:34.330972 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.330954    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/069ae680-92f6-4d0e-82ef-2933da3bd757-kube-api-access-586pn" (OuterVolumeSpecName: "kube-api-access-586pn") pod "069ae680-92f6-4d0e-82ef-2933da3bd757" (UID: "069ae680-92f6-4d0e-82ef-2933da3bd757"). InnerVolumeSpecName "kube-api-access-586pn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:48:34.331269 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.331241    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b335b18-c9fa-4693-9e38-e1a1686c0315-kube-api-access-kdr2r" (OuterVolumeSpecName: "kube-api-access-kdr2r") pod "8b335b18-c9fa-4693-9e38-e1a1686c0315" (UID: "8b335b18-c9fa-4693-9e38-e1a1686c0315"). InnerVolumeSpecName "kube-api-access-kdr2r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:48:34.331269 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.331265    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0e797f2-0eae-489c-997b-6a7cda092b82-kube-api-access-qwjq8" (OuterVolumeSpecName: "kube-api-access-qwjq8") pod "b0e797f2-0eae-489c-997b-6a7cda092b82" (UID: "b0e797f2-0eae-489c-997b-6a7cda092b82"). InnerVolumeSpecName "kube-api-access-qwjq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:48:34.430108 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.430075    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b335b18-c9fa-4693-9e38-e1a1686c0315-openshift-service-ca\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:48:34.430108 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.430103    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdr2r\" (UniqueName: \"kubernetes.io/projected/8b335b18-c9fa-4693-9e38-e1a1686c0315-kube-api-access-kdr2r\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:48:34.430305 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.430116    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b335b18-c9fa-4693-9e38-e1a1686c0315-maas-api-tls\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:48:34.430305 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.430128    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/069ae680-92f6-4d0e-82ef-2933da3bd757-plugins-config-volume\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:48:34.430305 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.430142    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0e797f2-0eae-489c-997b-6a7cda092b82-plugins-config-volume\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:48:34.430305 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.430154    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-586pn\" (UniqueName: \"kubernetes.io/projected/069ae680-92f6-4d0e-82ef-2933da3bd757-kube-api-access-586pn\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:48:34.430305 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.430165    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwjq8\" (UniqueName: \"kubernetes.io/projected/b0e797f2-0eae-489c-997b-6a7cda092b82-kube-api-access-qwjq8\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:48:34.561256 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.561197    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x"]
Jul 28 01:48:34.573109 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.573082    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f67c19-cf6c5f756-x9s8x"]
Jul 28 01:48:34.585640 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.585606    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g"]
Jul 28 01:48:34.588514 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.588484    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f67c19-54f7f5d5c5-9b79g"]
Jul 28 01:48:34.596337 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.596306    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m"]
Jul 28 01:48:34.599919 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:34.599892    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-f67c19-f9bcc4b9b-qzp7m"]
Jul 28 01:48:36.403126 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:36.403087    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="069ae680-92f6-4d0e-82ef-2933da3bd757" path="/var/lib/kubelet/pods/069ae680-92f6-4d0e-82ef-2933da3bd757/volumes"
Jul 28 01:48:36.403535 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:36.403519    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b335b18-c9fa-4693-9e38-e1a1686c0315" path="/var/lib/kubelet/pods/8b335b18-c9fa-4693-9e38-e1a1686c0315/volumes"
Jul 28 01:48:36.403892 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:36.403878    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0e797f2-0eae-489c-997b-6a7cda092b82" path="/var/lib/kubelet/pods/b0e797f2-0eae-489c-997b-6a7cda092b82/volumes"
Jul 28 01:48:49.038323 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.038284    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j"]
Jul 28 01:48:49.038718 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.038681    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0e797f2-0eae-489c-997b-6a7cda092b82" containerName="payload-pre-processing"
Jul 28 01:48:49.038718 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.038692    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0e797f2-0eae-489c-997b-6a7cda092b82" containerName="payload-pre-processing"
Jul 28 01:48:49.038718 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.038705    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="069ae680-92f6-4d0e-82ef-2933da3bd757" containerName="payload-processing"
Jul 28 01:48:49.038718 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.038712    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="069ae680-92f6-4d0e-82ef-2933da3bd757" containerName="payload-processing"
Jul 28 01:48:49.038849 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.038721    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac8bc5e8-bb27-4e57-9a10-fc35e2753605" containerName="main"
Jul 28 01:48:49.038849 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.038726    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac8bc5e8-bb27-4e57-9a10-fc35e2753605" containerName="main"
Jul 28 01:48:49.038849 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.038737    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6" containerName="payload-processing"
Jul 28 01:48:49.038849 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.038742    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6" containerName="payload-processing"
Jul 28 01:48:49.038849 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.038750    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6b500ee-e3b5-409c-967d-c4426ba9dfa2" containerName="payload-pre-processing"
Jul 28 01:48:49.038849 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.038755    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6b500ee-e3b5-409c-967d-c4426ba9dfa2" containerName="payload-pre-processing"
Jul 28 01:48:49.038849 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.038766    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57e57ec2-65c8-4073-a229-74741e1a1030" containerName="maas-api"
Jul 28 01:48:49.038849 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.038771    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="57e57ec2-65c8-4073-a229-74741e1a1030" containerName="maas-api"
Jul 28 01:48:49.038849 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.038777    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b335b18-c9fa-4693-9e38-e1a1686c0315" containerName="maas-api"
Jul 28 01:48:49.038849 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.038782    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b335b18-c9fa-4693-9e38-e1a1686c0315" containerName="maas-api"
Jul 28 01:48:49.038849 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.038844    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6b500ee-e3b5-409c-967d-c4426ba9dfa2" containerName="payload-pre-processing"
Jul 28 01:48:49.038849 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.038850    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="57e57ec2-65c8-4073-a229-74741e1a1030" containerName="maas-api"
Jul 28 01:48:49.039189 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.038857    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="34a4cd1e-9790-4c59-bd6e-ddc427a8f2e6" containerName="payload-processing"
Jul 28 01:48:49.039189 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.038865    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac8bc5e8-bb27-4e57-9a10-fc35e2753605" containerName="main"
Jul 28 01:48:49.039189 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.038872    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="069ae680-92f6-4d0e-82ef-2933da3bd757" containerName="payload-processing"
Jul 28 01:48:49.039189 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.038878    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0e797f2-0eae-489c-997b-6a7cda092b82" containerName="payload-pre-processing"
Jul 28 01:48:49.039189 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.038884    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b335b18-c9fa-4693-9e38-e1a1686c0315" containerName="maas-api"
Jul 28 01:48:49.043692 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.043673    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j"
Jul 28 01:48:49.046323 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.046299    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lrljm\""
Jul 28 01:48:49.046655 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.046636    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 01:48:49.047321 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.047302    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-64a682-serving-cert\""
Jul 28 01:48:49.047387 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.047322    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 01:48:49.057897 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.057816    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j"]
Jul 28 01:48:49.069777 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.069747    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81d11de9-e12c-4ab5-87ab-5c257d7b2984-openshift-service-ca\") pod \"maas-api-e2e-api-a-64a682-6755586778-ztq9j\" (UID: \"81d11de9-e12c-4ab5-87ab-5c257d7b2984\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j"
Jul 28 01:48:49.069965 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.069803    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81d11de9-e12c-4ab5-87ab-5c257d7b2984-maas-api-tls\") pod \"maas-api-e2e-api-a-64a682-6755586778-ztq9j\" (UID: \"81d11de9-e12c-4ab5-87ab-5c257d7b2984\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j"
Jul 28 01:48:49.069965 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.069856    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hfrz\" (UniqueName: \"kubernetes.io/projected/81d11de9-e12c-4ab5-87ab-5c257d7b2984-kube-api-access-8hfrz\") pod \"maas-api-e2e-api-a-64a682-6755586778-ztq9j\" (UID: \"81d11de9-e12c-4ab5-87ab-5c257d7b2984\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j"
Jul 28 01:48:49.171244 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.171193    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81d11de9-e12c-4ab5-87ab-5c257d7b2984-maas-api-tls\") pod \"maas-api-e2e-api-a-64a682-6755586778-ztq9j\" (UID: \"81d11de9-e12c-4ab5-87ab-5c257d7b2984\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j"
Jul 28 01:48:49.171408 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.171274    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hfrz\" (UniqueName: \"kubernetes.io/projected/81d11de9-e12c-4ab5-87ab-5c257d7b2984-kube-api-access-8hfrz\") pod \"maas-api-e2e-api-a-64a682-6755586778-ztq9j\" (UID: \"81d11de9-e12c-4ab5-87ab-5c257d7b2984\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j"
Jul 28 01:48:49.171408 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:48:49.171329    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-serving-cert: secret "maas-api-e2e-api-a-64a682-serving-cert" not found
Jul 28 01:48:49.171408 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.171386    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81d11de9-e12c-4ab5-87ab-5c257d7b2984-openshift-service-ca\") pod \"maas-api-e2e-api-a-64a682-6755586778-ztq9j\" (UID: \"81d11de9-e12c-4ab5-87ab-5c257d7b2984\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j"
Jul 28 01:48:49.171589 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:48:49.171413    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81d11de9-e12c-4ab5-87ab-5c257d7b2984-maas-api-tls podName:81d11de9-e12c-4ab5-87ab-5c257d7b2984 nodeName:}" failed. No retries permitted until 2026-07-28 01:48:49.671389149 +0000 UTC m=+3299.836197254 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/81d11de9-e12c-4ab5-87ab-5c257d7b2984-maas-api-tls") pod "maas-api-e2e-api-a-64a682-6755586778-ztq9j" (UID: "81d11de9-e12c-4ab5-87ab-5c257d7b2984") : secret "maas-api-e2e-api-a-64a682-serving-cert" not found
Jul 28 01:48:49.171945 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.171926    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81d11de9-e12c-4ab5-87ab-5c257d7b2984-openshift-service-ca\") pod \"maas-api-e2e-api-a-64a682-6755586778-ztq9j\" (UID: \"81d11de9-e12c-4ab5-87ab-5c257d7b2984\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j"
Jul 28 01:48:49.180067 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.180032    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hfrz\" (UniqueName: \"kubernetes.io/projected/81d11de9-e12c-4ab5-87ab-5c257d7b2984-kube-api-access-8hfrz\") pod \"maas-api-e2e-api-a-64a682-6755586778-ztq9j\" (UID: \"81d11de9-e12c-4ab5-87ab-5c257d7b2984\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j"
Jul 28 01:48:49.677650 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.677614    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81d11de9-e12c-4ab5-87ab-5c257d7b2984-maas-api-tls\") pod \"maas-api-e2e-api-a-64a682-6755586778-ztq9j\" (UID: \"81d11de9-e12c-4ab5-87ab-5c257d7b2984\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j"
Jul 28 01:48:49.677835 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:48:49.677781    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-serving-cert: secret "maas-api-e2e-api-a-64a682-serving-cert" not found
Jul 28 01:48:49.677876 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:48:49.677855    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81d11de9-e12c-4ab5-87ab-5c257d7b2984-maas-api-tls podName:81d11de9-e12c-4ab5-87ab-5c257d7b2984 nodeName:}" failed. No retries permitted until 2026-07-28 01:48:50.677837973 +0000 UTC m=+3300.842646061 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/81d11de9-e12c-4ab5-87ab-5c257d7b2984-maas-api-tls") pod "maas-api-e2e-api-a-64a682-6755586778-ztq9j" (UID: "81d11de9-e12c-4ab5-87ab-5c257d7b2984") : secret "maas-api-e2e-api-a-64a682-serving-cert" not found
Jul 28 01:48:49.904691 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.904644    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt"]
Jul 28 01:48:49.917104 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.916755    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt"
Jul 28 01:48:49.919471 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.919444    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-64a682\""
Jul 28 01:48:49.920662 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.920636    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt"]
Jul 28 01:48:49.935569 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.935499    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-64a682-cf87788c8-flb78"]
Jul 28 01:48:49.939417 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.939391    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-64a682-cf87788c8-flb78"
Jul 28 01:48:49.942143 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.942120    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-64a682-dockercfg-v95tg\""
Jul 28 01:48:49.947631 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.947571    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-64a682-cf87788c8-flb78"]
Jul 28 01:48:49.980229 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.980185    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/266fa083-3719-4737-9452-f8063b161c51-plugins-config-volume\") pod \"payload-processing-e2e-api-a-64a682-cf87788c8-flb78\" (UID: \"266fa083-3719-4737-9452-f8063b161c51\") " pod="openshift-ingress/payload-processing-e2e-api-a-64a682-cf87788c8-flb78"
Jul 28 01:48:49.980404 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.980276    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdhvp\" (UniqueName: \"kubernetes.io/projected/266fa083-3719-4737-9452-f8063b161c51-kube-api-access-rdhvp\") pod \"payload-processing-e2e-api-a-64a682-cf87788c8-flb78\" (UID: \"266fa083-3719-4737-9452-f8063b161c51\") " pod="openshift-ingress/payload-processing-e2e-api-a-64a682-cf87788c8-flb78"
Jul 28 01:48:49.980404 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.980307    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f2798e2-bc38-4b53-a44a-e628dc742376-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt\" (UID: \"2f2798e2-bc38-4b53-a44a-e628dc742376\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt"
Jul 28 01:48:49.980404 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:49.980347    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv68q\" (UniqueName: \"kubernetes.io/projected/2f2798e2-bc38-4b53-a44a-e628dc742376-kube-api-access-dv68q\") pod \"payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt\" (UID: \"2f2798e2-bc38-4b53-a44a-e628dc742376\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt"
Jul 28 01:48:50.081651 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:50.081608    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/266fa083-3719-4737-9452-f8063b161c51-plugins-config-volume\") pod \"payload-processing-e2e-api-a-64a682-cf87788c8-flb78\" (UID: \"266fa083-3719-4737-9452-f8063b161c51\") " pod="openshift-ingress/payload-processing-e2e-api-a-64a682-cf87788c8-flb78"
Jul 28 01:48:50.082123 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:50.081667    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdhvp\" (UniqueName: \"kubernetes.io/projected/266fa083-3719-4737-9452-f8063b161c51-kube-api-access-rdhvp\") pod \"payload-processing-e2e-api-a-64a682-cf87788c8-flb78\" (UID: \"266fa083-3719-4737-9452-f8063b161c51\") " pod="openshift-ingress/payload-processing-e2e-api-a-64a682-cf87788c8-flb78"
Jul 28 01:48:50.082123 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:50.081690    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f2798e2-bc38-4b53-a44a-e628dc742376-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt\" (UID: \"2f2798e2-bc38-4b53-a44a-e628dc742376\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt"
Jul 28 01:48:50.082123 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:50.081721    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv68q\" (UniqueName: \"kubernetes.io/projected/2f2798e2-bc38-4b53-a44a-e628dc742376-kube-api-access-dv68q\") pod \"payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt\" (UID: \"2f2798e2-bc38-4b53-a44a-e628dc742376\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt"
Jul 28 01:48:50.082655 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:50.082628    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f2798e2-bc38-4b53-a44a-e628dc742376-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt\" (UID: \"2f2798e2-bc38-4b53-a44a-e628dc742376\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt"
Jul 28 01:48:50.082765 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:50.082704    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/266fa083-3719-4737-9452-f8063b161c51-plugins-config-volume\") pod \"payload-processing-e2e-api-a-64a682-cf87788c8-flb78\" (UID: \"266fa083-3719-4737-9452-f8063b161c51\") " pod="openshift-ingress/payload-processing-e2e-api-a-64a682-cf87788c8-flb78"
Jul 28 01:48:50.095334 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:50.095303    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv68q\" (UniqueName: \"kubernetes.io/projected/2f2798e2-bc38-4b53-a44a-e628dc742376-kube-api-access-dv68q\") pod \"payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt\" (UID: \"2f2798e2-bc38-4b53-a44a-e628dc742376\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt"
Jul 28 01:48:50.096649 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:50.096629    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdhvp\" (UniqueName: \"kubernetes.io/projected/266fa083-3719-4737-9452-f8063b161c51-kube-api-access-rdhvp\") pod \"payload-processing-e2e-api-a-64a682-cf87788c8-flb78\" (UID: \"266fa083-3719-4737-9452-f8063b161c51\") " pod="openshift-ingress/payload-processing-e2e-api-a-64a682-cf87788c8-flb78"
Jul 28 01:48:50.229784 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:50.229562    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt"
Jul 28 01:48:50.254995 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:50.254963    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-64a682-dockercfg-v95tg\""
Jul 28 01:48:50.263246 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:50.263187    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-64a682-cf87788c8-flb78"
Jul 28 01:48:50.409879 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:50.409849    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt"]
Jul 28 01:48:50.412377 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:48:50.412330    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f2798e2_bc38_4b53_a44a_e628dc742376.slice/crio-f043a8cbec340afa961a41581c42519179e12bc067e1927d2ef9b1646de184ec WatchSource:0}: Error finding container f043a8cbec340afa961a41581c42519179e12bc067e1927d2ef9b1646de184ec: Status 404 returned error can't find the container with id f043a8cbec340afa961a41581c42519179e12bc067e1927d2ef9b1646de184ec
Jul 28 01:48:50.445234 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:50.445194    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-64a682-cf87788c8-flb78"]
Jul 28 01:48:50.457408 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:48:50.457381    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod266fa083_3719_4737_9452_f8063b161c51.slice/crio-709ce8d22e6c7a934d9e747d26160b14a6349fc31734563d89dc4c973fa1baac WatchSource:0}: Error finding container 709ce8d22e6c7a934d9e747d26160b14a6349fc31734563d89dc4c973fa1baac: Status 404 returned error can't find the container with id 709ce8d22e6c7a934d9e747d26160b14a6349fc31734563d89dc4c973fa1baac
Jul 28 01:48:50.688288 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:50.688251    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81d11de9-e12c-4ab5-87ab-5c257d7b2984-maas-api-tls\") pod \"maas-api-e2e-api-a-64a682-6755586778-ztq9j\" (UID: \"81d11de9-e12c-4ab5-87ab-5c257d7b2984\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j"
Jul 28 01:48:50.690739 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:50.690719    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81d11de9-e12c-4ab5-87ab-5c257d7b2984-maas-api-tls\") pod \"maas-api-e2e-api-a-64a682-6755586778-ztq9j\" (UID: \"81d11de9-e12c-4ab5-87ab-5c257d7b2984\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j"
Jul 28 01:48:50.809830 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:50.809743    2578 scope.go:117] "RemoveContainer" containerID="a8c75e3d38aa7f6fd0a63daafc716f9dc1ead20737e24d7c0cc87a6ef579dc43"
Jul 28 01:48:50.874565 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:50.874529    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lrljm\""
Jul 28 01:48:50.884296 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:50.884263    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j"
Jul 28 01:48:51.049869 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:48:51.049836    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81d11de9_e12c_4ab5_87ab_5c257d7b2984.slice/crio-b1a86148ed5e40941b97afd53a898348425bdfc52887f089e46675284019b890 WatchSource:0}: Error finding container b1a86148ed5e40941b97afd53a898348425bdfc52887f089e46675284019b890: Status 404 returned error can't find the container with id b1a86148ed5e40941b97afd53a898348425bdfc52887f089e46675284019b890
Jul 28 01:48:51.051263 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:51.051245    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 01:48:51.065102 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:51.065030    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j"]
Jul 28 01:48:51.347867 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:51.347773    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j" event={"ID":"81d11de9-e12c-4ab5-87ab-5c257d7b2984","Type":"ContainerStarted","Data":"b1a86148ed5e40941b97afd53a898348425bdfc52887f089e46675284019b890"}
Jul 28 01:48:51.349293 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:51.349267    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt" event={"ID":"2f2798e2-bc38-4b53-a44a-e628dc742376","Type":"ContainerStarted","Data":"a569ecace204f774156f6ac22e943553cf0022683320df2b0cc3a62a7ca6a7db"}
Jul 28 01:48:51.349413 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:51.349312    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt" event={"ID":"2f2798e2-bc38-4b53-a44a-e628dc742376","Type":"ContainerStarted","Data":"f043a8cbec340afa961a41581c42519179e12bc067e1927d2ef9b1646de184ec"}
Jul 28 01:48:51.349413 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:51.349386    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt"
Jul 28 01:48:51.353490 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:51.353465    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-64a682-cf87788c8-flb78" event={"ID":"266fa083-3719-4737-9452-f8063b161c51","Type":"ContainerStarted","Data":"6c83188b898ee7021dad6c5e2813d9bd27a29cce0e0d1bb5dd19a2990e4f8fb2"}
Jul 28 01:48:51.353604 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:51.353516    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-64a682-cf87788c8-flb78"
Jul 28 01:48:51.353604 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:51.353542    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-64a682-cf87788c8-flb78" event={"ID":"266fa083-3719-4737-9452-f8063b161c51","Type":"ContainerStarted","Data":"709ce8d22e6c7a934d9e747d26160b14a6349fc31734563d89dc4c973fa1baac"}
Jul 28 01:48:51.382174 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:51.382119    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt" podStartSLOduration=2.3821045659999998 podStartE2EDuration="2.382104566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:48:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:48:51.379450344 +0000 UTC m=+3301.544258452" watchObservedRunningTime="2026-07-28 01:48:51.382104566 +0000 UTC m=+3301.546912673"
Jul 28 01:48:51.412774 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:51.412719    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-64a682-cf87788c8-flb78" podStartSLOduration=2.412701171 podStartE2EDuration="2.412701171s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:48:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:48:51.41187373 +0000 UTC m=+3301.576681849" watchObservedRunningTime="2026-07-28 01:48:51.412701171 +0000 UTC m=+3301.577509277"
Jul 28 01:48:52.359820 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:52.359778    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j" event={"ID":"81d11de9-e12c-4ab5-87ab-5c257d7b2984","Type":"ContainerStarted","Data":"61003d429dad3e3a0a259e55fc9a8a0ca1d63942c97ea879b9c83f6af8e6d07c"}
Jul 28 01:48:52.360302 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:52.359952    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j"
Jul 28 01:48:52.378138 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:52.378085    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j" podStartSLOduration=2.7249270709999998 podStartE2EDuration="3.37806839s" totalImagesPullingTime="653.141319ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:48:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:48:52.376429669 +0000 UTC m=+3302.541237777" watchObservedRunningTime="2026-07-28 01:48:52.37806839 +0000 UTC m=+3302.542876497"
Jul 28 01:48:57.001078 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.001043    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-559478b5-2f9jk"]
Jul 28 01:48:57.004823 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.004799    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-559478b5-2f9jk"
Jul 28 01:48:57.007731 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.007706    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-3940b7-serving-cert\""
Jul 28 01:48:57.015442 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.015413    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-559478b5-2f9jk"]
Jul 28 01:48:57.050979 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.050944    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/762a4b61-0389-4156-8b55-8a72a44af553-openshift-service-ca\") pod \"maas-api-e2e-api-b-3940b7-559478b5-2f9jk\" (UID: \"762a4b61-0389-4156-8b55-8a72a44af553\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-559478b5-2f9jk"
Jul 28 01:48:57.050979 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.050987    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk92q\" (UniqueName: \"kubernetes.io/projected/762a4b61-0389-4156-8b55-8a72a44af553-kube-api-access-pk92q\") pod \"maas-api-e2e-api-b-3940b7-559478b5-2f9jk\" (UID: \"762a4b61-0389-4156-8b55-8a72a44af553\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-559478b5-2f9jk"
Jul 28 01:48:57.051174 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.051092    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/762a4b61-0389-4156-8b55-8a72a44af553-maas-api-tls\") pod \"maas-api-e2e-api-b-3940b7-559478b5-2f9jk\" (UID: \"762a4b61-0389-4156-8b55-8a72a44af553\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-559478b5-2f9jk"
Jul 28 01:48:57.151702 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.151667    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/762a4b61-0389-4156-8b55-8a72a44af553-openshift-service-ca\") pod \"maas-api-e2e-api-b-3940b7-559478b5-2f9jk\" (UID: \"762a4b61-0389-4156-8b55-8a72a44af553\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-559478b5-2f9jk"
Jul 28 01:48:57.151873 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.151710    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk92q\" (UniqueName: \"kubernetes.io/projected/762a4b61-0389-4156-8b55-8a72a44af553-kube-api-access-pk92q\") pod \"maas-api-e2e-api-b-3940b7-559478b5-2f9jk\" (UID: \"762a4b61-0389-4156-8b55-8a72a44af553\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-559478b5-2f9jk"
Jul 28 01:48:57.151873 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.151762    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/762a4b61-0389-4156-8b55-8a72a44af553-maas-api-tls\") pod \"maas-api-e2e-api-b-3940b7-559478b5-2f9jk\" (UID: \"762a4b61-0389-4156-8b55-8a72a44af553\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-559478b5-2f9jk"
Jul 28 01:48:57.151970 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:48:57.151900    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-serving-cert: secret "maas-api-e2e-api-b-3940b7-serving-cert" not found
Jul 28 01:48:57.152005 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:48:57.151973    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/762a4b61-0389-4156-8b55-8a72a44af553-maas-api-tls podName:762a4b61-0389-4156-8b55-8a72a44af553 nodeName:}" failed. No retries permitted until 2026-07-28 01:48:57.651951593 +0000 UTC m=+3307.816759695 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/762a4b61-0389-4156-8b55-8a72a44af553-maas-api-tls") pod "maas-api-e2e-api-b-3940b7-559478b5-2f9jk" (UID: "762a4b61-0389-4156-8b55-8a72a44af553") : secret "maas-api-e2e-api-b-3940b7-serving-cert" not found
Jul 28 01:48:57.152336 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.152315    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/762a4b61-0389-4156-8b55-8a72a44af553-openshift-service-ca\") pod \"maas-api-e2e-api-b-3940b7-559478b5-2f9jk\" (UID: \"762a4b61-0389-4156-8b55-8a72a44af553\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-559478b5-2f9jk"
Jul 28 01:48:57.160832 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.160801    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk92q\" (UniqueName: \"kubernetes.io/projected/762a4b61-0389-4156-8b55-8a72a44af553-kube-api-access-pk92q\") pod \"maas-api-e2e-api-b-3940b7-559478b5-2f9jk\" (UID: \"762a4b61-0389-4156-8b55-8a72a44af553\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-559478b5-2f9jk"
Jul 28 01:48:57.656793 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.656760    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/762a4b61-0389-4156-8b55-8a72a44af553-maas-api-tls\") pod \"maas-api-e2e-api-b-3940b7-559478b5-2f9jk\" (UID: \"762a4b61-0389-4156-8b55-8a72a44af553\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-559478b5-2f9jk"
Jul 28 01:48:57.659306 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.659268    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/762a4b61-0389-4156-8b55-8a72a44af553-maas-api-tls\") pod \"maas-api-e2e-api-b-3940b7-559478b5-2f9jk\" (UID: \"762a4b61-0389-4156-8b55-8a72a44af553\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-559478b5-2f9jk"
Jul 28 01:48:57.766626 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.766583    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-3940b7-758c87d88-bksxm"]
Jul 28 01:48:57.770747 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.770722    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-3940b7-758c87d88-bksxm"
Jul 28 01:48:57.773278 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.773252    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-3940b7\""
Jul 28 01:48:57.784101 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.784072    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-3940b7-758c87d88-bksxm"]
Jul 28 01:48:57.839246 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.839188    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv"]
Jul 28 01:48:57.842843 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.842822    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv"
Jul 28 01:48:57.845254 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.845232    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-3940b7-dockercfg-2ffwj\""
Jul 28 01:48:57.852957 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.852924    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv"]
Jul 28 01:48:57.858464 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.858435    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/004d148c-9a7e-442e-a534-56f0b4b8c299-plugins-config-volume\") pod \"payload-processing-e2e-api-b-3940b7-758c87d88-bksxm\" (UID: \"004d148c-9a7e-442e-a534-56f0b4b8c299\") " pod="openshift-ingress/payload-processing-e2e-api-b-3940b7-758c87d88-bksxm"
Jul 28 01:48:57.858611 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.858498    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgpb5\" (UniqueName: \"kubernetes.io/projected/004d148c-9a7e-442e-a534-56f0b4b8c299-kube-api-access-jgpb5\") pod \"payload-processing-e2e-api-b-3940b7-758c87d88-bksxm\" (UID: \"004d148c-9a7e-442e-a534-56f0b4b8c299\") " pod="openshift-ingress/payload-processing-e2e-api-b-3940b7-758c87d88-bksxm"
Jul 28 01:48:57.917513 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.917424    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-559478b5-2f9jk"
Jul 28 01:48:57.960353 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.959691    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/004d148c-9a7e-442e-a534-56f0b4b8c299-plugins-config-volume\") pod \"payload-processing-e2e-api-b-3940b7-758c87d88-bksxm\" (UID: \"004d148c-9a7e-442e-a534-56f0b4b8c299\") " pod="openshift-ingress/payload-processing-e2e-api-b-3940b7-758c87d88-bksxm"
Jul 28 01:48:57.960542 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.960507    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgpb5\" (UniqueName: \"kubernetes.io/projected/004d148c-9a7e-442e-a534-56f0b4b8c299-kube-api-access-jgpb5\") pod \"payload-processing-e2e-api-b-3940b7-758c87d88-bksxm\" (UID: \"004d148c-9a7e-442e-a534-56f0b4b8c299\") " pod="openshift-ingress/payload-processing-e2e-api-b-3940b7-758c87d88-bksxm"
Jul 28 01:48:57.960612 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.960546    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr2kg\" (UniqueName: \"kubernetes.io/projected/4c1eeb73-270e-4ba2-bce2-858ade657ee2-kube-api-access-fr2kg\") pod \"payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv\" (UID: \"4c1eeb73-270e-4ba2-bce2-858ade657ee2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv"
Jul 28 01:48:57.960612 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.960570    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/004d148c-9a7e-442e-a534-56f0b4b8c299-plugins-config-volume\") pod \"payload-processing-e2e-api-b-3940b7-758c87d88-bksxm\" (UID: \"004d148c-9a7e-442e-a534-56f0b4b8c299\") " pod="openshift-ingress/payload-processing-e2e-api-b-3940b7-758c87d88-bksxm"
Jul 28 01:48:57.960612 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.960582    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c1eeb73-270e-4ba2-bce2-858ade657ee2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv\" (UID: \"4c1eeb73-270e-4ba2-bce2-858ade657ee2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv"
Jul 28 01:48:57.971266 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:57.970701    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgpb5\" (UniqueName: \"kubernetes.io/projected/004d148c-9a7e-442e-a534-56f0b4b8c299-kube-api-access-jgpb5\") pod \"payload-processing-e2e-api-b-3940b7-758c87d88-bksxm\" (UID: \"004d148c-9a7e-442e-a534-56f0b4b8c299\") " pod="openshift-ingress/payload-processing-e2e-api-b-3940b7-758c87d88-bksxm"
Jul 28 01:48:58.056073 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:58.056043    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-559478b5-2f9jk"]
Jul 28 01:48:58.056959 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:48:58.056936    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod762a4b61_0389_4156_8b55_8a72a44af553.slice/crio-2c1e75da2a1403a9551521568af9a3569d9d06acefa4082f355ffb12565722de WatchSource:0}: Error finding container 2c1e75da2a1403a9551521568af9a3569d9d06acefa4082f355ffb12565722de: Status 404 returned error can't find the container with id 2c1e75da2a1403a9551521568af9a3569d9d06acefa4082f355ffb12565722de
Jul 28 01:48:58.061717 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:58.061689    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr2kg\" (UniqueName: \"kubernetes.io/projected/4c1eeb73-270e-4ba2-bce2-858ade657ee2-kube-api-access-fr2kg\") pod \"payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv\" (UID: \"4c1eeb73-270e-4ba2-bce2-858ade657ee2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv"
Jul 28 01:48:58.061823 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:58.061729    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c1eeb73-270e-4ba2-bce2-858ade657ee2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv\" (UID: \"4c1eeb73-270e-4ba2-bce2-858ade657ee2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv"
Jul 28 01:48:58.062521 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:58.062501    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c1eeb73-270e-4ba2-bce2-858ade657ee2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv\" (UID: \"4c1eeb73-270e-4ba2-bce2-858ade657ee2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv"
Jul 28 01:48:58.069773 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:58.069752    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr2kg\" (UniqueName: \"kubernetes.io/projected/4c1eeb73-270e-4ba2-bce2-858ade657ee2-kube-api-access-fr2kg\") pod \"payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv\" (UID: \"4c1eeb73-270e-4ba2-bce2-858ade657ee2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv"
Jul 28 01:48:58.083610 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:58.083584    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-3940b7-758c87d88-bksxm"
Jul 28 01:48:58.155188 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:58.155158    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv"
Jul 28 01:48:58.247798 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:58.247770    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-3940b7-758c87d88-bksxm"]
Jul 28 01:48:58.248341 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:48:58.248313    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod004d148c_9a7e_442e_a534_56f0b4b8c299.slice/crio-c3dde39c38a922f225621b58e0880753d4a6e598bdb2ae8b927351a993eec2fb WatchSource:0}: Error finding container c3dde39c38a922f225621b58e0880753d4a6e598bdb2ae8b927351a993eec2fb: Status 404 returned error can't find the container with id c3dde39c38a922f225621b58e0880753d4a6e598bdb2ae8b927351a993eec2fb
Jul 28 01:48:58.317268 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:58.317241    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv"]
Jul 28 01:48:58.318396 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:48:58.318363    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c1eeb73_270e_4ba2_bce2_858ade657ee2.slice/crio-32b3ac6e3118bca8684191d2dc6637b3b3c622918d776239494534fe4094e203 WatchSource:0}: Error finding container 32b3ac6e3118bca8684191d2dc6637b3b3c622918d776239494534fe4094e203: Status 404 returned error can't find the container with id 32b3ac6e3118bca8684191d2dc6637b3b3c622918d776239494534fe4094e203
Jul 28 01:48:58.372017 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:58.371994    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j"
Jul 28 01:48:58.403477 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:58.403439    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv" event={"ID":"4c1eeb73-270e-4ba2-bce2-858ade657ee2","Type":"ContainerStarted","Data":"458d4462b861aaadc93e8ab514d975a663e48cf68f97008f743960a732312770"}
Jul 28 01:48:58.403676 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:58.403499    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv"
Jul 28 01:48:58.403676 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:58.403523    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv" event={"ID":"4c1eeb73-270e-4ba2-bce2-858ade657ee2","Type":"ContainerStarted","Data":"32b3ac6e3118bca8684191d2dc6637b3b3c622918d776239494534fe4094e203"}
Jul 28 01:48:58.403676 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:58.403544    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-559478b5-2f9jk" event={"ID":"762a4b61-0389-4156-8b55-8a72a44af553","Type":"ContainerStarted","Data":"2c1e75da2a1403a9551521568af9a3569d9d06acefa4082f355ffb12565722de"}
Jul 28 01:48:58.403676 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:58.403570    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-3940b7-758c87d88-bksxm"
Jul 28 01:48:58.403676 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:58.403592    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-3940b7-758c87d88-bksxm" event={"ID":"004d148c-9a7e-442e-a534-56f0b4b8c299","Type":"ContainerStarted","Data":"8f4577a0be08b39c5d8a434fc6f73f24bbdbf06a3a3dc4c56d27fdcde12f6f47"}
Jul 28 01:48:58.403676 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:58.403611    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-3940b7-758c87d88-bksxm" event={"ID":"004d148c-9a7e-442e-a534-56f0b4b8c299","Type":"ContainerStarted","Data":"c3dde39c38a922f225621b58e0880753d4a6e598bdb2ae8b927351a993eec2fb"}
Jul 28 01:48:58.420687 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:58.420599    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv" podStartSLOduration=1.420584248 podStartE2EDuration="1.420584248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:48:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:48:58.418458582 +0000 UTC m=+3308.583266690" watchObservedRunningTime="2026-07-28 01:48:58.420584248 +0000 UTC m=+3308.585392355"
Jul 28 01:48:58.442182 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:58.442135    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-3940b7-758c87d88-bksxm" podStartSLOduration=1.442117814 podStartE2EDuration="1.442117814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:48:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:48:58.441098459 +0000 UTC m=+3308.605906582" watchObservedRunningTime="2026-07-28 01:48:58.442117814 +0000 UTC m=+3308.606925899"
Jul 28 01:48:59.406818 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:59.406775    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-559478b5-2f9jk" event={"ID":"762a4b61-0389-4156-8b55-8a72a44af553","Type":"ContainerStarted","Data":"23574eecbb9276addd6c3b679ab8b725e16e984d8ee105ef7f6e65377ac54785"}
Jul 28 01:48:59.407292 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:59.406846    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-559478b5-2f9jk"
Jul 28 01:48:59.427696 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:48:59.427637    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-559478b5-2f9jk" podStartSLOduration=3.109876872 podStartE2EDuration="3.427619382s" totalImagesPullingTime="317.74251ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:48:59.424682669 +0000 UTC m=+3309.589490776" watchObservedRunningTime="2026-07-28 01:48:59.427619382 +0000 UTC m=+3309.592427535"
Jul 28 01:49:02.362378 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:02.362346    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-64a682-cf87788c8-flb78"
Jul 28 01:49:02.363079 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:02.363058    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt"
Jul 28 01:49:05.418255 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:05.418227    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-559478b5-2f9jk"
Jul 28 01:49:09.409063 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:09.409029    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-3940b7-758c87d88-bksxm"
Jul 28 01:49:09.409584 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:09.409104    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv"
Jul 28 01:49:24.743378 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:24.743284    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j"]
Jul 28 01:49:24.743852 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:24.743573    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j" podUID="81d11de9-e12c-4ab5-87ab-5c257d7b2984" containerName="maas-api" containerID="cri-o://61003d429dad3e3a0a259e55fc9a8a0ca1d63942c97ea879b9c83f6af8e6d07c" gracePeriod=30
Jul 28 01:49:24.811870 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:24.811831    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-64a682-cf87788c8-flb78"]
Jul 28 01:49:24.812127 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:24.812090    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-64a682-cf87788c8-flb78" podUID="266fa083-3719-4737-9452-f8063b161c51" containerName="payload-processing" containerID="cri-o://6c83188b898ee7021dad6c5e2813d9bd27a29cce0e0d1bb5dd19a2990e4f8fb2" gracePeriod=30
Jul 28 01:49:24.821897 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:24.821867    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt"]
Jul 28 01:49:24.822132 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:24.822080    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt" podUID="2f2798e2-bc38-4b53-a44a-e628dc742376" containerName="payload-pre-processing" containerID="cri-o://a569ecace204f774156f6ac22e943553cf0022683320df2b0cc3a62a7ca6a7db" gracePeriod=30
Jul 28 01:49:25.086432 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.086350    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-64a682-cf87788c8-flb78"
Jul 28 01:49:25.096309 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.096281    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j"
Jul 28 01:49:25.132725 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.132691    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hfrz\" (UniqueName: \"kubernetes.io/projected/81d11de9-e12c-4ab5-87ab-5c257d7b2984-kube-api-access-8hfrz\") pod \"81d11de9-e12c-4ab5-87ab-5c257d7b2984\" (UID: \"81d11de9-e12c-4ab5-87ab-5c257d7b2984\") "
Jul 28 01:49:25.132900 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.132743    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/266fa083-3719-4737-9452-f8063b161c51-plugins-config-volume\") pod \"266fa083-3719-4737-9452-f8063b161c51\" (UID: \"266fa083-3719-4737-9452-f8063b161c51\") "
Jul 28 01:49:25.132900 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.132814    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81d11de9-e12c-4ab5-87ab-5c257d7b2984-openshift-service-ca\") pod \"81d11de9-e12c-4ab5-87ab-5c257d7b2984\" (UID: \"81d11de9-e12c-4ab5-87ab-5c257d7b2984\") "
Jul 28 01:49:25.132900 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.132851    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdhvp\" (UniqueName: \"kubernetes.io/projected/266fa083-3719-4737-9452-f8063b161c51-kube-api-access-rdhvp\") pod \"266fa083-3719-4737-9452-f8063b161c51\" (UID: \"266fa083-3719-4737-9452-f8063b161c51\") "
Jul 28 01:49:25.132900 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.132899    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81d11de9-e12c-4ab5-87ab-5c257d7b2984-maas-api-tls\") pod \"81d11de9-e12c-4ab5-87ab-5c257d7b2984\" (UID: \"81d11de9-e12c-4ab5-87ab-5c257d7b2984\") "
Jul 28 01:49:25.133226 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.133172    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81d11de9-e12c-4ab5-87ab-5c257d7b2984-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "81d11de9-e12c-4ab5-87ab-5c257d7b2984" (UID: "81d11de9-e12c-4ab5-87ab-5c257d7b2984"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:49:25.133226 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.133173    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/266fa083-3719-4737-9452-f8063b161c51-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "266fa083-3719-4737-9452-f8063b161c51" (UID: "266fa083-3719-4737-9452-f8063b161c51"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:49:25.134985 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.134960    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/266fa083-3719-4737-9452-f8063b161c51-kube-api-access-rdhvp" (OuterVolumeSpecName: "kube-api-access-rdhvp") pod "266fa083-3719-4737-9452-f8063b161c51" (UID: "266fa083-3719-4737-9452-f8063b161c51"). InnerVolumeSpecName "kube-api-access-rdhvp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:49:25.135103 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.135019    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81d11de9-e12c-4ab5-87ab-5c257d7b2984-kube-api-access-8hfrz" (OuterVolumeSpecName: "kube-api-access-8hfrz") pod "81d11de9-e12c-4ab5-87ab-5c257d7b2984" (UID: "81d11de9-e12c-4ab5-87ab-5c257d7b2984"). InnerVolumeSpecName "kube-api-access-8hfrz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:49:25.135443 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.135423    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81d11de9-e12c-4ab5-87ab-5c257d7b2984-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "81d11de9-e12c-4ab5-87ab-5c257d7b2984" (UID: "81d11de9-e12c-4ab5-87ab-5c257d7b2984"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:49:25.138756 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.138734    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt"
Jul 28 01:49:25.233894 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.233851    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f2798e2-bc38-4b53-a44a-e628dc742376-plugins-config-volume\") pod \"2f2798e2-bc38-4b53-a44a-e628dc742376\" (UID: \"2f2798e2-bc38-4b53-a44a-e628dc742376\") "
Jul 28 01:49:25.234077 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.233922    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dv68q\" (UniqueName: \"kubernetes.io/projected/2f2798e2-bc38-4b53-a44a-e628dc742376-kube-api-access-dv68q\") pod \"2f2798e2-bc38-4b53-a44a-e628dc742376\" (UID: \"2f2798e2-bc38-4b53-a44a-e628dc742376\") "
Jul 28 01:49:25.234185 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.234174    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hfrz\" (UniqueName: \"kubernetes.io/projected/81d11de9-e12c-4ab5-87ab-5c257d7b2984-kube-api-access-8hfrz\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:49:25.234256 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.234188    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/266fa083-3719-4737-9452-f8063b161c51-plugins-config-volume\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:49:25.234256 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.234198    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81d11de9-e12c-4ab5-87ab-5c257d7b2984-openshift-service-ca\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:49:25.234256 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.234206    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdhvp\" (UniqueName: \"kubernetes.io/projected/266fa083-3719-4737-9452-f8063b161c51-kube-api-access-rdhvp\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:49:25.234256 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.234248    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81d11de9-e12c-4ab5-87ab-5c257d7b2984-maas-api-tls\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:49:25.234392 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.234296    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f2798e2-bc38-4b53-a44a-e628dc742376-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2f2798e2-bc38-4b53-a44a-e628dc742376" (UID: "2f2798e2-bc38-4b53-a44a-e628dc742376"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:49:25.236004 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.235983    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f2798e2-bc38-4b53-a44a-e628dc742376-kube-api-access-dv68q" (OuterVolumeSpecName: "kube-api-access-dv68q") pod "2f2798e2-bc38-4b53-a44a-e628dc742376" (UID: "2f2798e2-bc38-4b53-a44a-e628dc742376"). InnerVolumeSpecName "kube-api-access-dv68q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:49:25.335512 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.335415    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f2798e2-bc38-4b53-a44a-e628dc742376-plugins-config-volume\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:49:25.335512 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.335448    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dv68q\" (UniqueName: \"kubernetes.io/projected/2f2798e2-bc38-4b53-a44a-e628dc742376-kube-api-access-dv68q\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:49:25.550507 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.550472    2578 generic.go:356] "Generic (PLEG): container finished" podID="81d11de9-e12c-4ab5-87ab-5c257d7b2984" containerID="61003d429dad3e3a0a259e55fc9a8a0ca1d63942c97ea879b9c83f6af8e6d07c" exitCode=0
Jul 28 01:49:25.550669 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.550522    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j"
Jul 28 01:49:25.550669 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.550549    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j" event={"ID":"81d11de9-e12c-4ab5-87ab-5c257d7b2984","Type":"ContainerDied","Data":"61003d429dad3e3a0a259e55fc9a8a0ca1d63942c97ea879b9c83f6af8e6d07c"}
Jul 28 01:49:25.550669 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.550598    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j" event={"ID":"81d11de9-e12c-4ab5-87ab-5c257d7b2984","Type":"ContainerDied","Data":"b1a86148ed5e40941b97afd53a898348425bdfc52887f089e46675284019b890"}
Jul 28 01:49:25.550669 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.550630    2578 scope.go:117] "RemoveContainer" containerID="61003d429dad3e3a0a259e55fc9a8a0ca1d63942c97ea879b9c83f6af8e6d07c"
Jul 28 01:49:25.551924 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.551895    2578 generic.go:356] "Generic (PLEG): container finished" podID="2f2798e2-bc38-4b53-a44a-e628dc742376" containerID="a569ecace204f774156f6ac22e943553cf0022683320df2b0cc3a62a7ca6a7db" exitCode=0
Jul 28 01:49:25.552054 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.551962    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt"
Jul 28 01:49:25.552054 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.551985    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt" event={"ID":"2f2798e2-bc38-4b53-a44a-e628dc742376","Type":"ContainerDied","Data":"a569ecace204f774156f6ac22e943553cf0022683320df2b0cc3a62a7ca6a7db"}
Jul 28 01:49:25.552054 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.552025    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt" event={"ID":"2f2798e2-bc38-4b53-a44a-e628dc742376","Type":"ContainerDied","Data":"f043a8cbec340afa961a41581c42519179e12bc067e1927d2ef9b1646de184ec"}
Jul 28 01:49:25.553406 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.553383    2578 generic.go:356] "Generic (PLEG): container finished" podID="266fa083-3719-4737-9452-f8063b161c51" containerID="6c83188b898ee7021dad6c5e2813d9bd27a29cce0e0d1bb5dd19a2990e4f8fb2" exitCode=0
Jul 28 01:49:25.553507 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.553416    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-64a682-cf87788c8-flb78" event={"ID":"266fa083-3719-4737-9452-f8063b161c51","Type":"ContainerDied","Data":"6c83188b898ee7021dad6c5e2813d9bd27a29cce0e0d1bb5dd19a2990e4f8fb2"}
Jul 28 01:49:25.553507 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.553433    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-64a682-cf87788c8-flb78"
Jul 28 01:49:25.553507 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.553459    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-64a682-cf87788c8-flb78" event={"ID":"266fa083-3719-4737-9452-f8063b161c51","Type":"ContainerDied","Data":"709ce8d22e6c7a934d9e747d26160b14a6349fc31734563d89dc4c973fa1baac"}
Jul 28 01:49:25.561056 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.561026    2578 scope.go:117] "RemoveContainer" containerID="61003d429dad3e3a0a259e55fc9a8a0ca1d63942c97ea879b9c83f6af8e6d07c"
Jul 28 01:49:25.561385 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:49:25.561363    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61003d429dad3e3a0a259e55fc9a8a0ca1d63942c97ea879b9c83f6af8e6d07c\": container with ID starting with 61003d429dad3e3a0a259e55fc9a8a0ca1d63942c97ea879b9c83f6af8e6d07c not found: ID does not exist" containerID="61003d429dad3e3a0a259e55fc9a8a0ca1d63942c97ea879b9c83f6af8e6d07c"
Jul 28 01:49:25.561481 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.561399    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61003d429dad3e3a0a259e55fc9a8a0ca1d63942c97ea879b9c83f6af8e6d07c"} err="failed to get container status \"61003d429dad3e3a0a259e55fc9a8a0ca1d63942c97ea879b9c83f6af8e6d07c\": rpc error: code = NotFound desc = could not find container \"61003d429dad3e3a0a259e55fc9a8a0ca1d63942c97ea879b9c83f6af8e6d07c\": container with ID starting with 61003d429dad3e3a0a259e55fc9a8a0ca1d63942c97ea879b9c83f6af8e6d07c not found: ID does not exist"
Jul 28 01:49:25.561481 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.561425    2578 scope.go:117] "RemoveContainer" containerID="a569ecace204f774156f6ac22e943553cf0022683320df2b0cc3a62a7ca6a7db"
Jul 28 01:49:25.571902 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.571885    2578 scope.go:117] "RemoveContainer" containerID="a569ecace204f774156f6ac22e943553cf0022683320df2b0cc3a62a7ca6a7db"
Jul 28 01:49:25.572191 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:49:25.572171    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a569ecace204f774156f6ac22e943553cf0022683320df2b0cc3a62a7ca6a7db\": container with ID starting with a569ecace204f774156f6ac22e943553cf0022683320df2b0cc3a62a7ca6a7db not found: ID does not exist" containerID="a569ecace204f774156f6ac22e943553cf0022683320df2b0cc3a62a7ca6a7db"
Jul 28 01:49:25.572287 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.572199    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a569ecace204f774156f6ac22e943553cf0022683320df2b0cc3a62a7ca6a7db"} err="failed to get container status \"a569ecace204f774156f6ac22e943553cf0022683320df2b0cc3a62a7ca6a7db\": rpc error: code = NotFound desc = could not find container \"a569ecace204f774156f6ac22e943553cf0022683320df2b0cc3a62a7ca6a7db\": container with ID starting with a569ecace204f774156f6ac22e943553cf0022683320df2b0cc3a62a7ca6a7db not found: ID does not exist"
Jul 28 01:49:25.572287 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.572244    2578 scope.go:117] "RemoveContainer" containerID="6c83188b898ee7021dad6c5e2813d9bd27a29cce0e0d1bb5dd19a2990e4f8fb2"
Jul 28 01:49:25.580401 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.580370    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j"]
Jul 28 01:49:25.581827 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.581805    2578 scope.go:117] "RemoveContainer" containerID="6c83188b898ee7021dad6c5e2813d9bd27a29cce0e0d1bb5dd19a2990e4f8fb2"
Jul 28 01:49:25.582163 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:49:25.582143    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c83188b898ee7021dad6c5e2813d9bd27a29cce0e0d1bb5dd19a2990e4f8fb2\": container with ID starting with 6c83188b898ee7021dad6c5e2813d9bd27a29cce0e0d1bb5dd19a2990e4f8fb2 not found: ID does not exist" containerID="6c83188b898ee7021dad6c5e2813d9bd27a29cce0e0d1bb5dd19a2990e4f8fb2"
Jul 28 01:49:25.582260 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.582174    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c83188b898ee7021dad6c5e2813d9bd27a29cce0e0d1bb5dd19a2990e4f8fb2"} err="failed to get container status \"6c83188b898ee7021dad6c5e2813d9bd27a29cce0e0d1bb5dd19a2990e4f8fb2\": rpc error: code = NotFound desc = could not find container \"6c83188b898ee7021dad6c5e2813d9bd27a29cce0e0d1bb5dd19a2990e4f8fb2\": container with ID starting with 6c83188b898ee7021dad6c5e2813d9bd27a29cce0e0d1bb5dd19a2990e4f8fb2 not found: ID does not exist"
Jul 28 01:49:25.584058 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.584036    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-64a682-6755586778-ztq9j"]
Jul 28 01:49:25.595950 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.595874    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt"]
Jul 28 01:49:25.600007 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.599980    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-64a682-5fdb6db6d4-nxxnt"]
Jul 28 01:49:25.610623 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.610592    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-64a682-cf87788c8-flb78"]
Jul 28 01:49:25.614976 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:25.614949    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-64a682-cf87788c8-flb78"]
Jul 28 01:49:26.408758 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:26.404120    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="266fa083-3719-4737-9452-f8063b161c51" path="/var/lib/kubelet/pods/266fa083-3719-4737-9452-f8063b161c51/volumes"
Jul 28 01:49:26.408758 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:26.404747    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f2798e2-bc38-4b53-a44a-e628dc742376" path="/var/lib/kubelet/pods/2f2798e2-bc38-4b53-a44a-e628dc742376/volumes"
Jul 28 01:49:26.408758 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:26.405328    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81d11de9-e12c-4ab5-87ab-5c257d7b2984" path="/var/lib/kubelet/pods/81d11de9-e12c-4ab5-87ab-5c257d7b2984/volumes"
Jul 28 01:49:36.227888 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:36.227852    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3940b7-l749h"]
Jul 28 01:49:36.228324 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:36.228274    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="266fa083-3719-4737-9452-f8063b161c51" containerName="payload-processing"
Jul 28 01:49:36.228324 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:36.228287    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="266fa083-3719-4737-9452-f8063b161c51" containerName="payload-processing"
Jul 28 01:49:36.228324 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:36.228306    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81d11de9-e12c-4ab5-87ab-5c257d7b2984" containerName="maas-api"
Jul 28 01:49:36.228324 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:36.228311    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="81d11de9-e12c-4ab5-87ab-5c257d7b2984" containerName="maas-api"
Jul 28 01:49:36.228324 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:36.228320    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f2798e2-bc38-4b53-a44a-e628dc742376" containerName="payload-pre-processing"
Jul 28 01:49:36.228324 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:36.228326    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f2798e2-bc38-4b53-a44a-e628dc742376" containerName="payload-pre-processing"
Jul 28 01:49:36.228541 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:36.228389    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f2798e2-bc38-4b53-a44a-e628dc742376" containerName="payload-pre-processing"
Jul 28 01:49:36.228541 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:36.228399    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="81d11de9-e12c-4ab5-87ab-5c257d7b2984" containerName="maas-api"
Jul 28 01:49:36.228541 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:36.228408    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="266fa083-3719-4737-9452-f8063b161c51" containerName="payload-processing"
Jul 28 01:49:36.231353 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:36.231337    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3940b7-l749h"
Jul 28 01:49:36.233893 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:36.233872    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8ph8w\""
Jul 28 01:49:36.238620 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:36.238551    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3940b7-l749h"]
Jul 28 01:49:36.348378 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:36.348343    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70e99a03-9d0a-46a9-ac2a-cbce516fcf36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-3940b7-l749h\" (UID: \"70e99a03-9d0a-46a9-ac2a-cbce516fcf36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3940b7-l749h"
Jul 28 01:49:36.449014 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:36.448971    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70e99a03-9d0a-46a9-ac2a-cbce516fcf36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-3940b7-l749h\" (UID: \"70e99a03-9d0a-46a9-ac2a-cbce516fcf36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3940b7-l749h"
Jul 28 01:49:36.449641 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:36.449622    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70e99a03-9d0a-46a9-ac2a-cbce516fcf36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-3940b7-l749h\" (UID: \"70e99a03-9d0a-46a9-ac2a-cbce516fcf36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3940b7-l749h"
Jul 28 01:49:36.544850 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:36.544765    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3940b7-l749h"
Jul 28 01:49:36.688039 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:36.688011    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3940b7-l749h"]
Jul 28 01:49:36.690165 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:49:36.690135    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70e99a03_9d0a_46a9_ac2a_cbce516fcf36.slice/crio-f6e3e9ee81312dfd8e80f21fe9a931e4557a4a56a07ae3e2beb282d7a741bf09 WatchSource:0}: Error finding container f6e3e9ee81312dfd8e80f21fe9a931e4557a4a56a07ae3e2beb282d7a741bf09: Status 404 returned error can't find the container with id f6e3e9ee81312dfd8e80f21fe9a931e4557a4a56a07ae3e2beb282d7a741bf09
Jul 28 01:49:37.624234 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:37.624125    2578 generic.go:356] "Generic (PLEG): container finished" podID="70e99a03-9d0a-46a9-ac2a-cbce516fcf36" containerID="7cdfbb58e25aa402861b060900a8f12c429ea58b778a0d2ae75bad9b4bfe6c9e" exitCode=0
Jul 28 01:49:37.624234 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:37.624175    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3940b7-l749h" event={"ID":"70e99a03-9d0a-46a9-ac2a-cbce516fcf36","Type":"ContainerDied","Data":"7cdfbb58e25aa402861b060900a8f12c429ea58b778a0d2ae75bad9b4bfe6c9e"}
Jul 28 01:49:37.624623 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:37.624251    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3940b7-l749h" event={"ID":"70e99a03-9d0a-46a9-ac2a-cbce516fcf36","Type":"ContainerStarted","Data":"f6e3e9ee81312dfd8e80f21fe9a931e4557a4a56a07ae3e2beb282d7a741bf09"}
Jul 28 01:49:38.766431 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:38.766404    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3940b7-l749h"
Jul 28 01:49:38.872949 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:38.872909    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70e99a03-9d0a-46a9-ac2a-cbce516fcf36-maas-api-service-ca\") pod \"70e99a03-9d0a-46a9-ac2a-cbce516fcf36\" (UID: \"70e99a03-9d0a-46a9-ac2a-cbce516fcf36\") "
Jul 28 01:49:38.873361 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:38.873338    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70e99a03-9d0a-46a9-ac2a-cbce516fcf36-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "70e99a03-9d0a-46a9-ac2a-cbce516fcf36" (UID: "70e99a03-9d0a-46a9-ac2a-cbce516fcf36"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:49:38.974494 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:38.974396    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70e99a03-9d0a-46a9-ac2a-cbce516fcf36-maas-api-service-ca\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:49:39.636400 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:39.636365    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3940b7-l749h" event={"ID":"70e99a03-9d0a-46a9-ac2a-cbce516fcf36","Type":"ContainerDied","Data":"f6e3e9ee81312dfd8e80f21fe9a931e4557a4a56a07ae3e2beb282d7a741bf09"}
Jul 28 01:49:39.636582 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:39.636421    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f6e3e9ee81312dfd8e80f21fe9a931e4557a4a56a07ae3e2beb282d7a741bf09"
Jul 28 01:49:39.636582 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:39.636375    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3940b7-l749h"
Jul 28 01:49:51.698225 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:51.698175    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-559478b5-2f9jk"]
Jul 28 01:49:51.698880 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:51.698823    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-559478b5-2f9jk" podUID="762a4b61-0389-4156-8b55-8a72a44af553" containerName="maas-api" containerID="cri-o://23574eecbb9276addd6c3b679ab8b725e16e984d8ee105ef7f6e65377ac54785" gracePeriod=30
Jul 28 01:49:51.755329 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:51.755297    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-3940b7-758c87d88-bksxm"]
Jul 28 01:49:51.755582 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:51.755543    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-3940b7-758c87d88-bksxm" podUID="004d148c-9a7e-442e-a534-56f0b4b8c299" containerName="payload-processing" containerID="cri-o://8f4577a0be08b39c5d8a434fc6f73f24bbdbf06a3a3dc4c56d27fdcde12f6f47" gracePeriod=30
Jul 28 01:49:51.764943 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:51.764671    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv"]
Jul 28 01:49:51.765198 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:51.764957    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv" podUID="4c1eeb73-270e-4ba2-bce2-858ade657ee2" containerName="payload-pre-processing" containerID="cri-o://458d4462b861aaadc93e8ab514d975a663e48cf68f97008f743960a732312770" gracePeriod=30
Jul 28 01:49:52.042750 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.036240    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-559478b5-2f9jk"
Jul 28 01:49:52.062149 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.062113    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv"
Jul 28 01:49:52.069790 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.069760    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-3940b7-758c87d88-bksxm"
Jul 28 01:49:52.203749 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.203661    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/762a4b61-0389-4156-8b55-8a72a44af553-openshift-service-ca\") pod \"762a4b61-0389-4156-8b55-8a72a44af553\" (UID: \"762a4b61-0389-4156-8b55-8a72a44af553\") "
Jul 28 01:49:52.203749 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.203724    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fr2kg\" (UniqueName: \"kubernetes.io/projected/4c1eeb73-270e-4ba2-bce2-858ade657ee2-kube-api-access-fr2kg\") pod \"4c1eeb73-270e-4ba2-bce2-858ade657ee2\" (UID: \"4c1eeb73-270e-4ba2-bce2-858ade657ee2\") "
Jul 28 01:49:52.203749 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.203754    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/762a4b61-0389-4156-8b55-8a72a44af553-maas-api-tls\") pod \"762a4b61-0389-4156-8b55-8a72a44af553\" (UID: \"762a4b61-0389-4156-8b55-8a72a44af553\") "
Jul 28 01:49:52.204042 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.203778    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pk92q\" (UniqueName: \"kubernetes.io/projected/762a4b61-0389-4156-8b55-8a72a44af553-kube-api-access-pk92q\") pod \"762a4b61-0389-4156-8b55-8a72a44af553\" (UID: \"762a4b61-0389-4156-8b55-8a72a44af553\") "
Jul 28 01:49:52.204042 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.203800    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c1eeb73-270e-4ba2-bce2-858ade657ee2-plugins-config-volume\") pod \"4c1eeb73-270e-4ba2-bce2-858ade657ee2\" (UID: \"4c1eeb73-270e-4ba2-bce2-858ade657ee2\") "
Jul 28 01:49:52.204042 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.203821    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgpb5\" (UniqueName: \"kubernetes.io/projected/004d148c-9a7e-442e-a534-56f0b4b8c299-kube-api-access-jgpb5\") pod \"004d148c-9a7e-442e-a534-56f0b4b8c299\" (UID: \"004d148c-9a7e-442e-a534-56f0b4b8c299\") "
Jul 28 01:49:52.204042 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.203883    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/004d148c-9a7e-442e-a534-56f0b4b8c299-plugins-config-volume\") pod \"004d148c-9a7e-442e-a534-56f0b4b8c299\" (UID: \"004d148c-9a7e-442e-a534-56f0b4b8c299\") "
Jul 28 01:49:52.204307 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.204054    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/762a4b61-0389-4156-8b55-8a72a44af553-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "762a4b61-0389-4156-8b55-8a72a44af553" (UID: "762a4b61-0389-4156-8b55-8a72a44af553"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:49:52.204307 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.204281    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/762a4b61-0389-4156-8b55-8a72a44af553-openshift-service-ca\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:49:52.204414 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.204362    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c1eeb73-270e-4ba2-bce2-858ade657ee2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4c1eeb73-270e-4ba2-bce2-858ade657ee2" (UID: "4c1eeb73-270e-4ba2-bce2-858ade657ee2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:49:52.204464 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.204405    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/004d148c-9a7e-442e-a534-56f0b4b8c299-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "004d148c-9a7e-442e-a534-56f0b4b8c299" (UID: "004d148c-9a7e-442e-a534-56f0b4b8c299"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:49:52.205981 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.205956    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/762a4b61-0389-4156-8b55-8a72a44af553-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "762a4b61-0389-4156-8b55-8a72a44af553" (UID: "762a4b61-0389-4156-8b55-8a72a44af553"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:49:52.206097 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.205987    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c1eeb73-270e-4ba2-bce2-858ade657ee2-kube-api-access-fr2kg" (OuterVolumeSpecName: "kube-api-access-fr2kg") pod "4c1eeb73-270e-4ba2-bce2-858ade657ee2" (UID: "4c1eeb73-270e-4ba2-bce2-858ade657ee2"). InnerVolumeSpecName "kube-api-access-fr2kg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:49:52.206455 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.206433    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/004d148c-9a7e-442e-a534-56f0b4b8c299-kube-api-access-jgpb5" (OuterVolumeSpecName: "kube-api-access-jgpb5") pod "004d148c-9a7e-442e-a534-56f0b4b8c299" (UID: "004d148c-9a7e-442e-a534-56f0b4b8c299"). InnerVolumeSpecName "kube-api-access-jgpb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:49:52.206527 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.206434    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/762a4b61-0389-4156-8b55-8a72a44af553-kube-api-access-pk92q" (OuterVolumeSpecName: "kube-api-access-pk92q") pod "762a4b61-0389-4156-8b55-8a72a44af553" (UID: "762a4b61-0389-4156-8b55-8a72a44af553"). InnerVolumeSpecName "kube-api-access-pk92q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:49:52.305024 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.304983    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fr2kg\" (UniqueName: \"kubernetes.io/projected/4c1eeb73-270e-4ba2-bce2-858ade657ee2-kube-api-access-fr2kg\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:49:52.305024 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.305018    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/762a4b61-0389-4156-8b55-8a72a44af553-maas-api-tls\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:49:52.305024 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.305027    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pk92q\" (UniqueName: \"kubernetes.io/projected/762a4b61-0389-4156-8b55-8a72a44af553-kube-api-access-pk92q\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:49:52.305305 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.305036    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c1eeb73-270e-4ba2-bce2-858ade657ee2-plugins-config-volume\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:49:52.305305 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.305046    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgpb5\" (UniqueName: \"kubernetes.io/projected/004d148c-9a7e-442e-a534-56f0b4b8c299-kube-api-access-jgpb5\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:49:52.305305 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.305054    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/004d148c-9a7e-442e-a534-56f0b4b8c299-plugins-config-volume\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:49:52.723579 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.723541    2578 generic.go:356] "Generic (PLEG): container finished" podID="4c1eeb73-270e-4ba2-bce2-858ade657ee2" containerID="458d4462b861aaadc93e8ab514d975a663e48cf68f97008f743960a732312770" exitCode=0
Jul 28 01:49:52.723984 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.723594    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv"
Jul 28 01:49:52.723984 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.723629    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv" event={"ID":"4c1eeb73-270e-4ba2-bce2-858ade657ee2","Type":"ContainerDied","Data":"458d4462b861aaadc93e8ab514d975a663e48cf68f97008f743960a732312770"}
Jul 28 01:49:52.723984 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.723698    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv" event={"ID":"4c1eeb73-270e-4ba2-bce2-858ade657ee2","Type":"ContainerDied","Data":"32b3ac6e3118bca8684191d2dc6637b3b3c622918d776239494534fe4094e203"}
Jul 28 01:49:52.723984 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.723734    2578 scope.go:117] "RemoveContainer" containerID="458d4462b861aaadc93e8ab514d975a663e48cf68f97008f743960a732312770"
Jul 28 01:49:52.725276 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.725251    2578 generic.go:356] "Generic (PLEG): container finished" podID="762a4b61-0389-4156-8b55-8a72a44af553" containerID="23574eecbb9276addd6c3b679ab8b725e16e984d8ee105ef7f6e65377ac54785" exitCode=0
Jul 28 01:49:52.725416 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.725289    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-559478b5-2f9jk" event={"ID":"762a4b61-0389-4156-8b55-8a72a44af553","Type":"ContainerDied","Data":"23574eecbb9276addd6c3b679ab8b725e16e984d8ee105ef7f6e65377ac54785"}
Jul 28 01:49:52.725416 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.725324    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-559478b5-2f9jk"
Jul 28 01:49:52.725416 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.725350    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-559478b5-2f9jk" event={"ID":"762a4b61-0389-4156-8b55-8a72a44af553","Type":"ContainerDied","Data":"2c1e75da2a1403a9551521568af9a3569d9d06acefa4082f355ffb12565722de"}
Jul 28 01:49:52.727044 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.726897    2578 generic.go:356] "Generic (PLEG): container finished" podID="004d148c-9a7e-442e-a534-56f0b4b8c299" containerID="8f4577a0be08b39c5d8a434fc6f73f24bbdbf06a3a3dc4c56d27fdcde12f6f47" exitCode=0
Jul 28 01:49:52.727044 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.726920    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-3940b7-758c87d88-bksxm" event={"ID":"004d148c-9a7e-442e-a534-56f0b4b8c299","Type":"ContainerDied","Data":"8f4577a0be08b39c5d8a434fc6f73f24bbdbf06a3a3dc4c56d27fdcde12f6f47"}
Jul 28 01:49:52.727044 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.726962    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-3940b7-758c87d88-bksxm"
Jul 28 01:49:52.727044 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.726975    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-3940b7-758c87d88-bksxm" event={"ID":"004d148c-9a7e-442e-a534-56f0b4b8c299","Type":"ContainerDied","Data":"c3dde39c38a922f225621b58e0880753d4a6e598bdb2ae8b927351a993eec2fb"}
Jul 28 01:49:52.741819 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.741797    2578 scope.go:117] "RemoveContainer" containerID="458d4462b861aaadc93e8ab514d975a663e48cf68f97008f743960a732312770"
Jul 28 01:49:52.742152 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:49:52.742120    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"458d4462b861aaadc93e8ab514d975a663e48cf68f97008f743960a732312770\": container with ID starting with 458d4462b861aaadc93e8ab514d975a663e48cf68f97008f743960a732312770 not found: ID does not exist" containerID="458d4462b861aaadc93e8ab514d975a663e48cf68f97008f743960a732312770"
Jul 28 01:49:52.742200 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.742162    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"458d4462b861aaadc93e8ab514d975a663e48cf68f97008f743960a732312770"} err="failed to get container status \"458d4462b861aaadc93e8ab514d975a663e48cf68f97008f743960a732312770\": rpc error: code = NotFound desc = could not find container \"458d4462b861aaadc93e8ab514d975a663e48cf68f97008f743960a732312770\": container with ID starting with 458d4462b861aaadc93e8ab514d975a663e48cf68f97008f743960a732312770 not found: ID does not exist"
Jul 28 01:49:52.742200 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.742182    2578 scope.go:117] "RemoveContainer" containerID="23574eecbb9276addd6c3b679ab8b725e16e984d8ee105ef7f6e65377ac54785"
Jul 28 01:49:52.750870 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.750791    2578 scope.go:117] "RemoveContainer" containerID="23574eecbb9276addd6c3b679ab8b725e16e984d8ee105ef7f6e65377ac54785"
Jul 28 01:49:52.751489 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:49:52.751330    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23574eecbb9276addd6c3b679ab8b725e16e984d8ee105ef7f6e65377ac54785\": container with ID starting with 23574eecbb9276addd6c3b679ab8b725e16e984d8ee105ef7f6e65377ac54785 not found: ID does not exist" containerID="23574eecbb9276addd6c3b679ab8b725e16e984d8ee105ef7f6e65377ac54785"
Jul 28 01:49:52.751489 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.751366    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23574eecbb9276addd6c3b679ab8b725e16e984d8ee105ef7f6e65377ac54785"} err="failed to get container status \"23574eecbb9276addd6c3b679ab8b725e16e984d8ee105ef7f6e65377ac54785\": rpc error: code = NotFound desc = could not find container \"23574eecbb9276addd6c3b679ab8b725e16e984d8ee105ef7f6e65377ac54785\": container with ID starting with 23574eecbb9276addd6c3b679ab8b725e16e984d8ee105ef7f6e65377ac54785 not found: ID does not exist"
Jul 28 01:49:52.751489 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.751392    2578 scope.go:117] "RemoveContainer" containerID="8f4577a0be08b39c5d8a434fc6f73f24bbdbf06a3a3dc4c56d27fdcde12f6f47"
Jul 28 01:49:52.754751 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.754722    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-559478b5-2f9jk"]
Jul 28 01:49:52.757821 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.757797    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3940b7-559478b5-2f9jk"]
Jul 28 01:49:52.761498 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.761478    2578 scope.go:117] "RemoveContainer" containerID="8f4577a0be08b39c5d8a434fc6f73f24bbdbf06a3a3dc4c56d27fdcde12f6f47"
Jul 28 01:49:52.761873 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:49:52.761849    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f4577a0be08b39c5d8a434fc6f73f24bbdbf06a3a3dc4c56d27fdcde12f6f47\": container with ID starting with 8f4577a0be08b39c5d8a434fc6f73f24bbdbf06a3a3dc4c56d27fdcde12f6f47 not found: ID does not exist" containerID="8f4577a0be08b39c5d8a434fc6f73f24bbdbf06a3a3dc4c56d27fdcde12f6f47"
Jul 28 01:49:52.761941 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.761887    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f4577a0be08b39c5d8a434fc6f73f24bbdbf06a3a3dc4c56d27fdcde12f6f47"} err="failed to get container status \"8f4577a0be08b39c5d8a434fc6f73f24bbdbf06a3a3dc4c56d27fdcde12f6f47\": rpc error: code = NotFound desc = could not find container \"8f4577a0be08b39c5d8a434fc6f73f24bbdbf06a3a3dc4c56d27fdcde12f6f47\": container with ID starting with 8f4577a0be08b39c5d8a434fc6f73f24bbdbf06a3a3dc4c56d27fdcde12f6f47 not found: ID does not exist"
Jul 28 01:49:52.766711 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.766671    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv"]
Jul 28 01:49:52.771174 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.771151    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-3940b7-54c4b89c8d-gw8lv"]
Jul 28 01:49:52.780818 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.780789    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-3940b7-758c87d88-bksxm"]
Jul 28 01:49:52.783277 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:52.783256    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-3940b7-758c87d88-bksxm"]
Jul 28 01:49:54.402730 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:54.402687    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="004d148c-9a7e-442e-a534-56f0b4b8c299" path="/var/lib/kubelet/pods/004d148c-9a7e-442e-a534-56f0b4b8c299/volumes"
Jul 28 01:49:54.403294 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:54.403273    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c1eeb73-270e-4ba2-bce2-858ade657ee2" path="/var/lib/kubelet/pods/4c1eeb73-270e-4ba2-bce2-858ade657ee2/volumes"
Jul 28 01:49:54.403716 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:54.403697    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="762a4b61-0389-4156-8b55-8a72a44af553" path="/var/lib/kubelet/pods/762a4b61-0389-4156-8b55-8a72a44af553/volumes"
Jul 28 01:49:56.302354 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:56.302320    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3940b7-l749h"]
Jul 28 01:49:56.304845 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:56.304821    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3940b7-l749h"]
Jul 28 01:49:56.402324 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:49:56.402295    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70e99a03-9d0a-46a9-ac2a-cbce516fcf36" path="/var/lib/kubelet/pods/70e99a03-9d0a-46a9-ac2a-cbce516fcf36/volumes"
Jul 28 01:50:06.540076 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:06.540039    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf"]
Jul 28 01:50:06.540652 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:06.540632    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="004d148c-9a7e-442e-a534-56f0b4b8c299" containerName="payload-processing"
Jul 28 01:50:06.540736 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:06.540656    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="004d148c-9a7e-442e-a534-56f0b4b8c299" containerName="payload-processing"
Jul 28 01:50:06.540736 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:06.540668    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c1eeb73-270e-4ba2-bce2-858ade657ee2" containerName="payload-pre-processing"
Jul 28 01:50:06.540736 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:06.540677    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c1eeb73-270e-4ba2-bce2-858ade657ee2" containerName="payload-pre-processing"
Jul 28 01:50:06.540736 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:06.540691    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70e99a03-9d0a-46a9-ac2a-cbce516fcf36" containerName="revoke-keys"
Jul 28 01:50:06.540736 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:06.540700    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="70e99a03-9d0a-46a9-ac2a-cbce516fcf36" containerName="revoke-keys"
Jul 28 01:50:06.540736 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:06.540727    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="762a4b61-0389-4156-8b55-8a72a44af553" containerName="maas-api"
Jul 28 01:50:06.540736 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:06.540735    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="762a4b61-0389-4156-8b55-8a72a44af553" containerName="maas-api"
Jul 28 01:50:06.541067 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:06.540836    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c1eeb73-270e-4ba2-bce2-858ade657ee2" containerName="payload-pre-processing"
Jul 28 01:50:06.541067 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:06.540851    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="762a4b61-0389-4156-8b55-8a72a44af553" containerName="maas-api"
Jul 28 01:50:06.541067 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:06.540862    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="70e99a03-9d0a-46a9-ac2a-cbce516fcf36" containerName="revoke-keys"
Jul 28 01:50:06.541067 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:06.540875    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="004d148c-9a7e-442e-a534-56f0b4b8c299" containerName="payload-processing"
Jul 28 01:50:06.544236 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:06.544199    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf"
Jul 28 01:50:06.546948 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:06.546924    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lrljm\""
Jul 28 01:50:06.547344 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:06.547321    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 01:50:06.548168 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:06.548149    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-73e2fd-serving-cert\""
Jul 28 01:50:06.548279 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:06.548171    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 01:50:06.558558 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:06.558459    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf"]
Jul 28 01:50:06.633431 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:06.633394    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r2wf\" (UniqueName: \"kubernetes.io/projected/0e17f9db-5dc6-4321-b9aa-7f1e7401f155-kube-api-access-4r2wf\") pod \"maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf\" (UID: \"0e17f9db-5dc6-4321-b9aa-7f1e7401f155\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf"
Jul 28 01:50:06.633626 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:06.633514    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e17f9db-5dc6-4321-b9aa-7f1e7401f155-maas-api-tls\") pod \"maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf\" (UID: \"0e17f9db-5dc6-4321-b9aa-7f1e7401f155\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf"
Jul 28 01:50:06.633626 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:06.633544    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e17f9db-5dc6-4321-b9aa-7f1e7401f155-openshift-service-ca\") pod \"maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf\" (UID: \"0e17f9db-5dc6-4321-b9aa-7f1e7401f155\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf"
Jul 28 01:50:06.734839 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:06.734801    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r2wf\" (UniqueName: \"kubernetes.io/projected/0e17f9db-5dc6-4321-b9aa-7f1e7401f155-kube-api-access-4r2wf\") pod \"maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf\" (UID: \"0e17f9db-5dc6-4321-b9aa-7f1e7401f155\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf"
Jul 28 01:50:06.735021 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:06.734916    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e17f9db-5dc6-4321-b9aa-7f1e7401f155-maas-api-tls\") pod \"maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf\" (UID: \"0e17f9db-5dc6-4321-b9aa-7f1e7401f155\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf"
Jul 28 01:50:06.735021 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:06.734947    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e17f9db-5dc6-4321-b9aa-7f1e7401f155-openshift-service-ca\") pod \"maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf\" (UID: \"0e17f9db-5dc6-4321-b9aa-7f1e7401f155\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf"
Jul 28 01:50:06.735117 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:50:06.735063    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-serving-cert: secret "maas-api-e2e-shared-a-73e2fd-serving-cert" not found
Jul 28 01:50:06.735156 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:50:06.735140    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e17f9db-5dc6-4321-b9aa-7f1e7401f155-maas-api-tls podName:0e17f9db-5dc6-4321-b9aa-7f1e7401f155 nodeName:}" failed. No retries permitted until 2026-07-28 01:50:07.235116077 +0000 UTC m=+3377.399924165 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0e17f9db-5dc6-4321-b9aa-7f1e7401f155-maas-api-tls") pod "maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf" (UID: "0e17f9db-5dc6-4321-b9aa-7f1e7401f155") : secret "maas-api-e2e-shared-a-73e2fd-serving-cert" not found
Jul 28 01:50:06.735656 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:06.735639    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e17f9db-5dc6-4321-b9aa-7f1e7401f155-openshift-service-ca\") pod \"maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf\" (UID: \"0e17f9db-5dc6-4321-b9aa-7f1e7401f155\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf"
Jul 28 01:50:06.747727 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:06.747690    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r2wf\" (UniqueName: \"kubernetes.io/projected/0e17f9db-5dc6-4321-b9aa-7f1e7401f155-kube-api-access-4r2wf\") pod \"maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf\" (UID: \"0e17f9db-5dc6-4321-b9aa-7f1e7401f155\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf"
Jul 28 01:50:07.240441 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.240396    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e17f9db-5dc6-4321-b9aa-7f1e7401f155-maas-api-tls\") pod \"maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf\" (UID: \"0e17f9db-5dc6-4321-b9aa-7f1e7401f155\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf"
Jul 28 01:50:07.242885 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.242859    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e17f9db-5dc6-4321-b9aa-7f1e7401f155-maas-api-tls\") pod \"maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf\" (UID: \"0e17f9db-5dc6-4321-b9aa-7f1e7401f155\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf"
Jul 28 01:50:07.318559 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.318497    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf"]
Jul 28 01:50:07.322058 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.322040    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf"
Jul 28 01:50:07.324821 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.324795    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-73e2fd\""
Jul 28 01:50:07.324821 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.324820    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-73e2fd-dockercfg-pmnl6\""
Jul 28 01:50:07.335861 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.335832    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf"]
Jul 28 01:50:07.341340 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.341307    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7dfe7cc1-54cf-4ce8-8314-438e2ec8decd-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf\" (UID: \"7dfe7cc1-54cf-4ce8-8314-438e2ec8decd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf"
Jul 28 01:50:07.341470 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.341350    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cvfr\" (UniqueName: \"kubernetes.io/projected/7dfe7cc1-54cf-4ce8-8314-438e2ec8decd-kube-api-access-7cvfr\") pod \"payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf\" (UID: \"7dfe7cc1-54cf-4ce8-8314-438e2ec8decd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf"
Jul 28 01:50:07.360026 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.359992    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4"]
Jul 28 01:50:07.363676 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.363658    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4"
Jul 28 01:50:07.376124 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.376097    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4"]
Jul 28 01:50:07.442031 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.441992    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlb6p\" (UniqueName: \"kubernetes.io/projected/20ec5aed-4c5c-42bd-b44b-6f1db3023d33-kube-api-access-rlb6p\") pod \"payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4\" (UID: \"20ec5aed-4c5c-42bd-b44b-6f1db3023d33\") " pod="openshift-ingress/payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4"
Jul 28 01:50:07.442257 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.442053    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7dfe7cc1-54cf-4ce8-8314-438e2ec8decd-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf\" (UID: \"7dfe7cc1-54cf-4ce8-8314-438e2ec8decd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf"
Jul 28 01:50:07.442257 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.442103    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20ec5aed-4c5c-42bd-b44b-6f1db3023d33-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4\" (UID: \"20ec5aed-4c5c-42bd-b44b-6f1db3023d33\") " pod="openshift-ingress/payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4"
Jul 28 01:50:07.442257 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.442164    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cvfr\" (UniqueName: \"kubernetes.io/projected/7dfe7cc1-54cf-4ce8-8314-438e2ec8decd-kube-api-access-7cvfr\") pod \"payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf\" (UID: \"7dfe7cc1-54cf-4ce8-8314-438e2ec8decd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf"
Jul 28 01:50:07.442753 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.442734    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7dfe7cc1-54cf-4ce8-8314-438e2ec8decd-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf\" (UID: \"7dfe7cc1-54cf-4ce8-8314-438e2ec8decd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf"
Jul 28 01:50:07.450370 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.450341    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cvfr\" (UniqueName: \"kubernetes.io/projected/7dfe7cc1-54cf-4ce8-8314-438e2ec8decd-kube-api-access-7cvfr\") pod \"payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf\" (UID: \"7dfe7cc1-54cf-4ce8-8314-438e2ec8decd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf"
Jul 28 01:50:07.460193 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.460173    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf"
Jul 28 01:50:07.543231 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.543176    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlb6p\" (UniqueName: \"kubernetes.io/projected/20ec5aed-4c5c-42bd-b44b-6f1db3023d33-kube-api-access-rlb6p\") pod \"payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4\" (UID: \"20ec5aed-4c5c-42bd-b44b-6f1db3023d33\") " pod="openshift-ingress/payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4"
Jul 28 01:50:07.543656 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.543291    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20ec5aed-4c5c-42bd-b44b-6f1db3023d33-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4\" (UID: \"20ec5aed-4c5c-42bd-b44b-6f1db3023d33\") " pod="openshift-ingress/payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4"
Jul 28 01:50:07.544248 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.544200    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20ec5aed-4c5c-42bd-b44b-6f1db3023d33-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4\" (UID: \"20ec5aed-4c5c-42bd-b44b-6f1db3023d33\") " pod="openshift-ingress/payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4"
Jul 28 01:50:07.551198 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.551170    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlb6p\" (UniqueName: \"kubernetes.io/projected/20ec5aed-4c5c-42bd-b44b-6f1db3023d33-kube-api-access-rlb6p\") pod \"payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4\" (UID: \"20ec5aed-4c5c-42bd-b44b-6f1db3023d33\") " pod="openshift-ingress/payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4"
Jul 28 01:50:07.606484 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.606455    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf"]
Jul 28 01:50:07.607489 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:50:07.607467    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e17f9db_5dc6_4321_b9aa_7f1e7401f155.slice/crio-6250fb63284585a6b3b9da76eb6566f4fbb1a997c6784b679ad16ae56ef07f18 WatchSource:0}: Error finding container 6250fb63284585a6b3b9da76eb6566f4fbb1a997c6784b679ad16ae56ef07f18: Status 404 returned error can't find the container with id 6250fb63284585a6b3b9da76eb6566f4fbb1a997c6784b679ad16ae56ef07f18
Jul 28 01:50:07.634181 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.634148    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf"
Jul 28 01:50:07.675069 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.675032    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4"
Jul 28 01:50:07.776408 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.776381    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf"]
Jul 28 01:50:07.777737 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:50:07.777705    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dfe7cc1_54cf_4ce8_8314_438e2ec8decd.slice/crio-c44d4ae15ce3e9a3bbbd568a9ddb969ab121aec4ee526b3b26f699c00e14a648 WatchSource:0}: Error finding container c44d4ae15ce3e9a3bbbd568a9ddb969ab121aec4ee526b3b26f699c00e14a648: Status 404 returned error can't find the container with id c44d4ae15ce3e9a3bbbd568a9ddb969ab121aec4ee526b3b26f699c00e14a648
Jul 28 01:50:07.820755 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.820720    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf" event={"ID":"7dfe7cc1-54cf-4ce8-8314-438e2ec8decd","Type":"ContainerStarted","Data":"c44d4ae15ce3e9a3bbbd568a9ddb969ab121aec4ee526b3b26f699c00e14a648"}
Jul 28 01:50:07.821921 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.821892    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf" event={"ID":"0e17f9db-5dc6-4321-b9aa-7f1e7401f155","Type":"ContainerStarted","Data":"6250fb63284585a6b3b9da76eb6566f4fbb1a997c6784b679ad16ae56ef07f18"}
Jul 28 01:50:07.841071 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:07.841043    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4"]
Jul 28 01:50:07.842084 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:50:07.842056    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20ec5aed_4c5c_42bd_b44b_6f1db3023d33.slice/crio-38d00fbf77ec9a62418ee94db03c0b3a0909b17ae500e2ebea8f4ba92cb64b7d WatchSource:0}: Error finding container 38d00fbf77ec9a62418ee94db03c0b3a0909b17ae500e2ebea8f4ba92cb64b7d: Status 404 returned error can't find the container with id 38d00fbf77ec9a62418ee94db03c0b3a0909b17ae500e2ebea8f4ba92cb64b7d
Jul 28 01:50:08.828995 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:08.828953    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf" event={"ID":"0e17f9db-5dc6-4321-b9aa-7f1e7401f155","Type":"ContainerStarted","Data":"12aab52760c14d86420c252a67f6863a3159d9b3df60681e2dee3082050af295"}
Jul 28 01:50:08.829435 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:08.829022    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf"
Jul 28 01:50:08.830535 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:08.830512    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4" event={"ID":"20ec5aed-4c5c-42bd-b44b-6f1db3023d33","Type":"ContainerStarted","Data":"d927136143c21c916e703890f268fdb0a1b0301cb33ebd0df6795f055db20fa6"}
Jul 28 01:50:08.830669 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:08.830550    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4" event={"ID":"20ec5aed-4c5c-42bd-b44b-6f1db3023d33","Type":"ContainerStarted","Data":"38d00fbf77ec9a62418ee94db03c0b3a0909b17ae500e2ebea8f4ba92cb64b7d"}
Jul 28 01:50:08.830669 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:08.830626    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4"
Jul 28 01:50:08.832027 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:08.832005    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf" event={"ID":"7dfe7cc1-54cf-4ce8-8314-438e2ec8decd","Type":"ContainerStarted","Data":"2f6a27867ebad0974e248ec21b1cbd19f10fc81ae80bdb704e7b120fae87fb9d"}
Jul 28 01:50:08.832149 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:08.832138    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf"
Jul 28 01:50:08.847831 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:08.847780    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf" podStartSLOduration=2.439229271 podStartE2EDuration="2.847765555s" totalImagesPullingTime="408.536284ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:50:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:50:08.846527854 +0000 UTC m=+3379.011335962" watchObservedRunningTime="2026-07-28 01:50:08.847765555 +0000 UTC m=+3379.012573661"
Jul 28 01:50:08.864296 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:08.864230    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf" podStartSLOduration=1.8641966330000002 podStartE2EDuration="1.864196633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:50:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:50:08.861192031 +0000 UTC m=+3379.026000139" watchObservedRunningTime="2026-07-28 01:50:08.864196633 +0000 UTC m=+3379.029004742"
Jul 28 01:50:08.877914 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:08.877861    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4" podStartSLOduration=1.877842386 podStartE2EDuration="1.877842386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:50:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:50:08.876612024 +0000 UTC m=+3379.041420133" watchObservedRunningTime="2026-07-28 01:50:08.877842386 +0000 UTC m=+3379.042650492"
Jul 28 01:50:14.559670 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:14.559631    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x"]
Jul 28 01:50:14.564564 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:14.564545    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x"
Jul 28 01:50:14.569158 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:14.569123    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-7d2c59-serving-cert\""
Jul 28 01:50:14.576725 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:14.576700    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x"]
Jul 28 01:50:14.613073 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:14.613037    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cbcb559-bb6f-4248-998a-47e6febbeb12-openshift-service-ca\") pod \"maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x\" (UID: \"4cbcb559-bb6f-4248-998a-47e6febbeb12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x"
Jul 28 01:50:14.613248 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:14.613095    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cbcb559-bb6f-4248-998a-47e6febbeb12-maas-api-tls\") pod \"maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x\" (UID: \"4cbcb559-bb6f-4248-998a-47e6febbeb12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x"
Jul 28 01:50:14.613248 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:14.613197    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql545\" (UniqueName: \"kubernetes.io/projected/4cbcb559-bb6f-4248-998a-47e6febbeb12-kube-api-access-ql545\") pod \"maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x\" (UID: \"4cbcb559-bb6f-4248-998a-47e6febbeb12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x"
Jul 28 01:50:14.714137 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:14.714084    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql545\" (UniqueName: \"kubernetes.io/projected/4cbcb559-bb6f-4248-998a-47e6febbeb12-kube-api-access-ql545\") pod \"maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x\" (UID: \"4cbcb559-bb6f-4248-998a-47e6febbeb12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x"
Jul 28 01:50:14.714381 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:14.714208    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cbcb559-bb6f-4248-998a-47e6febbeb12-openshift-service-ca\") pod \"maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x\" (UID: \"4cbcb559-bb6f-4248-998a-47e6febbeb12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x"
Jul 28 01:50:14.714381 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:14.714263    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cbcb559-bb6f-4248-998a-47e6febbeb12-maas-api-tls\") pod \"maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x\" (UID: \"4cbcb559-bb6f-4248-998a-47e6febbeb12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x"
Jul 28 01:50:14.714381 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:50:14.714364    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-serving-cert: secret "maas-api-e2e-shared-b-7d2c59-serving-cert" not found
Jul 28 01:50:14.714546 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:50:14.714411    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cbcb559-bb6f-4248-998a-47e6febbeb12-maas-api-tls podName:4cbcb559-bb6f-4248-998a-47e6febbeb12 nodeName:}" failed. No retries permitted until 2026-07-28 01:50:15.214396341 +0000 UTC m=+3385.379204427 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4cbcb559-bb6f-4248-998a-47e6febbeb12-maas-api-tls") pod "maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x" (UID: "4cbcb559-bb6f-4248-998a-47e6febbeb12") : secret "maas-api-e2e-shared-b-7d2c59-serving-cert" not found
Jul 28 01:50:14.714858 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:14.714838    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cbcb559-bb6f-4248-998a-47e6febbeb12-openshift-service-ca\") pod \"maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x\" (UID: \"4cbcb559-bb6f-4248-998a-47e6febbeb12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x"
Jul 28 01:50:14.726034 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:14.726007    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql545\" (UniqueName: \"kubernetes.io/projected/4cbcb559-bb6f-4248-998a-47e6febbeb12-kube-api-access-ql545\") pod \"maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x\" (UID: \"4cbcb559-bb6f-4248-998a-47e6febbeb12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x"
Jul 28 01:50:14.843367 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:14.843286    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf"
Jul 28 01:50:15.219134 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:15.219094    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cbcb559-bb6f-4248-998a-47e6febbeb12-maas-api-tls\") pod \"maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x\" (UID: \"4cbcb559-bb6f-4248-998a-47e6febbeb12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x"
Jul 28 01:50:15.222037 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:15.222006    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cbcb559-bb6f-4248-998a-47e6febbeb12-maas-api-tls\") pod \"maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x\" (UID: \"4cbcb559-bb6f-4248-998a-47e6febbeb12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x"
Jul 28 01:50:15.478926 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:15.478824    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x"
Jul 28 01:50:15.490343 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:15.490311    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f"]
Jul 28 01:50:15.496802 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:15.496776    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f"
Jul 28 01:50:15.499393 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:15.499367    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-7d2c59-dockercfg-9z52d\""
Jul 28 01:50:15.499522 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:15.499480    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-7d2c59\""
Jul 28 01:50:15.505661 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:15.505607    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f"]
Jul 28 01:50:15.575664 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:15.575074    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx"]
Jul 28 01:50:15.579467 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:15.579438    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx"
Jul 28 01:50:15.588490 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:15.588412    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx"]
Jul 28 01:50:15.623516 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:15.623475    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8n9t\" (UniqueName: \"kubernetes.io/projected/add66961-dca0-4db6-b815-18047d840472-kube-api-access-r8n9t\") pod \"payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx\" (UID: \"add66961-dca0-4db6-b815-18047d840472\") " pod="openshift-ingress/payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx"
Jul 28 01:50:15.623712 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:15.623527    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44c7d\" (UniqueName: \"kubernetes.io/projected/174fb214-b943-448c-bfe7-7317f889a50f-kube-api-access-44c7d\") pod \"payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f\" (UID: \"174fb214-b943-448c-bfe7-7317f889a50f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f"
Jul 28 01:50:15.623712 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:15.623605    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/add66961-dca0-4db6-b815-18047d840472-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx\" (UID: \"add66961-dca0-4db6-b815-18047d840472\") " pod="openshift-ingress/payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx"
Jul 28 01:50:15.623849 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:15.623722    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/174fb214-b943-448c-bfe7-7317f889a50f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f\" (UID: \"174fb214-b943-448c-bfe7-7317f889a50f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f"
Jul 28 01:50:15.637992 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:15.637962    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x"]
Jul 28 01:50:15.640839 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:50:15.640801    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cbcb559_bb6f_4248_998a_47e6febbeb12.slice/crio-6e2ba4ca3f2402c29a9c42017e4602d47ac8fed9e50cee48b3333be63dc8aecb WatchSource:0}: Error finding container 6e2ba4ca3f2402c29a9c42017e4602d47ac8fed9e50cee48b3333be63dc8aecb: Status 404 returned error can't find the container with id 6e2ba4ca3f2402c29a9c42017e4602d47ac8fed9e50cee48b3333be63dc8aecb
Jul 28 01:50:15.724678 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:15.724636    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/add66961-dca0-4db6-b815-18047d840472-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx\" (UID: \"add66961-dca0-4db6-b815-18047d840472\") " pod="openshift-ingress/payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx"
Jul 28 01:50:15.724832 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:15.724735    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/174fb214-b943-448c-bfe7-7317f889a50f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f\" (UID: \"174fb214-b943-448c-bfe7-7317f889a50f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f"
Jul 28 01:50:15.724832 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:15.724796    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8n9t\" (UniqueName: \"kubernetes.io/projected/add66961-dca0-4db6-b815-18047d840472-kube-api-access-r8n9t\") pod \"payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx\" (UID: \"add66961-dca0-4db6-b815-18047d840472\") " pod="openshift-ingress/payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx"
Jul 28 01:50:15.724832 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:15.724825    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44c7d\" (UniqueName: \"kubernetes.io/projected/174fb214-b943-448c-bfe7-7317f889a50f-kube-api-access-44c7d\") pod \"payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f\" (UID: \"174fb214-b943-448c-bfe7-7317f889a50f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f"
Jul 28 01:50:15.725475 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:15.725450    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/add66961-dca0-4db6-b815-18047d840472-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx\" (UID: \"add66961-dca0-4db6-b815-18047d840472\") " pod="openshift-ingress/payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx"
Jul 28 01:50:15.725529 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:15.725513    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/174fb214-b943-448c-bfe7-7317f889a50f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f\" (UID: \"174fb214-b943-448c-bfe7-7317f889a50f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f"
Jul 28 01:50:15.732973 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:15.732910    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44c7d\" (UniqueName: \"kubernetes.io/projected/174fb214-b943-448c-bfe7-7317f889a50f-kube-api-access-44c7d\") pod \"payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f\" (UID: \"174fb214-b943-448c-bfe7-7317f889a50f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f"
Jul 28 01:50:15.733316 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:15.733296    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8n9t\" (UniqueName: \"kubernetes.io/projected/add66961-dca0-4db6-b815-18047d840472-kube-api-access-r8n9t\") pod \"payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx\" (UID: \"add66961-dca0-4db6-b815-18047d840472\") " pod="openshift-ingress/payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx"
Jul 28 01:50:15.821968 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:15.821932    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f"
Jul 28 01:50:15.871808 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:15.871765    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x" event={"ID":"4cbcb559-bb6f-4248-998a-47e6febbeb12","Type":"ContainerStarted","Data":"6e2ba4ca3f2402c29a9c42017e4602d47ac8fed9e50cee48b3333be63dc8aecb"}
Jul 28 01:50:15.893992 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:15.893959    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx"
Jul 28 01:50:15.974223 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:50:15.974181    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod174fb214_b943_448c_bfe7_7317f889a50f.slice/crio-89db70875584c0e16c0be7c15204c290c1a009c65b1125f3b6d398e120e28961 WatchSource:0}: Error finding container 89db70875584c0e16c0be7c15204c290c1a009c65b1125f3b6d398e120e28961: Status 404 returned error can't find the container with id 89db70875584c0e16c0be7c15204c290c1a009c65b1125f3b6d398e120e28961
Jul 28 01:50:15.974631 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:15.974605    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f"]
Jul 28 01:50:16.048655 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:16.048629    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx"]
Jul 28 01:50:16.049423 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:50:16.049383    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadd66961_dca0_4db6_b815_18047d840472.slice/crio-1b883323a0f50b07c5ca6d5c89e5145e99eca5ab2f1bb70098c64cbe7ea322bc WatchSource:0}: Error finding container 1b883323a0f50b07c5ca6d5c89e5145e99eca5ab2f1bb70098c64cbe7ea322bc: Status 404 returned error can't find the container with id 1b883323a0f50b07c5ca6d5c89e5145e99eca5ab2f1bb70098c64cbe7ea322bc
Jul 28 01:50:16.881395 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:16.881346    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x" event={"ID":"4cbcb559-bb6f-4248-998a-47e6febbeb12","Type":"ContainerStarted","Data":"eef9513dd9db29978b403aef45146a03fc5d3206378d0e2d322921919abce64f"}
Jul 28 01:50:16.881911 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:16.881531    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x"
Jul 28 01:50:16.883070 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:16.883032    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx" event={"ID":"add66961-dca0-4db6-b815-18047d840472","Type":"ContainerStarted","Data":"7e9b7510ed0acda3fa5da63aa46996b5b09ab2c5cd0b804d93f3251d7f8e4951"}
Jul 28 01:50:16.883223 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:16.883093    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx" event={"ID":"add66961-dca0-4db6-b815-18047d840472","Type":"ContainerStarted","Data":"1b883323a0f50b07c5ca6d5c89e5145e99eca5ab2f1bb70098c64cbe7ea322bc"}
Jul 28 01:50:16.883223 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:16.883123    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx"
Jul 28 01:50:16.884559 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:16.884538    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f" event={"ID":"174fb214-b943-448c-bfe7-7317f889a50f","Type":"ContainerStarted","Data":"8d0a409aa0a8684670dd20f561a08f34717fa8cd0e746925c43f22acee451968"}
Jul 28 01:50:16.884638 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:16.884575    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f" event={"ID":"174fb214-b943-448c-bfe7-7317f889a50f","Type":"ContainerStarted","Data":"89db70875584c0e16c0be7c15204c290c1a009c65b1125f3b6d398e120e28961"}
Jul 28 01:50:16.884638 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:16.884615    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f"
Jul 28 01:50:16.901772 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:16.901717    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x" podStartSLOduration=2.452048655 podStartE2EDuration="2.901700963s" totalImagesPullingTime="449.652308ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:50:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:50:16.899057044 +0000 UTC m=+3387.063865190" watchObservedRunningTime="2026-07-28 01:50:16.901700963 +0000 UTC m=+3387.066509069"
Jul 28 01:50:16.921245 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:16.921163    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f" podStartSLOduration=1.9211443830000001 podStartE2EDuration="1.921144383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:50:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:50:16.918184489 +0000 UTC m=+3387.082992597" watchObservedRunningTime="2026-07-28 01:50:16.921144383 +0000 UTC m=+3387.085952468"
Jul 28 01:50:16.935365 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:16.935310    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx" podStartSLOduration=1.935294364 podStartE2EDuration="1.935294364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:50:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:50:16.93347729 +0000 UTC m=+3387.098285400" watchObservedRunningTime="2026-07-28 01:50:16.935294364 +0000 UTC m=+3387.100102471"
Jul 28 01:50:19.839684 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:19.839654    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4"
Jul 28 01:50:19.840089 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:19.839739    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf"
Jul 28 01:50:22.895509 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:22.895477    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x"
Jul 28 01:50:27.892342 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:27.892316    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx"
Jul 28 01:50:27.892849 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:50:27.892386    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f"
Jul 28 01:55:37.047678 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.047609    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"]
Jul 28 01:55:37.051362 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.051340    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"
Jul 28 01:55:37.053948 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.053927    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-73e2fd\"/\"openshift-service-ca.crt\""
Jul 28 01:55:37.055188 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.055152    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-73e2fd\"/\"kube-root-ca.crt\""
Jul 28 01:55:37.055335 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.055185    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-73e2fd\"/\"default-dockercfg-7pqzg\""
Jul 28 01:55:37.055335 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.055296    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-73e2fd\"/\"sub-test-model-73e2fd-kserve-self-signed-certs\""
Jul 28 01:55:37.060929 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.060344    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"]
Jul 28 01:55:37.122727 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.122674    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64172415-a87a-4b1b-b047-a42026c28f8e-tmp-dir\") pod \"sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q\" (UID: \"64172415-a87a-4b1b-b047-a42026c28f8e\") " pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"
Jul 28 01:55:37.122918 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.122753    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64172415-a87a-4b1b-b047-a42026c28f8e-tls-certs\") pod \"sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q\" (UID: \"64172415-a87a-4b1b-b047-a42026c28f8e\") " pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"
Jul 28 01:55:37.122918 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.122776    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64172415-a87a-4b1b-b047-a42026c28f8e-model-cache\") pod \"sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q\" (UID: \"64172415-a87a-4b1b-b047-a42026c28f8e\") " pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"
Jul 28 01:55:37.122918 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.122808    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64172415-a87a-4b1b-b047-a42026c28f8e-dshm\") pod \"sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q\" (UID: \"64172415-a87a-4b1b-b047-a42026c28f8e\") " pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"
Jul 28 01:55:37.122918 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.122873    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64172415-a87a-4b1b-b047-a42026c28f8e-home\") pod \"sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q\" (UID: \"64172415-a87a-4b1b-b047-a42026c28f8e\") " pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"
Jul 28 01:55:37.122918 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.122915    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8h9lx\" (UniqueName: \"kubernetes.io/projected/64172415-a87a-4b1b-b047-a42026c28f8e-kube-api-access-8h9lx\") pod \"sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q\" (UID: \"64172415-a87a-4b1b-b047-a42026c28f8e\") " pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"
Jul 28 01:55:37.224134 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.224098    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64172415-a87a-4b1b-b047-a42026c28f8e-home\") pod \"sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q\" (UID: \"64172415-a87a-4b1b-b047-a42026c28f8e\") " pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"
Jul 28 01:55:37.224336 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.224174    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h9lx\" (UniqueName: \"kubernetes.io/projected/64172415-a87a-4b1b-b047-a42026c28f8e-kube-api-access-8h9lx\") pod \"sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q\" (UID: \"64172415-a87a-4b1b-b047-a42026c28f8e\") " pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"
Jul 28 01:55:37.224336 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.224242    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64172415-a87a-4b1b-b047-a42026c28f8e-tmp-dir\") pod \"sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q\" (UID: \"64172415-a87a-4b1b-b047-a42026c28f8e\") " pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"
Jul 28 01:55:37.224336 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.224287    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64172415-a87a-4b1b-b047-a42026c28f8e-tls-certs\") pod \"sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q\" (UID: \"64172415-a87a-4b1b-b047-a42026c28f8e\") " pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"
Jul 28 01:55:37.224336 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.224310    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64172415-a87a-4b1b-b047-a42026c28f8e-model-cache\") pod \"sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q\" (UID: \"64172415-a87a-4b1b-b047-a42026c28f8e\") " pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"
Jul 28 01:55:37.224563 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.224463    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64172415-a87a-4b1b-b047-a42026c28f8e-dshm\") pod \"sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q\" (UID: \"64172415-a87a-4b1b-b047-a42026c28f8e\") " pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"
Jul 28 01:55:37.224622 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.224600    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64172415-a87a-4b1b-b047-a42026c28f8e-home\") pod \"sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q\" (UID: \"64172415-a87a-4b1b-b047-a42026c28f8e\") " pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"
Jul 28 01:55:37.224685 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.224626    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64172415-a87a-4b1b-b047-a42026c28f8e-model-cache\") pod \"sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q\" (UID: \"64172415-a87a-4b1b-b047-a42026c28f8e\") " pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"
Jul 28 01:55:37.224685 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.224673    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64172415-a87a-4b1b-b047-a42026c28f8e-tmp-dir\") pod \"sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q\" (UID: \"64172415-a87a-4b1b-b047-a42026c28f8e\") " pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"
Jul 28 01:55:37.226512 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.226482    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64172415-a87a-4b1b-b047-a42026c28f8e-dshm\") pod \"sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q\" (UID: \"64172415-a87a-4b1b-b047-a42026c28f8e\") " pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"
Jul 28 01:55:37.226991 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.226970    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64172415-a87a-4b1b-b047-a42026c28f8e-tls-certs\") pod \"sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q\" (UID: \"64172415-a87a-4b1b-b047-a42026c28f8e\") " pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"
Jul 28 01:55:37.231902 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.231875    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h9lx\" (UniqueName: \"kubernetes.io/projected/64172415-a87a-4b1b-b047-a42026c28f8e-kube-api-access-8h9lx\") pod \"sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q\" (UID: \"64172415-a87a-4b1b-b047-a42026c28f8e\") " pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"
Jul 28 01:55:37.363984 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.363949    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"
Jul 28 01:55:37.508187 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.508161    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"]
Jul 28 01:55:37.509541 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:55:37.509515    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod64172415_a87a_4b1b_b047_a42026c28f8e.slice/crio-8cc94cb9759fa81029661cfc22e05661bd763fd007c7c6f822f3ece04dfbc16f WatchSource:0}: Error finding container 8cc94cb9759fa81029661cfc22e05661bd763fd007c7c6f822f3ece04dfbc16f: Status 404 returned error can't find the container with id 8cc94cb9759fa81029661cfc22e05661bd763fd007c7c6f822f3ece04dfbc16f
Jul 28 01:55:37.639190 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.639109    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q" event={"ID":"64172415-a87a-4b1b-b047-a42026c28f8e","Type":"ContainerStarted","Data":"cded6d10d65cca1a2bd167975e59eb261e630efa33a15c31984249e89401a9b8"}
Jul 28 01:55:37.639190 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.639159    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q" event={"ID":"64172415-a87a-4b1b-b047-a42026c28f8e","Type":"ContainerStarted","Data":"8cc94cb9759fa81029661cfc22e05661bd763fd007c7c6f822f3ece04dfbc16f"}
Jul 28 01:55:37.656642 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:37.656594    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q" podStartSLOduration=0.656573192 podStartE2EDuration="656.573192ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:55:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:55:37.654105993 +0000 UTC m=+3707.818914105" watchObservedRunningTime="2026-07-28 01:55:37.656573192 +0000 UTC m=+3707.821381299"
Jul 28 01:55:47.364671 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:47.364633    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"
Jul 28 01:55:47.367419 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:47.364689    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"
Jul 28 01:55:47.378094 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:47.378072    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"
Jul 28 01:55:47.705355 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:47.705275    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"
Jul 28 01:55:50.937712 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:55:50.937675    2578 scope.go:117] "RemoveContainer" containerID="7cdfbb58e25aa402861b060900a8f12c429ea58b778a0d2ae75bad9b4bfe6c9e"
Jul 28 01:56:42.091194 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:42.091104    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"]
Jul 28 01:56:54.198730 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.198693    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf"]
Jul 28 01:56:54.199129 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.198900    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf" podUID="0e17f9db-5dc6-4321-b9aa-7f1e7401f155" containerName="maas-api" containerID="cri-o://12aab52760c14d86420c252a67f6863a3159d9b3df60681e2dee3082050af295" gracePeriod=30
Jul 28 01:56:54.283262 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.283227    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4"]
Jul 28 01:56:54.283460 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.283438    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4" podUID="20ec5aed-4c5c-42bd-b44b-6f1db3023d33" containerName="payload-processing" containerID="cri-o://d927136143c21c916e703890f268fdb0a1b0301cb33ebd0df6795f055db20fa6" gracePeriod=30
Jul 28 01:56:54.296459 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.296427    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf"]
Jul 28 01:56:54.296788 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.296750    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf" podUID="7dfe7cc1-54cf-4ce8-8314-438e2ec8decd" containerName="payload-pre-processing" containerID="cri-o://2f6a27867ebad0974e248ec21b1cbd19f10fc81ae80bdb704e7b120fae87fb9d" gracePeriod=30
Jul 28 01:56:54.558582 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.558551    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf"
Jul 28 01:56:54.588836 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.588814    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4"
Jul 28 01:56:54.601317 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.601296    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf"
Jul 28 01:56:54.614626 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.614598    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e17f9db-5dc6-4321-b9aa-7f1e7401f155-openshift-service-ca\") pod \"0e17f9db-5dc6-4321-b9aa-7f1e7401f155\" (UID: \"0e17f9db-5dc6-4321-b9aa-7f1e7401f155\") "
Jul 28 01:56:54.614787 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.614699    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e17f9db-5dc6-4321-b9aa-7f1e7401f155-maas-api-tls\") pod \"0e17f9db-5dc6-4321-b9aa-7f1e7401f155\" (UID: \"0e17f9db-5dc6-4321-b9aa-7f1e7401f155\") "
Jul 28 01:56:54.614787 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.614731    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4r2wf\" (UniqueName: \"kubernetes.io/projected/0e17f9db-5dc6-4321-b9aa-7f1e7401f155-kube-api-access-4r2wf\") pod \"0e17f9db-5dc6-4321-b9aa-7f1e7401f155\" (UID: \"0e17f9db-5dc6-4321-b9aa-7f1e7401f155\") "
Jul 28 01:56:54.614986 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.614955    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e17f9db-5dc6-4321-b9aa-7f1e7401f155-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0e17f9db-5dc6-4321-b9aa-7f1e7401f155" (UID: "0e17f9db-5dc6-4321-b9aa-7f1e7401f155"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:56:54.615076 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.615043    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e17f9db-5dc6-4321-b9aa-7f1e7401f155-openshift-service-ca\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:56:54.616975 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.616951    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e17f9db-5dc6-4321-b9aa-7f1e7401f155-kube-api-access-4r2wf" (OuterVolumeSpecName: "kube-api-access-4r2wf") pod "0e17f9db-5dc6-4321-b9aa-7f1e7401f155" (UID: "0e17f9db-5dc6-4321-b9aa-7f1e7401f155"). InnerVolumeSpecName "kube-api-access-4r2wf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:56:54.617330 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.617300    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e17f9db-5dc6-4321-b9aa-7f1e7401f155-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0e17f9db-5dc6-4321-b9aa-7f1e7401f155" (UID: "0e17f9db-5dc6-4321-b9aa-7f1e7401f155"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:56:54.716402 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.716311    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rlb6p\" (UniqueName: \"kubernetes.io/projected/20ec5aed-4c5c-42bd-b44b-6f1db3023d33-kube-api-access-rlb6p\") pod \"20ec5aed-4c5c-42bd-b44b-6f1db3023d33\" (UID: \"20ec5aed-4c5c-42bd-b44b-6f1db3023d33\") "
Jul 28 01:56:54.716402 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.716355    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cvfr\" (UniqueName: \"kubernetes.io/projected/7dfe7cc1-54cf-4ce8-8314-438e2ec8decd-kube-api-access-7cvfr\") pod \"7dfe7cc1-54cf-4ce8-8314-438e2ec8decd\" (UID: \"7dfe7cc1-54cf-4ce8-8314-438e2ec8decd\") "
Jul 28 01:56:54.716402 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.716380    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7dfe7cc1-54cf-4ce8-8314-438e2ec8decd-plugins-config-volume\") pod \"7dfe7cc1-54cf-4ce8-8314-438e2ec8decd\" (UID: \"7dfe7cc1-54cf-4ce8-8314-438e2ec8decd\") "
Jul 28 01:56:54.716687 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.716421    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20ec5aed-4c5c-42bd-b44b-6f1db3023d33-plugins-config-volume\") pod \"20ec5aed-4c5c-42bd-b44b-6f1db3023d33\" (UID: \"20ec5aed-4c5c-42bd-b44b-6f1db3023d33\") "
Jul 28 01:56:54.716747 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.716695    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e17f9db-5dc6-4321-b9aa-7f1e7401f155-maas-api-tls\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:56:54.716747 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.716714    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4r2wf\" (UniqueName: \"kubernetes.io/projected/0e17f9db-5dc6-4321-b9aa-7f1e7401f155-kube-api-access-4r2wf\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:56:54.716856 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.716814    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7dfe7cc1-54cf-4ce8-8314-438e2ec8decd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7dfe7cc1-54cf-4ce8-8314-438e2ec8decd" (UID: "7dfe7cc1-54cf-4ce8-8314-438e2ec8decd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:56:54.716913 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.716884    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20ec5aed-4c5c-42bd-b44b-6f1db3023d33-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "20ec5aed-4c5c-42bd-b44b-6f1db3023d33" (UID: "20ec5aed-4c5c-42bd-b44b-6f1db3023d33"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:56:54.718536 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.718511    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dfe7cc1-54cf-4ce8-8314-438e2ec8decd-kube-api-access-7cvfr" (OuterVolumeSpecName: "kube-api-access-7cvfr") pod "7dfe7cc1-54cf-4ce8-8314-438e2ec8decd" (UID: "7dfe7cc1-54cf-4ce8-8314-438e2ec8decd"). InnerVolumeSpecName "kube-api-access-7cvfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:56:54.718632 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.718556    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20ec5aed-4c5c-42bd-b44b-6f1db3023d33-kube-api-access-rlb6p" (OuterVolumeSpecName: "kube-api-access-rlb6p") pod "20ec5aed-4c5c-42bd-b44b-6f1db3023d33" (UID: "20ec5aed-4c5c-42bd-b44b-6f1db3023d33"). InnerVolumeSpecName "kube-api-access-rlb6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:56:54.817745 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.817709    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7dfe7cc1-54cf-4ce8-8314-438e2ec8decd-plugins-config-volume\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:56:54.817745 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.817739    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20ec5aed-4c5c-42bd-b44b-6f1db3023d33-plugins-config-volume\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:56:54.817745 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.817749    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlb6p\" (UniqueName: \"kubernetes.io/projected/20ec5aed-4c5c-42bd-b44b-6f1db3023d33-kube-api-access-rlb6p\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:56:54.817745 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:54.817757    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cvfr\" (UniqueName: \"kubernetes.io/projected/7dfe7cc1-54cf-4ce8-8314-438e2ec8decd-kube-api-access-7cvfr\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:56:55.058979 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:55.058888    2578 generic.go:356] "Generic (PLEG): container finished" podID="20ec5aed-4c5c-42bd-b44b-6f1db3023d33" containerID="d927136143c21c916e703890f268fdb0a1b0301cb33ebd0df6795f055db20fa6" exitCode=0
Jul 28 01:56:55.058979 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:55.058935    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4"
Jul 28 01:56:55.059266 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:55.058987    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4" event={"ID":"20ec5aed-4c5c-42bd-b44b-6f1db3023d33","Type":"ContainerDied","Data":"d927136143c21c916e703890f268fdb0a1b0301cb33ebd0df6795f055db20fa6"}
Jul 28 01:56:55.059266 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:55.059055    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4" event={"ID":"20ec5aed-4c5c-42bd-b44b-6f1db3023d33","Type":"ContainerDied","Data":"38d00fbf77ec9a62418ee94db03c0b3a0909b17ae500e2ebea8f4ba92cb64b7d"}
Jul 28 01:56:55.059266 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:55.059084    2578 scope.go:117] "RemoveContainer" containerID="d927136143c21c916e703890f268fdb0a1b0301cb33ebd0df6795f055db20fa6"
Jul 28 01:56:55.060447 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:55.060422    2578 generic.go:356] "Generic (PLEG): container finished" podID="7dfe7cc1-54cf-4ce8-8314-438e2ec8decd" containerID="2f6a27867ebad0974e248ec21b1cbd19f10fc81ae80bdb704e7b120fae87fb9d" exitCode=0
Jul 28 01:56:55.060582 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:55.060480    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf"
Jul 28 01:56:55.060582 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:55.060510    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf" event={"ID":"7dfe7cc1-54cf-4ce8-8314-438e2ec8decd","Type":"ContainerDied","Data":"2f6a27867ebad0974e248ec21b1cbd19f10fc81ae80bdb704e7b120fae87fb9d"}
Jul 28 01:56:55.060582 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:55.060570    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf" event={"ID":"7dfe7cc1-54cf-4ce8-8314-438e2ec8decd","Type":"ContainerDied","Data":"c44d4ae15ce3e9a3bbbd568a9ddb969ab121aec4ee526b3b26f699c00e14a648"}
Jul 28 01:56:55.063257 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:55.063205    2578 generic.go:356] "Generic (PLEG): container finished" podID="0e17f9db-5dc6-4321-b9aa-7f1e7401f155" containerID="12aab52760c14d86420c252a67f6863a3159d9b3df60681e2dee3082050af295" exitCode=0
Jul 28 01:56:55.063562 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:55.063326    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf" event={"ID":"0e17f9db-5dc6-4321-b9aa-7f1e7401f155","Type":"ContainerDied","Data":"12aab52760c14d86420c252a67f6863a3159d9b3df60681e2dee3082050af295"}
Jul 28 01:56:55.063727 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:55.063710    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf" event={"ID":"0e17f9db-5dc6-4321-b9aa-7f1e7401f155","Type":"ContainerDied","Data":"6250fb63284585a6b3b9da76eb6566f4fbb1a997c6784b679ad16ae56ef07f18"}
Jul 28 01:56:55.063864 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:55.063344    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf"
Jul 28 01:56:55.069800 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:55.069777    2578 scope.go:117] "RemoveContainer" containerID="d927136143c21c916e703890f268fdb0a1b0301cb33ebd0df6795f055db20fa6"
Jul 28 01:56:55.070152 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:56:55.070127    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d927136143c21c916e703890f268fdb0a1b0301cb33ebd0df6795f055db20fa6\": container with ID starting with d927136143c21c916e703890f268fdb0a1b0301cb33ebd0df6795f055db20fa6 not found: ID does not exist" containerID="d927136143c21c916e703890f268fdb0a1b0301cb33ebd0df6795f055db20fa6"
Jul 28 01:56:55.070266 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:55.070162    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d927136143c21c916e703890f268fdb0a1b0301cb33ebd0df6795f055db20fa6"} err="failed to get container status \"d927136143c21c916e703890f268fdb0a1b0301cb33ebd0df6795f055db20fa6\": rpc error: code = NotFound desc = could not find container \"d927136143c21c916e703890f268fdb0a1b0301cb33ebd0df6795f055db20fa6\": container with ID starting with d927136143c21c916e703890f268fdb0a1b0301cb33ebd0df6795f055db20fa6 not found: ID does not exist"
Jul 28 01:56:55.070266 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:55.070188    2578 scope.go:117] "RemoveContainer" containerID="2f6a27867ebad0974e248ec21b1cbd19f10fc81ae80bdb704e7b120fae87fb9d"
Jul 28 01:56:55.080375 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:55.080351    2578 scope.go:117] "RemoveContainer" containerID="2f6a27867ebad0974e248ec21b1cbd19f10fc81ae80bdb704e7b120fae87fb9d"
Jul 28 01:56:55.080677 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:56:55.080656    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f6a27867ebad0974e248ec21b1cbd19f10fc81ae80bdb704e7b120fae87fb9d\": container with ID starting with 2f6a27867ebad0974e248ec21b1cbd19f10fc81ae80bdb704e7b120fae87fb9d not found: ID does not exist" containerID="2f6a27867ebad0974e248ec21b1cbd19f10fc81ae80bdb704e7b120fae87fb9d"
Jul 28 01:56:55.080756 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:55.080692    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f6a27867ebad0974e248ec21b1cbd19f10fc81ae80bdb704e7b120fae87fb9d"} err="failed to get container status \"2f6a27867ebad0974e248ec21b1cbd19f10fc81ae80bdb704e7b120fae87fb9d\": rpc error: code = NotFound desc = could not find container \"2f6a27867ebad0974e248ec21b1cbd19f10fc81ae80bdb704e7b120fae87fb9d\": container with ID starting with 2f6a27867ebad0974e248ec21b1cbd19f10fc81ae80bdb704e7b120fae87fb9d not found: ID does not exist"
Jul 28 01:56:55.080756 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:55.080717    2578 scope.go:117] "RemoveContainer" containerID="12aab52760c14d86420c252a67f6863a3159d9b3df60681e2dee3082050af295"
Jul 28 01:56:55.089617 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:55.089594    2578 scope.go:117] "RemoveContainer" containerID="12aab52760c14d86420c252a67f6863a3159d9b3df60681e2dee3082050af295"
Jul 28 01:56:55.089968 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:56:55.089949    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12aab52760c14d86420c252a67f6863a3159d9b3df60681e2dee3082050af295\": container with ID starting with 12aab52760c14d86420c252a67f6863a3159d9b3df60681e2dee3082050af295 not found: ID does not exist" containerID="12aab52760c14d86420c252a67f6863a3159d9b3df60681e2dee3082050af295"
Jul 28 01:56:55.090012 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:55.089977    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12aab52760c14d86420c252a67f6863a3159d9b3df60681e2dee3082050af295"} err="failed to get container status \"12aab52760c14d86420c252a67f6863a3159d9b3df60681e2dee3082050af295\": rpc error: code = NotFound desc = could not find container \"12aab52760c14d86420c252a67f6863a3159d9b3df60681e2dee3082050af295\": container with ID starting with 12aab52760c14d86420c252a67f6863a3159d9b3df60681e2dee3082050af295 not found: ID does not exist"
Jul 28 01:56:55.091913 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:55.091893    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4"]
Jul 28 01:56:55.098108 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:55.098080    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-73e2fd-66b47b8c4b-k92h4"]
Jul 28 01:56:55.109349 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:55.109322    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf"]
Jul 28 01:56:55.114388 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:55.114363    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-73e2fd-bf98b6b4d-2gvvf"]
Jul 28 01:56:55.128859 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:55.128832    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf"]
Jul 28 01:56:55.136533 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:55.136503    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-73e2fd-69b7cfd445-jjxxf"]
Jul 28 01:56:56.402710 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:56.402680    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e17f9db-5dc6-4321-b9aa-7f1e7401f155" path="/var/lib/kubelet/pods/0e17f9db-5dc6-4321-b9aa-7f1e7401f155/volumes"
Jul 28 01:56:56.403109 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:56.403074    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20ec5aed-4c5c-42bd-b44b-6f1db3023d33" path="/var/lib/kubelet/pods/20ec5aed-4c5c-42bd-b44b-6f1db3023d33/volumes"
Jul 28 01:56:56.403423 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:56.403408    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7dfe7cc1-54cf-4ce8-8314-438e2ec8decd" path="/var/lib/kubelet/pods/7dfe7cc1-54cf-4ce8-8314-438e2ec8decd/volumes"
Jul 28 01:56:57.111347 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:57.111304    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q" podUID="64172415-a87a-4b1b-b047-a42026c28f8e" containerName="main" containerID="cri-o://cded6d10d65cca1a2bd167975e59eb261e630efa33a15c31984249e89401a9b8" gracePeriod=45
Jul 28 01:56:57.368119 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:57.368059    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"
Jul 28 01:56:57.442700 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:57.442670    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64172415-a87a-4b1b-b047-a42026c28f8e-model-cache\") pod \"64172415-a87a-4b1b-b047-a42026c28f8e\" (UID: \"64172415-a87a-4b1b-b047-a42026c28f8e\") "
Jul 28 01:56:57.443139 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:57.442714    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64172415-a87a-4b1b-b047-a42026c28f8e-home\") pod \"64172415-a87a-4b1b-b047-a42026c28f8e\" (UID: \"64172415-a87a-4b1b-b047-a42026c28f8e\") "
Jul 28 01:56:57.443139 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:57.442744    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64172415-a87a-4b1b-b047-a42026c28f8e-tls-certs\") pod \"64172415-a87a-4b1b-b047-a42026c28f8e\" (UID: \"64172415-a87a-4b1b-b047-a42026c28f8e\") "
Jul 28 01:56:57.443139 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:57.442817    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8h9lx\" (UniqueName: \"kubernetes.io/projected/64172415-a87a-4b1b-b047-a42026c28f8e-kube-api-access-8h9lx\") pod \"64172415-a87a-4b1b-b047-a42026c28f8e\" (UID: \"64172415-a87a-4b1b-b047-a42026c28f8e\") "
Jul 28 01:56:57.443139 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:57.442877    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64172415-a87a-4b1b-b047-a42026c28f8e-dshm\") pod \"64172415-a87a-4b1b-b047-a42026c28f8e\" (UID: \"64172415-a87a-4b1b-b047-a42026c28f8e\") "
Jul 28 01:56:57.443139 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:57.442935    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64172415-a87a-4b1b-b047-a42026c28f8e-tmp-dir\") pod \"64172415-a87a-4b1b-b047-a42026c28f8e\" (UID: \"64172415-a87a-4b1b-b047-a42026c28f8e\") "
Jul 28 01:56:57.443139 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:57.442987    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64172415-a87a-4b1b-b047-a42026c28f8e-model-cache" (OuterVolumeSpecName: "model-cache") pod "64172415-a87a-4b1b-b047-a42026c28f8e" (UID: "64172415-a87a-4b1b-b047-a42026c28f8e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:56:57.443139 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:57.443001    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64172415-a87a-4b1b-b047-a42026c28f8e-home" (OuterVolumeSpecName: "home") pod "64172415-a87a-4b1b-b047-a42026c28f8e" (UID: "64172415-a87a-4b1b-b047-a42026c28f8e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:56:57.443496 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:57.443257    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64172415-a87a-4b1b-b047-a42026c28f8e-model-cache\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:56:57.443496 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:57.443276    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64172415-a87a-4b1b-b047-a42026c28f8e-home\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:56:57.443496 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:57.443288    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64172415-a87a-4b1b-b047-a42026c28f8e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "64172415-a87a-4b1b-b047-a42026c28f8e" (UID: "64172415-a87a-4b1b-b047-a42026c28f8e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:56:57.445001 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:57.444972    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64172415-a87a-4b1b-b047-a42026c28f8e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "64172415-a87a-4b1b-b047-a42026c28f8e" (UID: "64172415-a87a-4b1b-b047-a42026c28f8e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:56:57.445336 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:57.445316    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64172415-a87a-4b1b-b047-a42026c28f8e-dshm" (OuterVolumeSpecName: "dshm") pod "64172415-a87a-4b1b-b047-a42026c28f8e" (UID: "64172415-a87a-4b1b-b047-a42026c28f8e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:56:57.445413 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:57.445362    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64172415-a87a-4b1b-b047-a42026c28f8e-kube-api-access-8h9lx" (OuterVolumeSpecName: "kube-api-access-8h9lx") pod "64172415-a87a-4b1b-b047-a42026c28f8e" (UID: "64172415-a87a-4b1b-b047-a42026c28f8e"). InnerVolumeSpecName "kube-api-access-8h9lx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:56:57.544539 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:57.544507    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64172415-a87a-4b1b-b047-a42026c28f8e-tls-certs\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:56:57.544539 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:57.544537    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8h9lx\" (UniqueName: \"kubernetes.io/projected/64172415-a87a-4b1b-b047-a42026c28f8e-kube-api-access-8h9lx\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:56:57.544539 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:57.544547    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64172415-a87a-4b1b-b047-a42026c28f8e-dshm\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:56:57.544776 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:57.544557    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64172415-a87a-4b1b-b047-a42026c28f8e-tmp-dir\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:56:58.084916 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:58.084884    2578 generic.go:356] "Generic (PLEG): container finished" podID="64172415-a87a-4b1b-b047-a42026c28f8e" containerID="cded6d10d65cca1a2bd167975e59eb261e630efa33a15c31984249e89401a9b8" exitCode=0
Jul 28 01:56:58.085066 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:58.084926    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"
Jul 28 01:56:58.085066 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:58.084968    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q" event={"ID":"64172415-a87a-4b1b-b047-a42026c28f8e","Type":"ContainerDied","Data":"cded6d10d65cca1a2bd167975e59eb261e630efa33a15c31984249e89401a9b8"}
Jul 28 01:56:58.085066 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:58.085011    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q" event={"ID":"64172415-a87a-4b1b-b047-a42026c28f8e","Type":"ContainerDied","Data":"8cc94cb9759fa81029661cfc22e05661bd763fd007c7c6f822f3ece04dfbc16f"}
Jul 28 01:56:58.085066 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:58.085029    2578 scope.go:117] "RemoveContainer" containerID="cded6d10d65cca1a2bd167975e59eb261e630efa33a15c31984249e89401a9b8"
Jul 28 01:56:58.094185 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:58.094165    2578 scope.go:117] "RemoveContainer" containerID="cded6d10d65cca1a2bd167975e59eb261e630efa33a15c31984249e89401a9b8"
Jul 28 01:56:58.094463 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:56:58.094444    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cded6d10d65cca1a2bd167975e59eb261e630efa33a15c31984249e89401a9b8\": container with ID starting with cded6d10d65cca1a2bd167975e59eb261e630efa33a15c31984249e89401a9b8 not found: ID does not exist" containerID="cded6d10d65cca1a2bd167975e59eb261e630efa33a15c31984249e89401a9b8"
Jul 28 01:56:58.094519 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:58.094472    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cded6d10d65cca1a2bd167975e59eb261e630efa33a15c31984249e89401a9b8"} err="failed to get container status \"cded6d10d65cca1a2bd167975e59eb261e630efa33a15c31984249e89401a9b8\": rpc error: code = NotFound desc = could not find container \"cded6d10d65cca1a2bd167975e59eb261e630efa33a15c31984249e89401a9b8\": container with ID starting with cded6d10d65cca1a2bd167975e59eb261e630efa33a15c31984249e89401a9b8 not found: ID does not exist"
Jul 28 01:56:58.106724 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:58.106696    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"]
Jul 28 01:56:58.112851 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:58.112823    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-73e2fd/sub-test-model-73e2fd-kserve-78cbdb6f65-z9z9q"]
Jul 28 01:56:58.402666 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:56:58.402636    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64172415-a87a-4b1b-b047-a42026c28f8e" path="/var/lib/kubelet/pods/64172415-a87a-4b1b-b047-a42026c28f8e/volumes"
Jul 28 01:57:05.190315 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:05.190270    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7d2c59-xjff4"]
Jul 28 01:57:05.191236 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:05.191184    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dfe7cc1-54cf-4ce8-8314-438e2ec8decd" containerName="payload-pre-processing"
Jul 28 01:57:05.191236 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:05.191228    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dfe7cc1-54cf-4ce8-8314-438e2ec8decd" containerName="payload-pre-processing"
Jul 28 01:57:05.191427 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:05.191249    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64172415-a87a-4b1b-b047-a42026c28f8e" containerName="main"
Jul 28 01:57:05.191427 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:05.191258    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="64172415-a87a-4b1b-b047-a42026c28f8e" containerName="main"
Jul 28 01:57:05.191427 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:05.191272    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20ec5aed-4c5c-42bd-b44b-6f1db3023d33" containerName="payload-processing"
Jul 28 01:57:05.191427 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:05.191287    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="20ec5aed-4c5c-42bd-b44b-6f1db3023d33" containerName="payload-processing"
Jul 28 01:57:05.191427 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:05.191303    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e17f9db-5dc6-4321-b9aa-7f1e7401f155" containerName="maas-api"
Jul 28 01:57:05.191427 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:05.191311    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e17f9db-5dc6-4321-b9aa-7f1e7401f155" containerName="maas-api"
Jul 28 01:57:05.191669 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:05.191541    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e17f9db-5dc6-4321-b9aa-7f1e7401f155" containerName="maas-api"
Jul 28 01:57:05.191669 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:05.191563    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="64172415-a87a-4b1b-b047-a42026c28f8e" containerName="main"
Jul 28 01:57:05.191669 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:05.191575    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7dfe7cc1-54cf-4ce8-8314-438e2ec8decd" containerName="payload-pre-processing"
Jul 28 01:57:05.191669 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:05.191593    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="20ec5aed-4c5c-42bd-b44b-6f1db3023d33" containerName="payload-processing"
Jul 28 01:57:05.195948 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:05.195922    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7d2c59-xjff4"
Jul 28 01:57:05.197947 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:05.197919    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7d2c59-xjff4"]
Jul 28 01:57:05.198476 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:05.198459    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8ph8w\""
Jul 28 01:57:05.326043 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:05.326004    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb35b862-2f00-49c5-b095-c3a21a9b436a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-7d2c59-xjff4\" (UID: \"bb35b862-2f00-49c5-b095-c3a21a9b436a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7d2c59-xjff4"
Jul 28 01:57:05.426809 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:05.426779    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb35b862-2f00-49c5-b095-c3a21a9b436a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-7d2c59-xjff4\" (UID: \"bb35b862-2f00-49c5-b095-c3a21a9b436a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7d2c59-xjff4"
Jul 28 01:57:05.427406 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:05.427386    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb35b862-2f00-49c5-b095-c3a21a9b436a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-7d2c59-xjff4\" (UID: \"bb35b862-2f00-49c5-b095-c3a21a9b436a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7d2c59-xjff4"
Jul 28 01:57:05.508947 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:05.508855    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7d2c59-xjff4"
Jul 28 01:57:08.651944 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:08.651919    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7d2c59-xjff4"]
Jul 28 01:57:08.654143 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:57:08.654120    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb35b862_2f00_49c5_b095_c3a21a9b436a.slice/crio-4324fb33e8ae0f832cf3ebd0ccb3cb644ee5db571570bd3177b26b8711611355 WatchSource:0}: Error finding container 4324fb33e8ae0f832cf3ebd0ccb3cb644ee5db571570bd3177b26b8711611355: Status 404 returned error can't find the container with id 4324fb33e8ae0f832cf3ebd0ccb3cb644ee5db571570bd3177b26b8711611355
Jul 28 01:57:09.147419 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:09.147379    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7d2c59-xjff4" event={"ID":"bb35b862-2f00-49c5-b095-c3a21a9b436a","Type":"ContainerStarted","Data":"e5e01bdc2c0fc983a87500e3d37da4405fbbfed33876cbde97bde869c606b8d3"}
Jul 28 01:57:09.147568 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:09.147439    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7d2c59-xjff4" event={"ID":"bb35b862-2f00-49c5-b095-c3a21a9b436a","Type":"ContainerStarted","Data":"4324fb33e8ae0f832cf3ebd0ccb3cb644ee5db571570bd3177b26b8711611355"}
Jul 28 01:57:09.162165 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:09.162109    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7d2c59-xjff4" podStartSLOduration=4.162092123 podStartE2EDuration="4.162092123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:57:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:57:09.161904434 +0000 UTC m=+3799.326712541" watchObservedRunningTime="2026-07-28 01:57:09.162092123 +0000 UTC m=+3799.326900231"
Jul 28 01:57:10.154545 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:10.154505    2578 generic.go:356] "Generic (PLEG): container finished" podID="bb35b862-2f00-49c5-b095-c3a21a9b436a" containerID="e5e01bdc2c0fc983a87500e3d37da4405fbbfed33876cbde97bde869c606b8d3" exitCode=0
Jul 28 01:57:10.155025 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:10.154593    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7d2c59-xjff4" event={"ID":"bb35b862-2f00-49c5-b095-c3a21a9b436a","Type":"ContainerDied","Data":"e5e01bdc2c0fc983a87500e3d37da4405fbbfed33876cbde97bde869c606b8d3"}
Jul 28 01:57:11.297168 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:11.297139    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7d2c59-xjff4"
Jul 28 01:57:11.389268 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:11.389206    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb35b862-2f00-49c5-b095-c3a21a9b436a-maas-api-service-ca\") pod \"bb35b862-2f00-49c5-b095-c3a21a9b436a\" (UID: \"bb35b862-2f00-49c5-b095-c3a21a9b436a\") "
Jul 28 01:57:11.389636 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:11.389613    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb35b862-2f00-49c5-b095-c3a21a9b436a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bb35b862-2f00-49c5-b095-c3a21a9b436a" (UID: "bb35b862-2f00-49c5-b095-c3a21a9b436a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:57:11.490420 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:11.490327    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb35b862-2f00-49c5-b095-c3a21a9b436a-maas-api-service-ca\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:57:12.166975 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:12.166943    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7d2c59-xjff4"
Jul 28 01:57:12.167153 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:12.166971    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7d2c59-xjff4" event={"ID":"bb35b862-2f00-49c5-b095-c3a21a9b436a","Type":"ContainerDied","Data":"4324fb33e8ae0f832cf3ebd0ccb3cb644ee5db571570bd3177b26b8711611355"}
Jul 28 01:57:12.167153 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:12.167054    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4324fb33e8ae0f832cf3ebd0ccb3cb644ee5db571570bd3177b26b8711611355"
Jul 28 01:57:15.780023 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:15.779983    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x"]
Jul 28 01:57:15.780457 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:15.780233    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x" podUID="4cbcb559-bb6f-4248-998a-47e6febbeb12" containerName="maas-api" containerID="cri-o://eef9513dd9db29978b403aef45146a03fc5d3206378d0e2d322921919abce64f" gracePeriod=30
Jul 28 01:57:15.834336 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:15.834298    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx"]
Jul 28 01:57:15.834581 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:15.834554    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx" podUID="add66961-dca0-4db6-b815-18047d840472" containerName="payload-processing" containerID="cri-o://7e9b7510ed0acda3fa5da63aa46996b5b09ab2c5cd0b804d93f3251d7f8e4951" gracePeriod=30
Jul 28 01:57:15.842996 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:15.842965    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f"]
Jul 28 01:57:15.843236 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:15.843193    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f" podUID="174fb214-b943-448c-bfe7-7317f889a50f" containerName="payload-pre-processing" containerID="cri-o://8d0a409aa0a8684670dd20f561a08f34717fa8cd0e746925c43f22acee451968" gracePeriod=30
Jul 28 01:57:16.117174 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.117132    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x"
Jul 28 01:57:16.157068 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.157044    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx"
Jul 28 01:57:16.173296 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.172855    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f"
Jul 28 01:57:16.189613 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.189289    2578 generic.go:356] "Generic (PLEG): container finished" podID="add66961-dca0-4db6-b815-18047d840472" containerID="7e9b7510ed0acda3fa5da63aa46996b5b09ab2c5cd0b804d93f3251d7f8e4951" exitCode=0
Jul 28 01:57:16.189613 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.189400    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx" event={"ID":"add66961-dca0-4db6-b815-18047d840472","Type":"ContainerDied","Data":"7e9b7510ed0acda3fa5da63aa46996b5b09ab2c5cd0b804d93f3251d7f8e4951"}
Jul 28 01:57:16.189613 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.189452    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx" event={"ID":"add66961-dca0-4db6-b815-18047d840472","Type":"ContainerDied","Data":"1b883323a0f50b07c5ca6d5c89e5145e99eca5ab2f1bb70098c64cbe7ea322bc"}
Jul 28 01:57:16.189613 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.189483    2578 scope.go:117] "RemoveContainer" containerID="7e9b7510ed0acda3fa5da63aa46996b5b09ab2c5cd0b804d93f3251d7f8e4951"
Jul 28 01:57:16.189613 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.189496    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx"
Jul 28 01:57:16.191898 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.191657    2578 generic.go:356] "Generic (PLEG): container finished" podID="174fb214-b943-448c-bfe7-7317f889a50f" containerID="8d0a409aa0a8684670dd20f561a08f34717fa8cd0e746925c43f22acee451968" exitCode=0
Jul 28 01:57:16.191898 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.191698    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f" event={"ID":"174fb214-b943-448c-bfe7-7317f889a50f","Type":"ContainerDied","Data":"8d0a409aa0a8684670dd20f561a08f34717fa8cd0e746925c43f22acee451968"}
Jul 28 01:57:16.191898 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.191767    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f" event={"ID":"174fb214-b943-448c-bfe7-7317f889a50f","Type":"ContainerDied","Data":"89db70875584c0e16c0be7c15204c290c1a009c65b1125f3b6d398e120e28961"}
Jul 28 01:57:16.191898 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.191710    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f"
Jul 28 01:57:16.193505 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.193480    2578 generic.go:356] "Generic (PLEG): container finished" podID="4cbcb559-bb6f-4248-998a-47e6febbeb12" containerID="eef9513dd9db29978b403aef45146a03fc5d3206378d0e2d322921919abce64f" exitCode=0
Jul 28 01:57:16.193623 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.193521    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x" event={"ID":"4cbcb559-bb6f-4248-998a-47e6febbeb12","Type":"ContainerDied","Data":"eef9513dd9db29978b403aef45146a03fc5d3206378d0e2d322921919abce64f"}
Jul 28 01:57:16.193623 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.193526    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x"
Jul 28 01:57:16.193623 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.193558    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x" event={"ID":"4cbcb559-bb6f-4248-998a-47e6febbeb12","Type":"ContainerDied","Data":"6e2ba4ca3f2402c29a9c42017e4602d47ac8fed9e50cee48b3333be63dc8aecb"}
Jul 28 01:57:16.201957 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.201926    2578 scope.go:117] "RemoveContainer" containerID="7e9b7510ed0acda3fa5da63aa46996b5b09ab2c5cd0b804d93f3251d7f8e4951"
Jul 28 01:57:16.202286 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:57:16.202263    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e9b7510ed0acda3fa5da63aa46996b5b09ab2c5cd0b804d93f3251d7f8e4951\": container with ID starting with 7e9b7510ed0acda3fa5da63aa46996b5b09ab2c5cd0b804d93f3251d7f8e4951 not found: ID does not exist" containerID="7e9b7510ed0acda3fa5da63aa46996b5b09ab2c5cd0b804d93f3251d7f8e4951"
Jul 28 01:57:16.202352 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.202296    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e9b7510ed0acda3fa5da63aa46996b5b09ab2c5cd0b804d93f3251d7f8e4951"} err="failed to get container status \"7e9b7510ed0acda3fa5da63aa46996b5b09ab2c5cd0b804d93f3251d7f8e4951\": rpc error: code = NotFound desc = could not find container \"7e9b7510ed0acda3fa5da63aa46996b5b09ab2c5cd0b804d93f3251d7f8e4951\": container with ID starting with 7e9b7510ed0acda3fa5da63aa46996b5b09ab2c5cd0b804d93f3251d7f8e4951 not found: ID does not exist"
Jul 28 01:57:16.202352 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.202313    2578 scope.go:117] "RemoveContainer" containerID="8d0a409aa0a8684670dd20f561a08f34717fa8cd0e746925c43f22acee451968"
Jul 28 01:57:16.211649 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.211632    2578 scope.go:117] "RemoveContainer" containerID="8d0a409aa0a8684670dd20f561a08f34717fa8cd0e746925c43f22acee451968"
Jul 28 01:57:16.211917 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:57:16.211901    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d0a409aa0a8684670dd20f561a08f34717fa8cd0e746925c43f22acee451968\": container with ID starting with 8d0a409aa0a8684670dd20f561a08f34717fa8cd0e746925c43f22acee451968 not found: ID does not exist" containerID="8d0a409aa0a8684670dd20f561a08f34717fa8cd0e746925c43f22acee451968"
Jul 28 01:57:16.211974 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.211925    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d0a409aa0a8684670dd20f561a08f34717fa8cd0e746925c43f22acee451968"} err="failed to get container status \"8d0a409aa0a8684670dd20f561a08f34717fa8cd0e746925c43f22acee451968\": rpc error: code = NotFound desc = could not find container \"8d0a409aa0a8684670dd20f561a08f34717fa8cd0e746925c43f22acee451968\": container with ID starting with 8d0a409aa0a8684670dd20f561a08f34717fa8cd0e746925c43f22acee451968 not found: ID does not exist"
Jul 28 01:57:16.211974 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.211942    2578 scope.go:117] "RemoveContainer" containerID="eef9513dd9db29978b403aef45146a03fc5d3206378d0e2d322921919abce64f"
Jul 28 01:57:16.220167 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.220147    2578 scope.go:117] "RemoveContainer" containerID="eef9513dd9db29978b403aef45146a03fc5d3206378d0e2d322921919abce64f"
Jul 28 01:57:16.220459 ip-10-0-132-238 kubenswrapper[2578]: E0728 01:57:16.220436    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eef9513dd9db29978b403aef45146a03fc5d3206378d0e2d322921919abce64f\": container with ID starting with eef9513dd9db29978b403aef45146a03fc5d3206378d0e2d322921919abce64f not found: ID does not exist" containerID="eef9513dd9db29978b403aef45146a03fc5d3206378d0e2d322921919abce64f"
Jul 28 01:57:16.220547 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.220470    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eef9513dd9db29978b403aef45146a03fc5d3206378d0e2d322921919abce64f"} err="failed to get container status \"eef9513dd9db29978b403aef45146a03fc5d3206378d0e2d322921919abce64f\": rpc error: code = NotFound desc = could not find container \"eef9513dd9db29978b403aef45146a03fc5d3206378d0e2d322921919abce64f\": container with ID starting with eef9513dd9db29978b403aef45146a03fc5d3206378d0e2d322921919abce64f not found: ID does not exist"
Jul 28 01:57:16.237791 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.237764    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44c7d\" (UniqueName: \"kubernetes.io/projected/174fb214-b943-448c-bfe7-7317f889a50f-kube-api-access-44c7d\") pod \"174fb214-b943-448c-bfe7-7317f889a50f\" (UID: \"174fb214-b943-448c-bfe7-7317f889a50f\") "
Jul 28 01:57:16.237896 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.237831    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ql545\" (UniqueName: \"kubernetes.io/projected/4cbcb559-bb6f-4248-998a-47e6febbeb12-kube-api-access-ql545\") pod \"4cbcb559-bb6f-4248-998a-47e6febbeb12\" (UID: \"4cbcb559-bb6f-4248-998a-47e6febbeb12\") "
Jul 28 01:57:16.238006 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.237982    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cbcb559-bb6f-4248-998a-47e6febbeb12-openshift-service-ca\") pod \"4cbcb559-bb6f-4248-998a-47e6febbeb12\" (UID: \"4cbcb559-bb6f-4248-998a-47e6febbeb12\") "
Jul 28 01:57:16.238062 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.238037    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8n9t\" (UniqueName: \"kubernetes.io/projected/add66961-dca0-4db6-b815-18047d840472-kube-api-access-r8n9t\") pod \"add66961-dca0-4db6-b815-18047d840472\" (UID: \"add66961-dca0-4db6-b815-18047d840472\") "
Jul 28 01:57:16.238119 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.238111    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/add66961-dca0-4db6-b815-18047d840472-plugins-config-volume\") pod \"add66961-dca0-4db6-b815-18047d840472\" (UID: \"add66961-dca0-4db6-b815-18047d840472\") "
Jul 28 01:57:16.238178 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.238152    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cbcb559-bb6f-4248-998a-47e6febbeb12-maas-api-tls\") pod \"4cbcb559-bb6f-4248-998a-47e6febbeb12\" (UID: \"4cbcb559-bb6f-4248-998a-47e6febbeb12\") "
Jul 28 01:57:16.238284 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.238251    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/174fb214-b943-448c-bfe7-7317f889a50f-plugins-config-volume\") pod \"174fb214-b943-448c-bfe7-7317f889a50f\" (UID: \"174fb214-b943-448c-bfe7-7317f889a50f\") "
Jul 28 01:57:16.238643 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.238617    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/add66961-dca0-4db6-b815-18047d840472-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "add66961-dca0-4db6-b815-18047d840472" (UID: "add66961-dca0-4db6-b815-18047d840472"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:57:16.238745 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.238645    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cbcb559-bb6f-4248-998a-47e6febbeb12-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4cbcb559-bb6f-4248-998a-47e6febbeb12" (UID: "4cbcb559-bb6f-4248-998a-47e6febbeb12"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:57:16.238745 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.238687    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/174fb214-b943-448c-bfe7-7317f889a50f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "174fb214-b943-448c-bfe7-7317f889a50f" (UID: "174fb214-b943-448c-bfe7-7317f889a50f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:57:16.240077 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.240051    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/add66961-dca0-4db6-b815-18047d840472-kube-api-access-r8n9t" (OuterVolumeSpecName: "kube-api-access-r8n9t") pod "add66961-dca0-4db6-b815-18047d840472" (UID: "add66961-dca0-4db6-b815-18047d840472"). InnerVolumeSpecName "kube-api-access-r8n9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:57:16.240175 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.240157    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cbcb559-bb6f-4248-998a-47e6febbeb12-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4cbcb559-bb6f-4248-998a-47e6febbeb12" (UID: "4cbcb559-bb6f-4248-998a-47e6febbeb12"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:57:16.240375 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.240357    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cbcb559-bb6f-4248-998a-47e6febbeb12-kube-api-access-ql545" (OuterVolumeSpecName: "kube-api-access-ql545") pod "4cbcb559-bb6f-4248-998a-47e6febbeb12" (UID: "4cbcb559-bb6f-4248-998a-47e6febbeb12"). InnerVolumeSpecName "kube-api-access-ql545". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:57:16.240375 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.240366    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/174fb214-b943-448c-bfe7-7317f889a50f-kube-api-access-44c7d" (OuterVolumeSpecName: "kube-api-access-44c7d") pod "174fb214-b943-448c-bfe7-7317f889a50f" (UID: "174fb214-b943-448c-bfe7-7317f889a50f"). InnerVolumeSpecName "kube-api-access-44c7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:57:16.339401 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.339309    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8n9t\" (UniqueName: \"kubernetes.io/projected/add66961-dca0-4db6-b815-18047d840472-kube-api-access-r8n9t\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:57:16.339401 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.339339    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/add66961-dca0-4db6-b815-18047d840472-plugins-config-volume\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:57:16.339401 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.339350    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cbcb559-bb6f-4248-998a-47e6febbeb12-maas-api-tls\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:57:16.339401 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.339374    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/174fb214-b943-448c-bfe7-7317f889a50f-plugins-config-volume\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:57:16.339401 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.339385    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44c7d\" (UniqueName: \"kubernetes.io/projected/174fb214-b943-448c-bfe7-7317f889a50f-kube-api-access-44c7d\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:57:16.339401 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.339394    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ql545\" (UniqueName: \"kubernetes.io/projected/4cbcb559-bb6f-4248-998a-47e6febbeb12-kube-api-access-ql545\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:57:16.339401 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.339405    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cbcb559-bb6f-4248-998a-47e6febbeb12-openshift-service-ca\") on node \"ip-10-0-132-238.ec2.internal\" DevicePath \"\""
Jul 28 01:57:16.508676 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.508640    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx"]
Jul 28 01:57:16.512852 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.512825    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-7d2c59-5b64ff7d94-f2ldx"]
Jul 28 01:57:16.523697 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.523672    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x"]
Jul 28 01:57:16.529317 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.529294    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7d2c59-74687fd4f9-4mx4x"]
Jul 28 01:57:16.539515 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.539491    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f"]
Jul 28 01:57:16.545309 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:16.545287    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-7d2c59-555b74ccfb-9cj7f"]
Jul 28 01:57:18.402444 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:18.402412    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="174fb214-b943-448c-bfe7-7317f889a50f" path="/var/lib/kubelet/pods/174fb214-b943-448c-bfe7-7317f889a50f/volumes"
Jul 28 01:57:18.402838 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:18.402815    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cbcb559-bb6f-4248-998a-47e6febbeb12" path="/var/lib/kubelet/pods/4cbcb559-bb6f-4248-998a-47e6febbeb12/volumes"
Jul 28 01:57:18.403127 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:18.403114    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="add66961-dca0-4db6-b815-18047d840472" path="/var/lib/kubelet/pods/add66961-dca0-4db6-b815-18047d840472/volumes"
Jul 28 01:57:20.267339 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:20.267307    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7d2c59-xjff4"]
Jul 28 01:57:20.269777 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:20.269753    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7d2c59-xjff4"]
Jul 28 01:57:20.402290 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:20.402261    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb35b862-2f00-49c5-b095-c3a21a9b436a" path="/var/lib/kubelet/pods/bb35b862-2f00-49c5-b095-c3a21a9b436a/volumes"
Jul 28 01:57:33.678611 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:33.678583    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6549c9954f-b8gxc_ccd038af-eb0f-4256-9b99-f6aed1604b13/copy-manifests/0.log"
Jul 28 01:57:33.696606 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:33.696574    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6549c9954f-b8gxc_ccd038af-eb0f-4256-9b99-f6aed1604b13/manager/0.log"
Jul 28 01:57:34.249683 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:34.249658    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29753355-qqhg7_39490d82-139f-4115-82fa-9f51f159433f/cleanup/0.log"
Jul 28 01:57:34.358975 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:34.358938    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29753370-sz8nm_414afbe9-fbbf-4b8f-aa86-1cd59624646b/cleanup/0.log"
Jul 28 01:57:34.466639 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:34.466609    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29753385-h76mj_a11c4d37-aa38-4789-90f2-2978495cfecd/cleanup/0.log"
Jul 28 01:57:36.087177 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:36.087143    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-bb8n5_96996e3b-d506-43f6-b825-a126c7da9720/manager/0.log"
Jul 28 01:57:36.199356 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:36.199327    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-gmml8_503b066d-ed59-42da-8aa1-e23fedc4157a/kuadrant-console-plugin/0.log"
Jul 28 01:57:36.434437 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:36.434401    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-qhx46_ed3db210-4391-45ce-a306-6f9a4a948496/manager/0.log"
Jul 28 01:57:37.321655 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:37.321629    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-fbf9bd945-z8bb2_da1e8311-1c0d-4ff5-b203-74171e7c80e8/kube-auth-proxy/0.log"
Jul 28 01:57:38.125663 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:38.125630    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-fxxzj_7d3c1751-c420-4949-b199-9b86bf99183c/main/0.log"
Jul 28 01:57:38.679351 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:38.679318    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-fhjcx_9b8e9d3d-446f-4241-b181-e78333d4f5a9/main/0.log"
Jul 28 01:57:43.807549 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:43.807514    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hgbjn/must-gather-fpbn5"]
Jul 28 01:57:43.808032 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:43.807907    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="add66961-dca0-4db6-b815-18047d840472" containerName="payload-processing"
Jul 28 01:57:43.808032 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:43.807918    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="add66961-dca0-4db6-b815-18047d840472" containerName="payload-processing"
Jul 28 01:57:43.808032 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:43.807929    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb35b862-2f00-49c5-b095-c3a21a9b436a" containerName="revoke-keys"
Jul 28 01:57:43.808032 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:43.807935    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb35b862-2f00-49c5-b095-c3a21a9b436a" containerName="revoke-keys"
Jul 28 01:57:43.808032 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:43.807943    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="174fb214-b943-448c-bfe7-7317f889a50f" containerName="payload-pre-processing"
Jul 28 01:57:43.808032 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:43.807949    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="174fb214-b943-448c-bfe7-7317f889a50f" containerName="payload-pre-processing"
Jul 28 01:57:43.808032 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:43.807967    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cbcb559-bb6f-4248-998a-47e6febbeb12" containerName="maas-api"
Jul 28 01:57:43.808032 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:43.807974    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cbcb559-bb6f-4248-998a-47e6febbeb12" containerName="maas-api"
Jul 28 01:57:43.808032 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:43.808036    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="174fb214-b943-448c-bfe7-7317f889a50f" containerName="payload-pre-processing"
Jul 28 01:57:43.808339 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:43.808048    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb35b862-2f00-49c5-b095-c3a21a9b436a" containerName="revoke-keys"
Jul 28 01:57:43.808339 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:43.808054    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cbcb559-bb6f-4248-998a-47e6febbeb12" containerName="maas-api"
Jul 28 01:57:43.808339 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:43.808061    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="add66961-dca0-4db6-b815-18047d840472" containerName="payload-processing"
Jul 28 01:57:43.811311 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:43.811292    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hgbjn/must-gather-fpbn5"
Jul 28 01:57:43.813960 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:43.813930    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hgbjn\"/\"default-dockercfg-vscst\""
Jul 28 01:57:43.814183 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:43.814132    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hgbjn\"/\"kube-root-ca.crt\""
Jul 28 01:57:43.814280 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:43.814190    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hgbjn\"/\"openshift-service-ca.crt\""
Jul 28 01:57:43.823906 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:43.823876    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hgbjn/must-gather-fpbn5"]
Jul 28 01:57:44.009524 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:44.009474    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/710ea447-416a-4000-8f95-28e6319a1aea-must-gather-output\") pod \"must-gather-fpbn5\" (UID: \"710ea447-416a-4000-8f95-28e6319a1aea\") " pod="openshift-must-gather-hgbjn/must-gather-fpbn5"
Jul 28 01:57:44.009757 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:44.009548    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blswn\" (UniqueName: \"kubernetes.io/projected/710ea447-416a-4000-8f95-28e6319a1aea-kube-api-access-blswn\") pod \"must-gather-fpbn5\" (UID: \"710ea447-416a-4000-8f95-28e6319a1aea\") " pod="openshift-must-gather-hgbjn/must-gather-fpbn5"
Jul 28 01:57:44.110639 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:44.110598    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/710ea447-416a-4000-8f95-28e6319a1aea-must-gather-output\") pod \"must-gather-fpbn5\" (UID: \"710ea447-416a-4000-8f95-28e6319a1aea\") " pod="openshift-must-gather-hgbjn/must-gather-fpbn5"
Jul 28 01:57:44.110831 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:44.110669    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blswn\" (UniqueName: \"kubernetes.io/projected/710ea447-416a-4000-8f95-28e6319a1aea-kube-api-access-blswn\") pod \"must-gather-fpbn5\" (UID: \"710ea447-416a-4000-8f95-28e6319a1aea\") " pod="openshift-must-gather-hgbjn/must-gather-fpbn5"
Jul 28 01:57:44.110966 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:44.110942    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/710ea447-416a-4000-8f95-28e6319a1aea-must-gather-output\") pod \"must-gather-fpbn5\" (UID: \"710ea447-416a-4000-8f95-28e6319a1aea\") " pod="openshift-must-gather-hgbjn/must-gather-fpbn5"
Jul 28 01:57:44.118941 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:44.118911    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blswn\" (UniqueName: \"kubernetes.io/projected/710ea447-416a-4000-8f95-28e6319a1aea-kube-api-access-blswn\") pod \"must-gather-fpbn5\" (UID: \"710ea447-416a-4000-8f95-28e6319a1aea\") " pod="openshift-must-gather-hgbjn/must-gather-fpbn5"
Jul 28 01:57:44.120806 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:44.120786    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hgbjn/must-gather-fpbn5"
Jul 28 01:57:44.253679 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:44.253638    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hgbjn/must-gather-fpbn5"]
Jul 28 01:57:44.254735 ip-10-0-132-238 kubenswrapper[2578]: W0728 01:57:44.254707    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod710ea447_416a_4000_8f95_28e6319a1aea.slice/crio-872f79ff6269011d2f70b4a054265a656a07e8de8736ed47ff1be2c9b2328bdf WatchSource:0}: Error finding container 872f79ff6269011d2f70b4a054265a656a07e8de8736ed47ff1be2c9b2328bdf: Status 404 returned error can't find the container with id 872f79ff6269011d2f70b4a054265a656a07e8de8736ed47ff1be2c9b2328bdf
Jul 28 01:57:44.256710 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:44.256690    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 01:57:44.341406 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:44.341370    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hgbjn/must-gather-fpbn5" event={"ID":"710ea447-416a-4000-8f95-28e6319a1aea","Type":"ContainerStarted","Data":"872f79ff6269011d2f70b4a054265a656a07e8de8736ed47ff1be2c9b2328bdf"}
Jul 28 01:57:45.350822 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:45.350720    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hgbjn/must-gather-fpbn5" event={"ID":"710ea447-416a-4000-8f95-28e6319a1aea","Type":"ContainerStarted","Data":"785ff59e56211a1c0908ee6a1f4960ad452d362f591ea0248331d560f61b91cc"}
Jul 28 01:57:45.350822 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:45.350772    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hgbjn/must-gather-fpbn5" event={"ID":"710ea447-416a-4000-8f95-28e6319a1aea","Type":"ContainerStarted","Data":"22d38c4ced37fcc1e77a8cb5b47038cd15284d3338646f62a0339b2e96a16b58"}
Jul 28 01:57:45.369118 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:45.369051    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hgbjn/must-gather-fpbn5" podStartSLOduration=1.610501571 podStartE2EDuration="2.369029163s" totalImagesPullingTime="758.527592ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:57:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:57:45.367939814 +0000 UTC m=+3835.532747922" watchObservedRunningTime="2026-07-28 01:57:45.369029163 +0000 UTC m=+3835.533837271"
Jul 28 01:57:46.673735 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:46.673694    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lvd59_27305211-7ed9-4558-bd84-f703dcff265a/global-pull-secret-syncer/0.log"
Jul 28 01:57:46.816579 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:46.816547    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xqfwc_a2f32b69-b720-4c3c-b345-4cbd68398e2d/konnectivity-agent/0.log"
Jul 28 01:57:46.848034 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:46.848004    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-238.ec2.internal_7f112d9abde8daeb5837b42f75048eeb/haproxy/0.log"
Jul 28 01:57:52.451232 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:52.451188    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-bb8n5_96996e3b-d506-43f6-b825-a126c7da9720/manager/0.log"
Jul 28 01:57:52.482935 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:52.482900    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-gmml8_503b066d-ed59-42da-8aa1-e23fedc4157a/kuadrant-console-plugin/0.log"
Jul 28 01:57:52.780103 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:52.780061    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-qhx46_ed3db210-4391-45ce-a306-6f9a4a948496/manager/0.log"
Jul 28 01:57:54.551531 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:54.551461    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_37112156-d7b1-42c5-9f56-08e5ccb8472b/alertmanager/0.log"
Jul 28 01:57:54.581313 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:54.581201    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_37112156-d7b1-42c5-9f56-08e5ccb8472b/config-reloader/0.log"
Jul 28 01:57:54.605515 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:54.605483    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_37112156-d7b1-42c5-9f56-08e5ccb8472b/kube-rbac-proxy-web/0.log"
Jul 28 01:57:54.627448 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:54.627417    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_37112156-d7b1-42c5-9f56-08e5ccb8472b/kube-rbac-proxy/0.log"
Jul 28 01:57:54.652386 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:54.652351    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_37112156-d7b1-42c5-9f56-08e5ccb8472b/kube-rbac-proxy-metric/0.log"
Jul 28 01:57:54.678135 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:54.678054    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_37112156-d7b1-42c5-9f56-08e5ccb8472b/prom-label-proxy/0.log"
Jul 28 01:57:54.700199 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:54.700175    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_37112156-d7b1-42c5-9f56-08e5ccb8472b/init-config-reloader/0.log"
Jul 28 01:57:54.788966 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:54.788928    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-fnjzx_bd8976a7-8e1e-45b9-851c-7312c17fab2d/kube-state-metrics/0.log"
Jul 28 01:57:54.812905 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:54.812881    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-fnjzx_bd8976a7-8e1e-45b9-851c-7312c17fab2d/kube-rbac-proxy-main/0.log"
Jul 28 01:57:54.840059 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:54.840024    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-fnjzx_bd8976a7-8e1e-45b9-851c-7312c17fab2d/kube-rbac-proxy-self/0.log"
Jul 28 01:57:54.871458 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:54.871414    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6f59fc7c69-d2796_b1fe546e-f927-4a9d-ac62-273236c89387/metrics-server/0.log"
Jul 28 01:57:54.901721 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:54.901688    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-slx94_365ce136-2a8b-4d7f-a5b5-8dbb810ed6a2/monitoring-plugin/0.log"
Jul 28 01:57:55.099275 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:55.099244    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qz2nn_24ea961e-dcc1-471c-841d-f851ad0b3969/node-exporter/0.log"
Jul 28 01:57:55.123074 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:55.123042    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qz2nn_24ea961e-dcc1-471c-841d-f851ad0b3969/kube-rbac-proxy/0.log"
Jul 28 01:57:55.147091 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:55.147052    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qz2nn_24ea961e-dcc1-471c-841d-f851ad0b3969/init-textfile/0.log"
Jul 28 01:57:55.178206 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:55.178180    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-s2ng7_8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1/kube-rbac-proxy-main/0.log"
Jul 28 01:57:55.218853 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:55.218823    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-s2ng7_8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1/kube-rbac-proxy-self/0.log"
Jul 28 01:57:55.242380 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:55.242298    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-s2ng7_8fd3d0c3-7107-4334-8e7d-2448d4c8b6e1/openshift-state-metrics/0.log"
Jul 28 01:57:55.279522 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:55.279489    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e345eadb-4917-47d1-81d0-3ab515d785d8/prometheus/0.log"
Jul 28 01:57:55.299681 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:55.299637    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e345eadb-4917-47d1-81d0-3ab515d785d8/config-reloader/0.log"
Jul 28 01:57:55.320857 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:55.320826    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e345eadb-4917-47d1-81d0-3ab515d785d8/thanos-sidecar/0.log"
Jul 28 01:57:55.343323 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:55.343299    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e345eadb-4917-47d1-81d0-3ab515d785d8/kube-rbac-proxy-web/0.log"
Jul 28 01:57:55.369230 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:55.369182    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e345eadb-4917-47d1-81d0-3ab515d785d8/kube-rbac-proxy/0.log"
Jul 28 01:57:55.401774 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:55.401746    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e345eadb-4917-47d1-81d0-3ab515d785d8/kube-rbac-proxy-thanos/0.log"
Jul 28 01:57:55.432371 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:55.432342    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e345eadb-4917-47d1-81d0-3ab515d785d8/init-config-reloader/0.log"
Jul 28 01:57:55.470033 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:55.469589    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-h9sks_f55f9612-aaa8-41e9-a785-4ec4cb622e64/prometheus-operator/0.log"
Jul 28 01:57:55.486963 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:55.486905    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-h9sks_f55f9612-aaa8-41e9-a785-4ec4cb622e64/kube-rbac-proxy/0.log"
Jul 28 01:57:55.513949 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:55.513875    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-9xv8h_c528c7c4-3a25-4163-8375-922fbe1c07e3/prometheus-operator-admission-webhook/0.log"
Jul 28 01:57:55.540751 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:55.540717    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7497f8b8d6-25h9t_627ac9a1-6f65-49d9-9cad-58858ea00f26/telemeter-client/0.log"
Jul 28 01:57:55.561386 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:55.561358    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7497f8b8d6-25h9t_627ac9a1-6f65-49d9-9cad-58858ea00f26/reload/0.log"
Jul 28 01:57:55.584691 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:55.584657    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7497f8b8d6-25h9t_627ac9a1-6f65-49d9-9cad-58858ea00f26/kube-rbac-proxy/0.log"
Jul 28 01:57:55.614495 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:55.614467    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78977f5d4b-pl7n7_8bf4b440-aaa7-4750-ac41-f1be52a77a51/thanos-query/0.log"
Jul 28 01:57:55.646509 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:55.646463    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78977f5d4b-pl7n7_8bf4b440-aaa7-4750-ac41-f1be52a77a51/kube-rbac-proxy-web/0.log"
Jul 28 01:57:55.667091 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:55.667060    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78977f5d4b-pl7n7_8bf4b440-aaa7-4750-ac41-f1be52a77a51/kube-rbac-proxy/0.log"
Jul 28 01:57:55.687436 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:55.687395    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78977f5d4b-pl7n7_8bf4b440-aaa7-4750-ac41-f1be52a77a51/prom-label-proxy/0.log"
Jul 28 01:57:55.709461 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:55.709426    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78977f5d4b-pl7n7_8bf4b440-aaa7-4750-ac41-f1be52a77a51/kube-rbac-proxy-rules/0.log"
Jul 28 01:57:55.734963 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:55.734936    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78977f5d4b-pl7n7_8bf4b440-aaa7-4750-ac41-f1be52a77a51/kube-rbac-proxy-metrics/0.log"
Jul 28 01:57:56.833190 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:56.833146    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hgbjn/perf-node-gather-daemonset-cxstq"]
Jul 28 01:57:56.839178 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:56.839155    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hgbjn/perf-node-gather-daemonset-cxstq"
Jul 28 01:57:56.845660 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:56.845630    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hgbjn/perf-node-gather-daemonset-cxstq"]
Jul 28 01:57:56.963951 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:56.963912    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f96f8882-0e01-454a-b394-1e7c9d28e787-proc\") pod \"perf-node-gather-daemonset-cxstq\" (UID: \"f96f8882-0e01-454a-b394-1e7c9d28e787\") " pod="openshift-must-gather-hgbjn/perf-node-gather-daemonset-cxstq"
Jul 28 01:57:56.964130 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:56.964039    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f96f8882-0e01-454a-b394-1e7c9d28e787-lib-modules\") pod \"perf-node-gather-daemonset-cxstq\" (UID: \"f96f8882-0e01-454a-b394-1e7c9d28e787\") " pod="openshift-must-gather-hgbjn/perf-node-gather-daemonset-cxstq"
Jul 28 01:57:56.964130 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:56.964082    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4m9v\" (UniqueName: \"kubernetes.io/projected/f96f8882-0e01-454a-b394-1e7c9d28e787-kube-api-access-d4m9v\") pod \"perf-node-gather-daemonset-cxstq\" (UID: \"f96f8882-0e01-454a-b394-1e7c9d28e787\") " pod="openshift-must-gather-hgbjn/perf-node-gather-daemonset-cxstq"
Jul 28 01:57:56.964130 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:56.964127    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f96f8882-0e01-454a-b394-1e7c9d28e787-podres\") pod \"perf-node-gather-daemonset-cxstq\" (UID: \"f96f8882-0e01-454a-b394-1e7c9d28e787\") " pod="openshift-must-gather-hgbjn/perf-node-gather-daemonset-cxstq"
Jul 28 01:57:56.964332 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:56.964160    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f96f8882-0e01-454a-b394-1e7c9d28e787-sys\") pod \"perf-node-gather-daemonset-cxstq\" (UID: \"f96f8882-0e01-454a-b394-1e7c9d28e787\") " pod="openshift-must-gather-hgbjn/perf-node-gather-daemonset-cxstq"
Jul 28 01:57:57.065717 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:57.065681    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f96f8882-0e01-454a-b394-1e7c9d28e787-lib-modules\") pod \"perf-node-gather-daemonset-cxstq\" (UID: \"f96f8882-0e01-454a-b394-1e7c9d28e787\") " pod="openshift-must-gather-hgbjn/perf-node-gather-daemonset-cxstq"
Jul 28 01:57:57.065911 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:57.065742    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4m9v\" (UniqueName: \"kubernetes.io/projected/f96f8882-0e01-454a-b394-1e7c9d28e787-kube-api-access-d4m9v\") pod \"perf-node-gather-daemonset-cxstq\" (UID: \"f96f8882-0e01-454a-b394-1e7c9d28e787\") " pod="openshift-must-gather-hgbjn/perf-node-gather-daemonset-cxstq"
Jul 28 01:57:57.065911 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:57.065788    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f96f8882-0e01-454a-b394-1e7c9d28e787-podres\") pod \"perf-node-gather-daemonset-cxstq\" (UID: \"f96f8882-0e01-454a-b394-1e7c9d28e787\") " pod="openshift-must-gather-hgbjn/perf-node-gather-daemonset-cxstq"
Jul 28 01:57:57.065911 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:57.065822    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f96f8882-0e01-454a-b394-1e7c9d28e787-sys\") pod \"perf-node-gather-daemonset-cxstq\" (UID: \"f96f8882-0e01-454a-b394-1e7c9d28e787\") " pod="openshift-must-gather-hgbjn/perf-node-gather-daemonset-cxstq"
Jul 28 01:57:57.065911 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:57.065868    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f96f8882-0e01-454a-b394-1e7c9d28e787-proc\") pod \"perf-node-gather-daemonset-cxstq\" (UID: \"f96f8882-0e01-454a-b394-1e7c9d28e787\") " pod="openshift-must-gather-hgbjn/perf-node-gather-daemonset-cxstq"
Jul 28 01:57:57.066128 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:57.065977    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f96f8882-0e01-454a-b394-1e7c9d28e787-proc\") pod \"perf-node-gather-daemonset-cxstq\" (UID: \"f96f8882-0e01-454a-b394-1e7c9d28e787\") " pod="openshift-must-gather-hgbjn/perf-node-gather-daemonset-cxstq"
Jul 28 01:57:57.066356 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:57.066333    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f96f8882-0e01-454a-b394-1e7c9d28e787-lib-modules\") pod \"perf-node-gather-daemonset-cxstq\" (UID: \"f96f8882-0e01-454a-b394-1e7c9d28e787\") " pod="openshift-must-gather-hgbjn/perf-node-gather-daemonset-cxstq"
Jul 28 01:57:57.066453 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:57.066355    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f96f8882-0e01-454a-b394-1e7c9d28e787-podres\") pod \"perf-node-gather-daemonset-cxstq\" (UID: \"f96f8882-0e01-454a-b394-1e7c9d28e787\") " pod="openshift-must-gather-hgbjn/perf-node-gather-daemonset-cxstq"
Jul 28 01:57:57.066510 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:57.066392    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f96f8882-0e01-454a-b394-1e7c9d28e787-sys\") pod \"perf-node-gather-daemonset-cxstq\" (UID: \"f96f8882-0e01-454a-b394-1e7c9d28e787\") " pod="openshift-must-gather-hgbjn/perf-node-gather-daemonset-cxstq"
Jul 28 01:57:57.079257 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:57.079037    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4m9v\" (UniqueName: \"kubernetes.io/projected/f96f8882-0e01-454a-b394-1e7c9d28e787-kube-api-access-d4m9v\") pod \"perf-node-gather-daemonset-cxstq\" (UID: \"f96f8882-0e01-454a-b394-1e7c9d28e787\") " pod="openshift-must-gather-hgbjn/perf-node-gather-daemonset-cxstq"
Jul 28 01:57:57.153352 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:57.153313    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hgbjn/perf-node-gather-daemonset-cxstq"
Jul 28 01:57:57.371190 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:57.370315    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hgbjn/perf-node-gather-daemonset-cxstq"]
Jul 28 01:57:57.458235 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:57.457806    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hgbjn/perf-node-gather-daemonset-cxstq" event={"ID":"f96f8882-0e01-454a-b394-1e7c9d28e787","Type":"ContainerStarted","Data":"13ee5f6a9a69c4e562c6109b0babb8c2b3d573f701d0efa294348d7aa9939b0e"}
Jul 28 01:57:58.062400 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:58.062367    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5c4f87f695-hxhpn_ce759661-df26-4039-8996-0e4579d975e4/console/0.log"
Jul 28 01:57:58.467168 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:58.467134    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hgbjn/perf-node-gather-daemonset-cxstq" event={"ID":"f96f8882-0e01-454a-b394-1e7c9d28e787","Type":"ContainerStarted","Data":"4e19d6aa785dc8e6480605893087881c82fd65d5d0158b7d3509fbd8eafb47c1"}
Jul 28 01:57:58.467385 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:58.467253    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hgbjn/perf-node-gather-daemonset-cxstq"
Jul 28 01:57:58.491113 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:58.491045    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hgbjn/perf-node-gather-daemonset-cxstq" podStartSLOduration=2.491023515 podStartE2EDuration="2.491023515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:57:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:57:58.488419901 +0000 UTC m=+3848.653228009" watchObservedRunningTime="2026-07-28 01:57:58.491023515 +0000 UTC m=+3848.655831625"
Jul 28 01:57:59.671028 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:59.670998    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dbcvj_19425b8d-e929-41df-824d-49b9ec05ec72/dns/0.log"
Jul 28 01:57:59.694719 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:59.694677    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dbcvj_19425b8d-e929-41df-824d-49b9ec05ec72/kube-rbac-proxy/0.log"
Jul 28 01:57:59.864910 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:57:59.864875    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pwmfh_0d08553a-6760-43c3-bc66-4799fd934954/dns-node-resolver/0.log"
Jul 28 01:58:00.439957 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:58:00.439928    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cgdpl_894bd991-2d81-4275-96c3-a6464fc1d74a/node-ca/0.log"
Jul 28 01:58:01.587030 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:58:01.586985    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-fbf9bd945-z8bb2_da1e8311-1c0d-4ff5-b203-74171e7c80e8/kube-auth-proxy/0.log"
Jul 28 01:58:02.602244 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:58:02.602192    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ptgdm_c32f7a50-f215-4648-a7bc-8cf984ef6fb0/serve-healthcheck-canary/0.log"
Jul 28 01:58:03.104724 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:58:03.104700    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-46nnl_b87dd1d9-c9dd-45c5-a9f5-f8eb70385708/kube-rbac-proxy/0.log"
Jul 28 01:58:03.128396 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:58:03.128366    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-46nnl_b87dd1d9-c9dd-45c5-a9f5-f8eb70385708/exporter/0.log"
Jul 28 01:58:03.151695 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:58:03.151659    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-46nnl_b87dd1d9-c9dd-45c5-a9f5-f8eb70385708/extractor/0.log"
Jul 28 01:58:04.486057 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:58:04.486019    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hgbjn/perf-node-gather-daemonset-cxstq"
Jul 28 01:58:06.214057 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:58:06.214022    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6549c9954f-b8gxc_ccd038af-eb0f-4256-9b99-f6aed1604b13/manager/0.log"
Jul 28 01:58:06.235666 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:58:06.235634    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6549c9954f-b8gxc_ccd038af-eb0f-4256-9b99-f6aed1604b13/copy-manifests/0.log"
Jul 28 01:58:07.605675 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:58:07.605604    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6478f6f65f-s844n_8b90f209-f469-48bc-8884-de4eb961649f/manager/0.log"
Jul 28 01:58:07.634500 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:58:07.634473    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-n59n4_c836bcbc-cfd0-4190-8b9e-eb4bf3967557/openshift-lws-operator/0.log"
Jul 28 01:58:08.016687 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:58:08.016589    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-fbf9bd945-z8bb2_da1e8311-1c0d-4ff5-b203-74171e7c80e8/kube-auth-proxy/0.log"
Jul 28 01:58:09.555410 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:58:09.555376    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-fbf9bd945-z8bb2_da1e8311-1c0d-4ff5-b203-74171e7c80e8/kube-auth-proxy/0.log"
Jul 28 01:58:14.060301 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:58:14.060188    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-28rt8_b333573a-411c-4999-964f-daa8b82b48e4/kube-multus/0.log"
Jul 28 01:58:14.086907 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:58:14.086879    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7cgzs_074938e2-4436-4f39-bf00-6333580a6916/kube-multus-additional-cni-plugins/0.log"
Jul 28 01:58:14.110120 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:58:14.110091    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7cgzs_074938e2-4436-4f39-bf00-6333580a6916/egress-router-binary-copy/0.log"
Jul 28 01:58:14.131128 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:58:14.131089    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7cgzs_074938e2-4436-4f39-bf00-6333580a6916/cni-plugins/0.log"
Jul 28 01:58:14.152392 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:58:14.152367    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7cgzs_074938e2-4436-4f39-bf00-6333580a6916/bond-cni-plugin/0.log"
Jul 28 01:58:14.174722 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:58:14.174690    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7cgzs_074938e2-4436-4f39-bf00-6333580a6916/routeoverride-cni/0.log"
Jul 28 01:58:14.202518 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:58:14.202487    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7cgzs_074938e2-4436-4f39-bf00-6333580a6916/whereabouts-cni-bincopy/0.log"
Jul 28 01:58:14.225902 ip-10-0-132-238 kubenswrapper[2578]: I0728 01:58:14.225876    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7cgzs_074938e2-4436-4f39-bf00-6333580a6916/whereabouts-cni/0.log"