Sep 08 15:17:27.180379 ip-10-0-137-43 systemd[1]: Starting Kubernetes Kubelet...
Sep 08 15:17:27.550784 ip-10-0-137-43 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 15:17:27.550784 ip-10-0-137-43 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 08 15:17:27.550784 ip-10-0-137-43 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 15:17:27.550784 ip-10-0-137-43 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 08 15:17:27.550784 ip-10-0-137-43 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 15:17:27.551859 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.551772    2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 08 15:17:27.561714 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561688    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 15:17:27.561714 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561706    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 15:17:27.561714 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561710    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 15:17:27.561714 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561713    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 15:17:27.561714 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561716    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 15:17:27.561714 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561719    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 15:17:27.561714 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561722    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 15:17:27.561953 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561725    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 15:17:27.561953 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561728    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 15:17:27.561953 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561730    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 15:17:27.561953 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561734    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 15:17:27.561953 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561736    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 15:17:27.561953 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561739    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 15:17:27.561953 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561742    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 15:17:27.561953 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561745    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 15:17:27.561953 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561747    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 15:17:27.561953 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561750    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 15:17:27.561953 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561752    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 15:17:27.561953 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561755    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 15:17:27.561953 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561757    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 15:17:27.561953 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561760    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 15:17:27.561953 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561762    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 15:17:27.561953 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561768    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 15:17:27.561953 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561770    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 15:17:27.561953 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561773    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 15:17:27.561953 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561775    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 15:17:27.561953 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561778    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 15:17:27.562450 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561780    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 15:17:27.562450 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561783    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 15:17:27.562450 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561786    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 15:17:27.562450 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561788    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 15:17:27.562450 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561791    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 15:17:27.562450 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561793    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 15:17:27.562450 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561796    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 15:17:27.562450 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561799    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 15:17:27.562450 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561801    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 15:17:27.562450 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561804    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 15:17:27.562450 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561806    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 15:17:27.562450 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561809    2560 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 15:17:27.562450 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561811    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 15:17:27.562450 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561814    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 15:17:27.562450 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561817    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 15:17:27.562450 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561820    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 15:17:27.562450 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561822    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 15:17:27.562450 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561825    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 15:17:27.562450 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561828    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 15:17:27.562450 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561830    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 15:17:27.562945 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561833    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 15:17:27.562945 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561835    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 15:17:27.562945 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561840    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 15:17:27.562945 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561844    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 15:17:27.562945 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561847    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 15:17:27.562945 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561851    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 15:17:27.562945 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561854    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 15:17:27.562945 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561856    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 15:17:27.562945 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561859    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 15:17:27.562945 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561861    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 15:17:27.562945 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561864    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 15:17:27.562945 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561866    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 15:17:27.562945 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561869    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 15:17:27.562945 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561872    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 15:17:27.562945 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561874    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 15:17:27.562945 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561877    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 15:17:27.562945 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561879    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 15:17:27.562945 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561881    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 15:17:27.562945 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561884    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 15:17:27.563425 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561886    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 15:17:27.563425 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561889    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 15:17:27.563425 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561891    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 15:17:27.563425 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561894    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 15:17:27.563425 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561896    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 15:17:27.563425 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561899    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 15:17:27.563425 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561903    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 15:17:27.563425 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561908    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 15:17:27.563425 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561911    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 15:17:27.563425 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561915    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 15:17:27.563425 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561917    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 15:17:27.563425 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561920    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 15:17:27.563425 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561922    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 15:17:27.563425 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561925    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 15:17:27.563425 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561928    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 15:17:27.563425 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561930    2560 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 15:17:27.563425 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561933    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 15:17:27.563425 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561937    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 15:17:27.563425 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561940    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 15:17:27.563891 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561942    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 15:17:27.563891 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561945    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 15:17:27.563891 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561950    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 15:17:27.563891 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561953    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 15:17:27.563891 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561955    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 15:17:27.563891 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561958    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 15:17:27.563891 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561961    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 15:17:27.563891 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561964    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 15:17:27.563891 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561966    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 15:17:27.563891 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561969    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 15:17:27.563891 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561972    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 15:17:27.563891 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561976    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 15:17:27.563891 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561979    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 15:17:27.563891 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.561983    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 15:17:27.563891 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562665    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 15:17:27.563891 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562672    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 15:17:27.563891 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562675    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 15:17:27.563891 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562678    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 15:17:27.563891 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562681    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 15:17:27.563891 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562684    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 15:17:27.564387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562687    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 15:17:27.564387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562690    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 15:17:27.564387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562692    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 15:17:27.564387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562695    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 15:17:27.564387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562698    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 15:17:27.564387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562701    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 15:17:27.564387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562703    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 15:17:27.564387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562706    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 15:17:27.564387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562710    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 15:17:27.564387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562714    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 15:17:27.564387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562717    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 15:17:27.564387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562720    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 15:17:27.564387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562723    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 15:17:27.564387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562726    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 15:17:27.564387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562729    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 15:17:27.564387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562732    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 15:17:27.564387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562734    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 15:17:27.564387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562737    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 15:17:27.564387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562739    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 15:17:27.564874 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562742    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 15:17:27.564874 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562744    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 15:17:27.564874 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562747    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 15:17:27.564874 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562750    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 15:17:27.564874 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562753    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 15:17:27.564874 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562756    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 15:17:27.564874 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562758    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 15:17:27.564874 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562761    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 15:17:27.564874 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562763    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 15:17:27.564874 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562766    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 15:17:27.564874 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562768    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 15:17:27.564874 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562771    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 15:17:27.564874 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562773    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 15:17:27.564874 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562776    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 15:17:27.564874 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562779    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 15:17:27.564874 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562782    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 15:17:27.564874 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562785    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 15:17:27.564874 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562787    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 15:17:27.564874 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562790    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 15:17:27.564874 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562792    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 15:17:27.565388 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562795    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 15:17:27.565388 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562798    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 15:17:27.565388 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562800    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 15:17:27.565388 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562803    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 15:17:27.565388 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562805    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 15:17:27.565388 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562809    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 15:17:27.565388 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562811    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 15:17:27.565388 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562814    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 15:17:27.565388 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562817    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 15:17:27.565388 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562819    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 15:17:27.565388 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562822    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 15:17:27.565388 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562825    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 15:17:27.565388 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562828    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 15:17:27.565388 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562830    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 15:17:27.565388 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562833    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 15:17:27.565388 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562835    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 15:17:27.565388 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562838    2560 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 15:17:27.565388 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562841    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 15:17:27.565388 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562844    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 15:17:27.565388 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562846    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 15:17:27.565886 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562850    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 15:17:27.565886 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562852    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 15:17:27.565886 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562855    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 15:17:27.565886 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562858    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 15:17:27.565886 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562861    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 15:17:27.565886 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562864    2560 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 15:17:27.565886 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562866    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 15:17:27.565886 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562870    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 15:17:27.565886 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562873    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 15:17:27.565886 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562875    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 15:17:27.565886 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562878    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 15:17:27.565886 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562881    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 15:17:27.565886 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562883    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 15:17:27.565886 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562887    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 15:17:27.565886 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562890    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 15:17:27.565886 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562893    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 15:17:27.565886 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562896    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 15:17:27.565886 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562899    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 15:17:27.565886 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562902    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 15:17:27.565886 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562905    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 15:17:27.566387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562907    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 15:17:27.566387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562910    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 15:17:27.566387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562913    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 15:17:27.566387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562917    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 15:17:27.566387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562920    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 15:17:27.566387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562924    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 15:17:27.566387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562926    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 15:17:27.566387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562929    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 15:17:27.566387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562932    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 15:17:27.566387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562935    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 15:17:27.566387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562938    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 15:17:27.566387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562941    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 15:17:27.566387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562944    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 15:17:27.566387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.562946    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 15:17:27.566387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563335    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 15:17:27.566387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563340    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 15:17:27.566387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563343    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 15:17:27.566387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563346    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 15:17:27.566387 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563349    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 15:17:27.566879 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563351    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 15:17:27.566879 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563354    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 15:17:27.566879 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563357    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 15:17:27.566879 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563361    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 15:17:27.566879 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563364    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 15:17:27.566879 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563366    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 15:17:27.566879 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563369    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 15:17:27.566879 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563371    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 15:17:27.566879 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563374    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 15:17:27.566879 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563376    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 15:17:27.566879 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563379    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 15:17:27.566879 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563382    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 15:17:27.566879 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563385    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 15:17:27.566879 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563387    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 15:17:27.566879 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563390    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 15:17:27.566879 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563392    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 15:17:27.566879 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563395    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 15:17:27.566879 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563397    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 15:17:27.566879 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563400    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 15:17:27.567383 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563402    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 15:17:27.567383 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563407    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 15:17:27.567383 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563409    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 15:17:27.567383 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563412    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 15:17:27.567383 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563414    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 15:17:27.567383 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563417    2560 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 15:17:27.567383 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563419    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 15:17:27.567383 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563422    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 15:17:27.567383 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563425    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 15:17:27.567383 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563427    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 15:17:27.567383 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563430    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 15:17:27.567383 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563433    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 15:17:27.567383 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563435    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 15:17:27.567383 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563438    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 15:17:27.567383 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563441    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 15:17:27.567383 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563443    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 15:17:27.567383 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563446    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 15:17:27.567383 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563449    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 15:17:27.567383 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563451    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 15:17:27.567383 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563454    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 15:17:27.567873 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563458    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 15:17:27.567873 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563461    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 15:17:27.567873 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563464    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 15:17:27.567873 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563467    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 15:17:27.567873 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563469    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 15:17:27.567873 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563472    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 15:17:27.567873 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563475    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 15:17:27.567873 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563478    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 15:17:27.567873 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563481    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 15:17:27.567873 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563483    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 15:17:27.567873 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563486    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 15:17:27.567873 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563489    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 15:17:27.567873 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563491    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 15:17:27.567873 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563495    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 15:17:27.567873 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563497    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 15:17:27.567873 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563500    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 15:17:27.567873 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563504    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 15:17:27.567873 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563507    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 15:17:27.567873 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563511    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 15:17:27.568360 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563513    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 15:17:27.568360 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563516    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 15:17:27.568360 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563519    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 15:17:27.568360 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563522    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 15:17:27.568360 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563524    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 15:17:27.568360 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563527    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 15:17:27.568360 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563529    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 15:17:27.568360 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563532    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 15:17:27.568360 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563534    2560 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 15:17:27.568360 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563537    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 15:17:27.568360 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563539    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 15:17:27.568360 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563542    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 15:17:27.568360 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563544    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 15:17:27.568360 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563547    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 15:17:27.568360 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563549    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 15:17:27.568360 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563552    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 15:17:27.568360 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563554    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 15:17:27.568360 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563556    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 15:17:27.568360 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563559    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 15:17:27.568360 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563562    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 15:17:27.568982 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563565    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 15:17:27.568982 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563567    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 15:17:27.568982 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563570    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 15:17:27.568982 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563572    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 15:17:27.568982 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563575    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 15:17:27.568982 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563577    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 15:17:27.568982 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563581    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 15:17:27.568982 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563583    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 15:17:27.568982 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563586    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 15:17:27.568982 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563588    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 15:17:27.568982 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563591    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 15:17:27.568982 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563594    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 15:17:27.568982 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563596    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 15:17:27.568982 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563603    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 15:17:27.568982 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563605    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 15:17:27.568982 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.563608    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 15:17:27.568982 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563679    2560 flags.go:64] FLAG: --address="0.0.0.0"
Sep 08 15:17:27.568982 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563712    2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 08 15:17:27.568982 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563718    2560 flags.go:64] FLAG: --anonymous-auth="true"
Sep 08 15:17:27.568982 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563722    2560 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 08 15:17:27.568982 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563726    2560 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 08 15:17:27.569527 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563729    2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 08 15:17:27.569527 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563733    2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 08 15:17:27.569527 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563742    2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 08 15:17:27.569527 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563745    2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 08 15:17:27.569527 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563748    2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 08 15:17:27.569527 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563752    2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 08 15:17:27.569527 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563755    2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 08 15:17:27.569527 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563758    2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 08 15:17:27.569527 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563761    2560 flags.go:64] FLAG: --cgroup-root=""
Sep 08 15:17:27.569527 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563764    2560 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 08 15:17:27.569527 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563767    2560 flags.go:64] FLAG: --client-ca-file=""
Sep 08 15:17:27.569527 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563770    2560 flags.go:64] FLAG: --cloud-provider="external"
Sep 08 15:17:27.569527 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563773    2560 flags.go:64] FLAG: --cluster-dns="[]"
Sep 08 15:17:27.569527 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563778    2560 flags.go:64] FLAG: --cluster-domain=""
Sep 08 15:17:27.569527 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563781    2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 08 15:17:27.569527 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563784    2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 08 15:17:27.569527 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563787    2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 08 15:17:27.569527 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563792    2560 flags.go:64] FLAG: --container-log-max-files="5"
Sep 08 15:17:27.569527 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563796    2560 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 08 15:17:27.569527 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563798    2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 08 15:17:27.569527 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563802    2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 08 15:17:27.569527 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563807    2560 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 08 15:17:27.569527 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563810    2560 flags.go:64] FLAG: --contention-profiling="false"
Sep 08 15:17:27.569527 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563813    2560 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 08 15:17:27.570161 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563816    2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 08 15:17:27.570161 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563819    2560 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 08 15:17:27.570161 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563822    2560 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 08 15:17:27.570161 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563843    2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 08 15:17:27.570161 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563847    2560 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 08 15:17:27.570161 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563850    2560 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 08 15:17:27.570161 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563854    2560 flags.go:64] FLAG: --enable-load-reader="false"
Sep 08 15:17:27.570161 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563857    2560 flags.go:64] FLAG: --enable-server="true"
Sep 08 15:17:27.570161 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563860    2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 08 15:17:27.570161 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563864    2560 flags.go:64] FLAG: --event-burst="100"
Sep 08 15:17:27.570161 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563867    2560 flags.go:64] FLAG: --event-qps="50"
Sep 08 15:17:27.570161 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563870    2560 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 08 15:17:27.570161 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563873    2560 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 08 15:17:27.570161 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563876    2560 flags.go:64] FLAG: --eviction-hard=""
Sep 08 15:17:27.570161 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563879    2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 08 15:17:27.570161 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563882    2560 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 08 15:17:27.570161 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563885    2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 08 15:17:27.570161 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563888    2560 flags.go:64] FLAG: --eviction-soft=""
Sep 08 15:17:27.570161 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563891    2560 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 08 15:17:27.570161 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563893    2560 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 08 15:17:27.570161 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563897    2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 08 15:17:27.570161 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563900    2560 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 08 15:17:27.570161 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563903    2560 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 08 15:17:27.570161 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563906    2560 flags.go:64] FLAG: --fail-swap-on="true"
Sep 08 15:17:27.570161 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563909    2560 flags.go:64] FLAG: --feature-gates=""
Sep 08 15:17:27.570787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563913    2560 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 08 15:17:27.570787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563916    2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 08 15:17:27.570787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563919    2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 08 15:17:27.570787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563922    2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 08 15:17:27.570787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563926    2560 flags.go:64] FLAG: --healthz-port="10248"
Sep 08 15:17:27.570787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563929    2560 flags.go:64] FLAG: --help="false"
Sep 08 15:17:27.570787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563932    2560 flags.go:64] FLAG: --hostname-override="ip-10-0-137-43.ec2.internal"
Sep 08 15:17:27.570787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563935    2560 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 08 15:17:27.570787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563938    2560 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 08 15:17:27.570787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563941    2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 08 15:17:27.570787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563944    2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 08 15:17:27.570787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563948    2560 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 08 15:17:27.570787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563951    2560 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 08 15:17:27.570787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563954    2560 flags.go:64] FLAG: --image-service-endpoint=""
Sep 08 15:17:27.570787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563957    2560 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 08 15:17:27.570787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563960    2560 flags.go:64] FLAG: --kube-api-burst="100"
Sep 08 15:17:27.570787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563962    2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 08 15:17:27.570787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563966    2560 flags.go:64] FLAG: --kube-api-qps="50"
Sep 08 15:17:27.570787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563969    2560 flags.go:64] FLAG: --kube-reserved=""
Sep 08 15:17:27.570787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563972    2560 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 08 15:17:27.570787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563974    2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 08 15:17:27.570787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563977    2560 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 08 15:17:27.570787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563980    2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 08 15:17:27.570787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563983    2560 flags.go:64] FLAG: --lock-file=""
Sep 08 15:17:27.571397 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563986    2560 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 08 15:17:27.571397 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563989    2560 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 08 15:17:27.571397 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563991    2560 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 08 15:17:27.571397 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.563997    2560 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 08 15:17:27.571397 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564000    2560 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 08 15:17:27.571397 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564003    2560 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 08 15:17:27.571397 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564006    2560 flags.go:64] FLAG: --logging-format="text"
Sep 08 15:17:27.571397 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564009    2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 08 15:17:27.571397 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564013    2560 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 08 15:17:27.571397 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564016    2560 flags.go:64] FLAG: --manifest-url=""
Sep 08 15:17:27.571397 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564018    2560 flags.go:64] FLAG: --manifest-url-header=""
Sep 08 15:17:27.571397 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564022    2560 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 08 15:17:27.571397 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564027    2560 flags.go:64] FLAG: --max-open-files="1000000"
Sep 08 15:17:27.571397 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564030    2560 flags.go:64] FLAG: --max-pods="110"
Sep 08 15:17:27.571397 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564033    2560 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 08 15:17:27.571397 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564036    2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 08 15:17:27.571397 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564039    2560 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 08 15:17:27.571397 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564042    2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 08 15:17:27.571397 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564045    2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 08 15:17:27.571397 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564048    2560 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 08 15:17:27.571397 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564051    2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 08 15:17:27.571397 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564058    2560 flags.go:64] FLAG: --node-status-max-images="50"
Sep 08 15:17:27.571397 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564061    2560 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 08 15:17:27.571397 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564064    2560 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 08 15:17:27.571397 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564079    2560 flags.go:64] FLAG: --pod-cidr=""
Sep 08 15:17:27.572014 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564083    2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 08 15:17:27.572014 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564088    2560 flags.go:64] FLAG: --pod-manifest-path=""
Sep 08 15:17:27.572014 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564090    2560 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 08 15:17:27.572014 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564093    2560 flags.go:64] FLAG: --pods-per-core="0"
Sep 08 15:17:27.572014 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564096    2560 flags.go:64] FLAG: --port="10250"
Sep 08 15:17:27.572014 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564099    2560 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 08 15:17:27.572014 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564102    2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-027efdda3e771a8f4"
Sep 08 15:17:27.572014 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564105    2560 flags.go:64] FLAG: --qos-reserved=""
Sep 08 15:17:27.572014 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564108    2560 flags.go:64] FLAG: --read-only-port="10255"
Sep 08 15:17:27.572014 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564111    2560 flags.go:64] FLAG: --register-node="true"
Sep 08 15:17:27.572014 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564114    2560 flags.go:64] FLAG: --register-with-taints=""
Sep 08 15:17:27.572014 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564117    2560 flags.go:64] FLAG: --registry-burst="10"
Sep 08 15:17:27.572014 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564121    2560 flags.go:64] FLAG: --registry-qps="5"
Sep 08 15:17:27.572014 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564124    2560 flags.go:64] FLAG: --reserved-cpus=""
Sep 08 15:17:27.572014 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564127    2560 flags.go:64] FLAG: --reserved-memory=""
Sep 08 15:17:27.572014 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564131    2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 08 15:17:27.572014 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564134    2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 08 15:17:27.572014 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564137    2560 flags.go:64] FLAG: --rotate-certificates="false"
Sep 08 15:17:27.572014 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564140    2560 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 08 15:17:27.572014 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564143    2560 flags.go:64] FLAG: --runonce="false"
Sep 08 15:17:27.572014 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564147    2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 08 15:17:27.572014 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564151    2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 08 15:17:27.572014 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564153    2560 flags.go:64] FLAG: --seccomp-default="false"
Sep 08 15:17:27.572014 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564156    2560 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 08 15:17:27.572014 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564159    2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 08 15:17:27.572663 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564162    2560 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 08 15:17:27.572663 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564165    2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 08 15:17:27.572663 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564168    2560 flags.go:64] FLAG: --storage-driver-password="root"
Sep 08 15:17:27.572663 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564171    2560 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 08 15:17:27.572663 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564174    2560 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 08 15:17:27.572663 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564177    2560 flags.go:64] FLAG: --storage-driver-user="root"
Sep 08 15:17:27.572663 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564180    2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 08 15:17:27.572663 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564183    2560 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 08 15:17:27.572663 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564185    2560 flags.go:64] FLAG: --system-cgroups=""
Sep 08 15:17:27.572663 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564188    2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 08 15:17:27.572663 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564194    2560 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 08 15:17:27.572663 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564197    2560 flags.go:64] FLAG: --tls-cert-file=""
Sep 08 15:17:27.572663 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564199    2560 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 08 15:17:27.572663 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564204    2560 flags.go:64] FLAG: --tls-min-version=""
Sep 08 15:17:27.572663 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564206    2560 flags.go:64] FLAG: --tls-private-key-file=""
Sep 08 15:17:27.572663 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564209    2560 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 08 15:17:27.572663 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564212    2560 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 08 15:17:27.572663 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564215    2560 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 08 15:17:27.572663 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564218    2560 flags.go:64] FLAG: --v="2"
Sep 08 15:17:27.572663 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564222    2560 flags.go:64] FLAG: --version="false"
Sep 08 15:17:27.572663 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564226    2560 flags.go:64] FLAG: --vmodule=""
Sep 08 15:17:27.572663 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564230    2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 08 15:17:27.572663 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.564233    2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 08 15:17:27.572663 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564341    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 15:17:27.572663 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564345    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 15:17:27.573288 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564348    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 15:17:27.573288 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564351    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 15:17:27.573288 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564356    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 15:17:27.573288 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564358    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 15:17:27.573288 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564361    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 15:17:27.573288 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564364    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 15:17:27.573288 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564367    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 15:17:27.573288 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564370    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 15:17:27.573288 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564373    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 15:17:27.573288 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564375    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 15:17:27.573288 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564378    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 15:17:27.573288 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564381    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 15:17:27.573288 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564384    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 15:17:27.573288 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564387    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 15:17:27.573288 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564390    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 15:17:27.573288 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564393    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 15:17:27.573288 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564396    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 15:17:27.573288 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564399    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 15:17:27.573288 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564402    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 15:17:27.573288 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564404    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 15:17:27.573827 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564407    2560 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 15:17:27.573827 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564410    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 15:17:27.573827 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564412    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 15:17:27.573827 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564415    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 15:17:27.573827 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564418    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 15:17:27.573827 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564420    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 15:17:27.573827 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564423    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 15:17:27.573827 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564425    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 15:17:27.573827 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564428    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 15:17:27.573827 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564433    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 15:17:27.573827 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564435    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 15:17:27.573827 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564438    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 15:17:27.573827 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564441    2560 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 15:17:27.573827 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564443    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 15:17:27.573827 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564449    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 15:17:27.573827 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564452    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 15:17:27.573827 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564454    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 15:17:27.573827 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564457    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 15:17:27.573827 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564460    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 15:17:27.573827 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564462    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 15:17:27.574329 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564465    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 15:17:27.574329 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564467    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 15:17:27.574329 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564471    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 15:17:27.574329 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564475    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 15:17:27.574329 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564479    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 15:17:27.574329 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564482    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 15:17:27.574329 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564485    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 15:17:27.574329 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564488    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 15:17:27.574329 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564491    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 15:17:27.574329 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564493    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 15:17:27.574329 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564503    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 15:17:27.574329 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564506    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 15:17:27.574329 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564508    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 15:17:27.574329 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564511    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 15:17:27.574329 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564513    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 15:17:27.574329 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564516    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 15:17:27.574329 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564518    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 15:17:27.574329 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564521    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 15:17:27.574329 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564524    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 15:17:27.574806 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564526    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 15:17:27.574806 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564529    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 15:17:27.574806 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564533    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 15:17:27.574806 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564536    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 15:17:27.574806 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564539    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 15:17:27.574806 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564542    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 15:17:27.574806 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564544    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 15:17:27.574806 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564548    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 15:17:27.574806 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564551    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 15:17:27.574806 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564553    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 15:17:27.574806 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564556    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 15:17:27.574806 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564558    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 15:17:27.574806 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564561    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 15:17:27.574806 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564563    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 15:17:27.574806 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564566    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 15:17:27.574806 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564569    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 15:17:27.574806 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564572    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 15:17:27.574806 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564574    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 15:17:27.574806 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564577    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 15:17:27.575295 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564579    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 15:17:27.575295 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564590    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 15:17:27.575295 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564593    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 15:17:27.575295 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564595    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 15:17:27.575295 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564597    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 15:17:27.575295 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564600    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 15:17:27.575295 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564603    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 15:17:27.575295 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564606    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 15:17:27.575295 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564608    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 15:17:27.575295 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564611    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 15:17:27.575295 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564614    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 15:17:27.575295 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564616    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 15:17:27.575295 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564619    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 15:17:27.575295 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564623    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 15:17:27.575295 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564626    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 15:17:27.575295 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564631    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 15:17:27.575295 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564634    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 15:17:27.575295 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564637    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 15:17:27.575295 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.564641    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 15:17:27.575767 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.565536    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 15:17:27.575767 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.573439    2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 08 15:17:27.575767 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.573456    2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 08 15:17:27.575767 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.574313    2560 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 08 15:17:27.575767 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.574330    2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 08 15:17:27.575767 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574390    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 15:17:27.575767 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574395    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 15:17:27.575767 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574399    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 15:17:27.575767 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574403    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 15:17:27.575767 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574406    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 15:17:27.575767 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574409    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 15:17:27.575767 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574414    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 15:17:27.575767 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574418    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 15:17:27.575767 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574421    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 15:17:27.576137 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574424    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 15:17:27.576137 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574426    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 15:17:27.576137 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574429    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 15:17:27.576137 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574432    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 15:17:27.576137 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574435    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 15:17:27.576137 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574438    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 15:17:27.576137 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574441    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 15:17:27.576137 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574444    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 15:17:27.576137 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574447    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 15:17:27.576137 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574449    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 15:17:27.576137 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574452    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 15:17:27.576137 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574455    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 15:17:27.576137 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574457    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 15:17:27.576137 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574460    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 15:17:27.576137 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574463    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 15:17:27.576137 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574466    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 15:17:27.576137 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574468    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 15:17:27.576137 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574471    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 15:17:27.576137 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574473    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 15:17:27.576137 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574476    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 15:17:27.576643 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574481    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 15:17:27.576643 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574484    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 15:17:27.576643 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574486    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 15:17:27.576643 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574489    2560 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 15:17:27.576643 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574491    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 15:17:27.576643 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574494    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 15:17:27.576643 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574498    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 15:17:27.576643 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574501    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 15:17:27.576643 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574504    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 15:17:27.576643 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574507    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 15:17:27.576643 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574510    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 15:17:27.576643 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574513    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 15:17:27.576643 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574515    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 15:17:27.576643 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574518    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 15:17:27.576643 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574521    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 15:17:27.576643 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574523    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 15:17:27.576643 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574526    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 15:17:27.576643 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574528    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 15:17:27.576643 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574531    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 15:17:27.577146 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574533    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 15:17:27.577146 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574536    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 15:17:27.577146 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574538    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 15:17:27.577146 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574541    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 15:17:27.577146 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574543    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 15:17:27.577146 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574546    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 15:17:27.577146 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574548    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 15:17:27.577146 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574551    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 15:17:27.577146 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574553    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 15:17:27.577146 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574556    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 15:17:27.577146 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574559    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 15:17:27.577146 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574561    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 15:17:27.577146 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574564    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 15:17:27.577146 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574567    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 15:17:27.577146 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574570    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 15:17:27.577146 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574572    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 15:17:27.577146 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574575    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 15:17:27.577146 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574577    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 15:17:27.577146 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574580    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 15:17:27.577146 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574582    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 15:17:27.577630 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574585    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 15:17:27.577630 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574587    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 15:17:27.577630 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574590    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 15:17:27.577630 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574594    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 15:17:27.577630 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574597    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 15:17:27.577630 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574600    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 15:17:27.577630 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574603    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 15:17:27.577630 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574605    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 15:17:27.577630 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574608    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 15:17:27.577630 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574610    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 15:17:27.577630 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574613    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 15:17:27.577630 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574615    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 15:17:27.577630 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574618    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 15:17:27.577630 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574620    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 15:17:27.577630 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574623    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 15:17:27.577630 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574626    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 15:17:27.577630 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574628    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 15:17:27.577630 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574631    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 15:17:27.577630 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574633    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 15:17:27.577630 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574636    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 15:17:27.578129 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574638    2560 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 15:17:27.578129 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574641    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 15:17:27.578129 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574644    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 15:17:27.578129 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574646    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 15:17:27.578129 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574649    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 15:17:27.578129 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574652    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 15:17:27.578129 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574654    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 15:17:27.578129 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574657    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 15:17:27.578129 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574660    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 15:17:27.578129 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574662    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 15:17:27.578129 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574665    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 15:17:27.578129 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.574670    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 15:17:27.578129 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574782    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 15:17:27.578129 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574786    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 15:17:27.578129 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574790    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 15:17:27.578509 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574793    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 15:17:27.578509 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574795    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 15:17:27.578509 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574798    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 15:17:27.578509 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574800    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 15:17:27.578509 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574803    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 15:17:27.578509 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574806    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 15:17:27.578509 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574808    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 15:17:27.578509 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574811    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 15:17:27.578509 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574814    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 15:17:27.578509 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574817    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 15:17:27.578509 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574821    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 15:17:27.578509 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574825    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 15:17:27.578509 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574827    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 15:17:27.578509 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574830    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 15:17:27.578509 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574833    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 15:17:27.578509 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574836    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 15:17:27.578509 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574839    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 15:17:27.578509 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574842    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 15:17:27.578509 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574844    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 15:17:27.578509 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574847    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 15:17:27.579000 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574850    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 15:17:27.579000 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574853    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 15:17:27.579000 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574856    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 15:17:27.579000 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574858    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 15:17:27.579000 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574861    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 15:17:27.579000 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574864    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 15:17:27.579000 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574866    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 15:17:27.579000 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574869    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 15:17:27.579000 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574872    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 15:17:27.579000 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574874    2560 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 15:17:27.579000 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574877    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 15:17:27.579000 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574879    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 15:17:27.579000 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574882    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 15:17:27.579000 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574885    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 15:17:27.579000 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574887    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 15:17:27.579000 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574890    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 15:17:27.579000 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574892    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 15:17:27.579000 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574895    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 15:17:27.579000 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574897    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 15:17:27.579000 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574900    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 15:17:27.579497 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574902    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 15:17:27.579497 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574905    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 15:17:27.579497 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574909    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 15:17:27.579497 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574911    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 15:17:27.579497 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574914    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 15:17:27.579497 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574917    2560 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 15:17:27.579497 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574919    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 15:17:27.579497 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574922    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 15:17:27.579497 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574924    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 15:17:27.579497 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574944    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 15:17:27.579497 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574948    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 15:17:27.579497 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574951    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 15:17:27.579497 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574954    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 15:17:27.579497 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574957    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 15:17:27.579497 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574959    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 15:17:27.579497 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574962    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 15:17:27.579497 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574964    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 15:17:27.579497 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574967    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 15:17:27.579497 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574970    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 15:17:27.579961 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574972    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 15:17:27.579961 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574975    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 15:17:27.579961 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574977    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 15:17:27.579961 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574980    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 15:17:27.579961 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574983    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 15:17:27.579961 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574985    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 15:17:27.579961 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574988    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 15:17:27.579961 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574990    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 15:17:27.579961 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574992    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 15:17:27.579961 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574995    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 15:17:27.579961 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.574997    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 15:17:27.579961 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.575000    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 15:17:27.579961 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.575003    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 15:17:27.579961 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.575005    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 15:17:27.579961 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.575008    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 15:17:27.579961 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.575010    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 15:17:27.579961 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.575012    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 15:17:27.579961 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.575015    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 15:17:27.579961 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.575020    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 15:17:27.579961 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.575023    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 15:17:27.580461 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.575025    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 15:17:27.580461 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.575028    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 15:17:27.580461 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.575030    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 15:17:27.580461 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.575033    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 15:17:27.580461 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.575036    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 15:17:27.580461 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.575038    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 15:17:27.580461 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.575041    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 15:17:27.580461 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.575043    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 15:17:27.580461 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.575046    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 15:17:27.580461 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.575048    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 15:17:27.580461 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.575051    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 15:17:27.580461 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.575053    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 15:17:27.580461 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.575056    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 15:17:27.580461 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.575059    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 15:17:27.580461 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.575061    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 15:17:27.580461 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.575065    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 15:17:27.580461 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:27.575084    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 15:17:27.580898 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.575089    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 15:17:27.580898 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.575194    2560 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 08 15:17:27.580898 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.578522    2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 08 15:17:27.580898 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.579525    2560 server.go:1019] "Starting client certificate rotation"
Sep 08 15:17:27.580898 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.579618    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 08 15:17:27.580898 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.579658    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 08 15:17:27.583558 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.583544    2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 08 15:17:27.588511 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.588488    2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 08 15:17:27.603176 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.603154    2560 log.go:25] "Validated CRI v1 runtime API"
Sep 08 15:17:27.606894 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.606880    2560 log.go:25] "Validated CRI v1 image API"
Sep 08 15:17:27.607725 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.607703    2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 08 15:17:27.610128 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.610110    2560 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8bc84d4e-90e1-488b-a349-217a97ed9512:/dev/nvme0n1p4 a225db07-c243-4515-99a7-a79c75bc4907:/dev/nvme0n1p3]
Sep 08 15:17:27.610177 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.610128    2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 08 15:17:27.612623 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.612601    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 08 15:17:27.615590 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.615487    2560 manager.go:216] Machine: {Timestamp:2026-09-08 15:17:27.614014778 +0000 UTC m=+0.325957260 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3112320 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29a214a38d4572f1d1ada749f0cd39 SystemUUID:ec29a214-a38d-4572-f1d1-ada749f0cd39 BootID:4e5b89d5-adf5-4c96-a157-42e3ffaf8227 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:70:16:f5:cf:15 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:70:16:f5:cf:15 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f6:e0:1a:cd:4b:9f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 08 15:17:27.615590 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.615586    2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 08 15:17:27.615689 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.615665    2560 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 08 15:17:27.618796 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.618774    2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 08 15:17:27.618937 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.618800    2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-43.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 08 15:17:27.619721 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.619711    2560 topology_manager.go:138] "Creating topology manager with none policy"
Sep 08 15:17:27.619761 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.619722    2560 container_manager_linux.go:309] "Creating device plugin manager"
Sep 08 15:17:27.619761 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.619730    2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 08 15:17:27.619761 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.619744    2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 08 15:17:27.620882 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.620871    2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 08 15:17:27.624235 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.624224    2560 state_mem.go:36] "Initialized new in-memory state store"
Sep 08 15:17:27.624344 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.624334    2560 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 08 15:17:27.624380 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.624363    2560 kubelet.go:486] "Attempting to sync node with API server"
Sep 08 15:17:27.624380 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.624373    2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 08 15:17:27.624442 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.624388    2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 08 15:17:27.624442 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.624397    2560 kubelet.go:398] "Adding apiserver pod source"
Sep 08 15:17:27.624442 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.624405    2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 08 15:17:27.624765 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.624753    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 08 15:17:27.624839 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.624772    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 08 15:17:27.627976 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.627951    2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 08 15:17:27.630382 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.630365    2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 08 15:17:27.630460 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.630404    2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 08 15:17:27.631672 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.631659    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 08 15:17:27.631735 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.631680    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 08 15:17:27.631735 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.631690    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 08 15:17:27.631735 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.631698    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 08 15:17:27.631735 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.631711    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 08 15:17:27.631735 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.631721    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 08 15:17:27.631917 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.631744    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 08 15:17:27.631917 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.631753    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 08 15:17:27.631917 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.631764    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 08 15:17:27.631917 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.631773    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 08 15:17:27.631917 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.631786    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 08 15:17:27.631917 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.631796    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 08 15:17:27.632661 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.632649    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 08 15:17:27.632720 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.632663    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 08 15:17:27.634905 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.634884    2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-43.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 08 15:17:27.635013 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:27.634903    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-43.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 08 15:17:27.635013 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:27.634961    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 08 15:17:27.636447 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.636431    2560 server.go:1268] "Started kubelet"
Sep 08 15:17:27.637253 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.637224    2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 08 15:17:27.638037 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.638006    2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 08 15:17:27.638046 ip-10-0-137-43 systemd[1]: Started Kubernetes Kubelet.
Sep 08 15:17:27.639901 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.639832    2560 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 08 15:17:27.641902 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.641885    2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 08 15:17:27.644402 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.644352    2560 server.go:310] "Adding debug handlers to kubelet server"
Sep 08 15:17:27.644719 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:27.644677    2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 08 15:17:27.647194 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.647178    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 08 15:17:27.647836 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.647822    2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 08 15:17:27.648013 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:27.647988    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-43.ec2.internal\" not found"
Sep 08 15:17:27.648588 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.648573    2560 factory.go:55] Registering systemd factory
Sep 08 15:17:27.648588 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.648589    2560 factory.go:223] Registration of the systemd container factory successfully
Sep 08 15:17:27.648709 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.648635    2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 08 15:17:27.648709 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.648637    2560 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 08 15:17:27.648709 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.648663    2560 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 08 15:17:27.648840 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.648775    2560 reconstruct.go:97] "Volume reconstruction finished"
Sep 08 15:17:27.648840 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.648782    2560 reconciler.go:29] "Reconciler: start to sync state"
Sep 08 15:17:27.648930 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.648845    2560 factory.go:153] Registering CRI-O factory
Sep 08 15:17:27.648930 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.648857    2560 factory.go:223] Registration of the crio container factory successfully
Sep 08 15:17:27.648930 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.648905    2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 08 15:17:27.648930 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.648922    2560 factory.go:103] Registering Raw factory
Sep 08 15:17:27.649097 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.648936    2560 manager.go:1196] Started watching for new ooms in manager
Sep 08 15:17:27.649875 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.649861    2560 manager.go:349] Starting recovery of all containers
Sep 08 15:17:27.649875 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:27.648828    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-43.ec2.internal.18d361789bf03bdc  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-43.ec2.internal,UID:ip-10-0-137-43.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-43.ec2.internal,},FirstTimestamp:2026-09-08 15:17:27.636401116 +0000 UTC m=+0.348343598,LastTimestamp:2026-09-08 15:17:27.636401116 +0000 UTC m=+0.348343598,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-43.ec2.internal,}"
Sep 08 15:17:27.655866 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:27.655844    2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-43.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 08 15:17:27.656131 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:27.655941    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 08 15:17:27.659840 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.659824    2560 manager.go:354] Recovery completed
Sep 08 15:17:27.661790 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.661766    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tqmtm"
Sep 08 15:17:27.663953 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.663939    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:17:27.665564 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.665545    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-43.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:17:27.665654 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.665621    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-43.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:17:27.665654 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.665648    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-43.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:17:27.666351 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.666332    2560 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 08 15:17:27.666351 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.666350    2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 08 15:17:27.666484 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.666357    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:17:27.666484 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.666369    2560 state_mem.go:36] "Initialized new in-memory state store"
Sep 08 15:17:27.667025 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.667003    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-43.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:17:27.667025 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.667033    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-43.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:17:27.667184 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.667047    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-43.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:17:27.667394 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:27.667307    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-43.ec2.internal.18d361789dad3443  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-43.ec2.internal,UID:ip-10-0-137-43.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-43.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-43.ec2.internal,},FirstTimestamp:2026-09-08 15:17:27.665562691 +0000 UTC m=+0.377505178,LastTimestamp:2026-09-08 15:17:27.665562691 +0000 UTC m=+0.377505178,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-43.ec2.internal,}"
Sep 08 15:17:27.667746 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.667735    2560 policy_none.go:49] "None policy: Start"
Sep 08 15:17:27.667782 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.667749    2560 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 08 15:17:27.667782 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.667760    2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 08 15:17:27.668599 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.668587    2560 policy_none.go:47] "Start"
Sep 08 15:17:27.670098 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.670080    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tqmtm"
Sep 08 15:17:27.676623 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:27.676540    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-43.ec2.internal.18d361789dae3d3c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-43.ec2.internal,UID:ip-10-0-137-43.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-43.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-43.ec2.internal,},FirstTimestamp:2026-09-08 15:17:27.665630524 +0000 UTC m=+0.377573007,LastTimestamp:2026-09-08 15:17:27.665630524 +0000 UTC m=+0.377573007,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-43.ec2.internal,}"
Sep 08 15:17:27.698664 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.698648    2560 manager.go:337] "Starting Device Plugin manager"
Sep 08 15:17:27.712191 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:27.698689    2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 08 15:17:27.712191 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.698701    2560 server.go:87] "Starting device plugin registration server"
Sep 08 15:17:27.712191 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.699016    2560 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 08 15:17:27.712191 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.699037    2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 08 15:17:27.712191 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.699134    2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 08 15:17:27.712191 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.699201    2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 08 15:17:27.712191 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.699207    2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 08 15:17:27.712191 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:27.699788    2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 08 15:17:27.712191 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:27.699830    2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-43.ec2.internal\" not found"
Sep 08 15:17:27.792159 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.792117    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 08 15:17:27.793390 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.793363    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 08 15:17:27.793517 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.793422    2560 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 08 15:17:27.793517 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.793448    2560 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 08 15:17:27.793517 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:27.793492    2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 08 15:17:27.796254 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.796234    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 15:17:27.799307 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.799294    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:17:27.800121 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.800100    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-43.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:17:27.800225 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.800132    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-43.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:17:27.800225 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.800144    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-43.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:17:27.800225 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.800166    2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-43.ec2.internal"
Sep 08 15:17:27.810959 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.810873    2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-43.ec2.internal"
Sep 08 15:17:27.810959 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:27.810898    2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-43.ec2.internal\": node \"ip-10-0-137-43.ec2.internal\" not found"
Sep 08 15:17:27.839491 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:27.839466    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-43.ec2.internal\" not found"
Sep 08 15:17:27.894256 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.894225    2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-43.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-43.ec2.internal"]
Sep 08 15:17:27.894359 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.894293    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:17:27.895221 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.895205    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-43.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:17:27.895280 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.895237    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-43.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:17:27.895280 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.895250    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-43.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:17:27.896595 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.896582    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:17:27.896724 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.896709    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-43.ec2.internal"
Sep 08 15:17:27.896760 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.896743    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:17:27.897307 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.897291    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-43.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:17:27.897403 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.897296    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-43.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:17:27.897403 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.897321    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-43.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:17:27.897403 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.897332    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-43.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:17:27.897403 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.897339    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-43.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:17:27.897403 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.897349    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-43.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:17:27.898554 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.898539    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-43.ec2.internal"
Sep 08 15:17:27.898651 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.898568    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:17:27.899264 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.899252    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-43.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:17:27.899346 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.899278    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-43.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:17:27.899346 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.899292    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-43.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:17:27.925680 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:27.925662    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-43.ec2.internal\" not found" node="ip-10-0-137-43.ec2.internal"
Sep 08 15:17:27.930014 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:27.929997    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-43.ec2.internal\" not found" node="ip-10-0-137-43.ec2.internal"
Sep 08 15:17:27.939768 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:27.939751    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-43.ec2.internal\" not found"
Sep 08 15:17:27.950728 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.950708    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f7c1882d19ab40f24851c6d6b5b879ef-config\") pod \"kube-apiserver-proxy-ip-10-0-137-43.ec2.internal\" (UID: \"f7c1882d19ab40f24851c6d6b5b879ef\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-43.ec2.internal"
Sep 08 15:17:27.950780 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.950732    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/613a137b636df8e8d8c322049cd80abe-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-43.ec2.internal\" (UID: \"613a137b636df8e8d8c322049cd80abe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-43.ec2.internal"
Sep 08 15:17:27.950780 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:27.950750    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/613a137b636df8e8d8c322049cd80abe-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-43.ec2.internal\" (UID: \"613a137b636df8e8d8c322049cd80abe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-43.ec2.internal"
Sep 08 15:17:28.040501 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:28.040471    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-43.ec2.internal\" not found"
Sep 08 15:17:28.051858 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:28.051836    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/613a137b636df8e8d8c322049cd80abe-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-43.ec2.internal\" (UID: \"613a137b636df8e8d8c322049cd80abe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-43.ec2.internal"
Sep 08 15:17:28.051919 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:28.051863    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/613a137b636df8e8d8c322049cd80abe-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-43.ec2.internal\" (UID: \"613a137b636df8e8d8c322049cd80abe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-43.ec2.internal"
Sep 08 15:17:28.051919 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:28.051881    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f7c1882d19ab40f24851c6d6b5b879ef-config\") pod \"kube-apiserver-proxy-ip-10-0-137-43.ec2.internal\" (UID: \"f7c1882d19ab40f24851c6d6b5b879ef\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-43.ec2.internal"
Sep 08 15:17:28.051993 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:28.051947    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f7c1882d19ab40f24851c6d6b5b879ef-config\") pod \"kube-apiserver-proxy-ip-10-0-137-43.ec2.internal\" (UID: \"f7c1882d19ab40f24851c6d6b5b879ef\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-43.ec2.internal"
Sep 08 15:17:28.051993 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:28.051955    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/613a137b636df8e8d8c322049cd80abe-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-43.ec2.internal\" (UID: \"613a137b636df8e8d8c322049cd80abe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-43.ec2.internal"
Sep 08 15:17:28.051993 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:28.051955    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/613a137b636df8e8d8c322049cd80abe-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-43.ec2.internal\" (UID: \"613a137b636df8e8d8c322049cd80abe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-43.ec2.internal"
Sep 08 15:17:28.141329 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:28.141261    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-43.ec2.internal\" not found"
Sep 08 15:17:28.227823 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:28.227791    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-43.ec2.internal"
Sep 08 15:17:28.233468 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:28.233451    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-43.ec2.internal"
Sep 08 15:17:28.241622 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:28.241604    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-43.ec2.internal\" not found"
Sep 08 15:17:28.342128 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:28.342092    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-43.ec2.internal\" not found"
Sep 08 15:17:28.442664 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:28.442605    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-43.ec2.internal\" not found"
Sep 08 15:17:28.543653 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:28.543629    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-43.ec2.internal\" not found"
Sep 08 15:17:28.578995 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:28.578970    2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 08 15:17:28.579406 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:28.579130    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 08 15:17:28.643930 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:28.643904    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-43.ec2.internal\" not found"
Sep 08 15:17:28.648373 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:28.648352    2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 15:17:28.677306 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:28.677268    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-07 15:12:28 +0000 UTC" deadline="2028-04-08 05:06:58.349581142 +0000 UTC"
Sep 08 15:17:28.677306 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:28.677299    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13861h49m29.672285115s"
Sep 08 15:17:28.744321 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:28.744240    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-43.ec2.internal\" not found"
Sep 08 15:17:28.797347 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:28.797328    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:17:28.798320 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:28.798303    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-43.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:17:28.798399 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:28.798334    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-43.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:17:28.798399 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:28.798344    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-43.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:17:28.844587 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:28.844556    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-43.ec2.internal\" not found"
Sep 08 15:17:28.879226 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:28.879202    2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 15:17:28.901433 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:28.901402    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7c1882d19ab40f24851c6d6b5b879ef.slice/crio-7a4d04ba7d86587cc73edb3d20abdb7226f69ccd9307997c9bf81edccd4963e1 WatchSource:0}: Error finding container 7a4d04ba7d86587cc73edb3d20abdb7226f69ccd9307997c9bf81edccd4963e1: Status 404 returned error can't find the container with id 7a4d04ba7d86587cc73edb3d20abdb7226f69ccd9307997c9bf81edccd4963e1
Sep 08 15:17:28.905582 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:28.905567    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 15:17:28.948270 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:28.948250    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-43.ec2.internal"
Sep 08 15:17:28.963018 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:28.963000    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 08 15:17:28.964841 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:28.964827    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-43.ec2.internal"
Sep 08 15:17:28.975600 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:28.975583    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 08 15:17:29.216496 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.216399    2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 15:17:29.330953 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:29.330923    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod613a137b636df8e8d8c322049cd80abe.slice/crio-3c890bbb3e46d68a2f607494d38131e8a89f2123a4d97e0bcf8081ed3b9b8f54 WatchSource:0}: Error finding container 3c890bbb3e46d68a2f607494d38131e8a89f2123a4d97e0bcf8081ed3b9b8f54: Status 404 returned error can't find the container with id 3c890bbb3e46d68a2f607494d38131e8a89f2123a4d97e0bcf8081ed3b9b8f54
Sep 08 15:17:29.627506 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.627408    2560 apiserver.go:52] "Watching apiserver"
Sep 08 15:17:29.632168 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.632130    2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 08 15:17:29.632525 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.632502    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-137-43.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q","openshift-cluster-node-tuning-operator/tuned-b66wt","openshift-dns/node-resolver-dvlfj","openshift-multus/multus-additional-cni-plugins-jstb5","openshift-multus/network-metrics-daemon-dv2mt","openshift-image-registry/node-ca-g7blt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-43.ec2.internal","openshift-multus/multus-d8449","openshift-network-diagnostics/network-check-target-mb5jg","openshift-network-operator/iptables-alerter-n7qbx","openshift-ovn-kubernetes/ovnkube-node-rgrvb","kube-system/global-pull-secret-syncer-4wzj5","kube-system/konnectivity-agent-5kkjk"]
Sep 08 15:17:29.635364 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.635343    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:17:29.635445 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:29.635424    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mb5jg" podUID="31fb2a24-8166-40fc-b65f-4e3bbcdba36c"
Sep 08 15:17:29.637472 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.637456    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-n7qbx"
Sep 08 15:17:29.639367 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.639344    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 08 15:17:29.639509 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.639370    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 08 15:17:29.639509 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.639400    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-85dt6\""
Sep 08 15:17:29.639509 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.639352    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 08 15:17:29.639736 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.639722    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5kkjk"
Sep 08 15:17:29.642177 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.642160    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.642770 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.642747    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 08 15:17:29.643021 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.643003    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-x5ww6\""
Sep 08 15:17:29.643221 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.643205    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 08 15:17:29.643847 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.643831    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mzqpl\""
Sep 08 15:17:29.644098 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.644064    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 08 15:17:29.644197 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.644107    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 08 15:17:29.644197 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.644145    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 08 15:17:29.644197 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.644160    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 08 15:17:29.644451 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.644436    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 08 15:17:29.644543 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.644496    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 08 15:17:29.644671 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.644656    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jstb5"
Sep 08 15:17:29.648476 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.647394    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:17:29.648476 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:29.647559    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dv2mt" podUID="443fd4ee-6833-4ac4-a15f-d4ebdf0925d5"
Sep 08 15:17:29.648476 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.648275    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 08 15:17:29.648476 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.648390    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 08 15:17:29.648476 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.648440    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5vf69\""
Sep 08 15:17:29.648476 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.648455    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 08 15:17:29.648836 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.648673    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 08 15:17:29.649183 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.649163    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 08 15:17:29.649435 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.649414    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 08 15:17:29.651586 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.651562    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g7blt"
Sep 08 15:17:29.653740 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.653664    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 08 15:17:29.653936 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.653917    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z55zc\""
Sep 08 15:17:29.654078 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.654059    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 08 15:17:29.654388 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.654370    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:29.654473 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:29.654441    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wzj5" podUID="4b05229d-340e-424a-8533-87c77e6b97d5"
Sep 08 15:17:29.654473 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.654449    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 08 15:17:29.657071 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.657056    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q"
Sep 08 15:17:29.660435 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.660409    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.661525 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.661504    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 08 15:17:29.661618 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.661533    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 08 15:17:29.661736 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.661720    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-c6hlm\""
Sep 08 15:17:29.661873 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.661852    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 08 15:17:29.661973 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.661962    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-run-openvswitch\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.662028 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.661983    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/202dffc4-2ade-40df-96be-4f6d20b0d998-cnibin\") pod \"multus-additional-cni-plugins-jstb5\" (UID: \"202dffc4-2ade-40df-96be-4f6d20b0d998\") " pod="openshift-multus/multus-additional-cni-plugins-jstb5"
Sep 08 15:17:29.662028 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662007    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk6sj\" (UniqueName: \"kubernetes.io/projected/443fd4ee-6833-4ac4-a15f-d4ebdf0925d5-kube-api-access-tk6sj\") pod \"network-metrics-daemon-dv2mt\" (UID: \"443fd4ee-6833-4ac4-a15f-d4ebdf0925d5\") " pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:17:29.662028 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662025    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/762d10ee-64c5-40e1-896c-47e0cd55d613-host\") pod \"node-ca-g7blt\" (UID: \"762d10ee-64c5-40e1-896c-47e0cd55d613\") " pod="openshift-image-registry/node-ca-g7blt"
Sep 08 15:17:29.662248 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662039    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-host-cni-bin\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.662248 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662056    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.662248 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662071    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-env-overrides\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.662248 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662086    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/202dffc4-2ade-40df-96be-4f6d20b0d998-cni-binary-copy\") pod \"multus-additional-cni-plugins-jstb5\" (UID: \"202dffc4-2ade-40df-96be-4f6d20b0d998\") " pod="openshift-multus/multus-additional-cni-plugins-jstb5"
Sep 08 15:17:29.662248 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662128    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/202dffc4-2ade-40df-96be-4f6d20b0d998-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jstb5\" (UID: \"202dffc4-2ade-40df-96be-4f6d20b0d998\") " pod="openshift-multus/multus-additional-cni-plugins-jstb5"
Sep 08 15:17:29.662248 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662149    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/08200567-2c28-4a77-83fb-c6f82cd9f42e-agent-certs\") pod \"konnectivity-agent-5kkjk\" (UID: \"08200567-2c28-4a77-83fb-c6f82cd9f42e\") " pod="kube-system/konnectivity-agent-5kkjk"
Sep 08 15:17:29.662248 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662172    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-host-kubelet\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.662248 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662198    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-systemd-units\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.662248 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662224    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-host-cni-netd\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.662248 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662244    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/443fd4ee-6833-4ac4-a15f-d4ebdf0925d5-metrics-certs\") pod \"network-metrics-daemon-dv2mt\" (UID: \"443fd4ee-6833-4ac4-a15f-d4ebdf0925d5\") " pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:17:29.662702 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662259    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-log-socket\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.662702 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662274    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhz7w\" (UniqueName: \"kubernetes.io/projected/202dffc4-2ade-40df-96be-4f6d20b0d998-kube-api-access-qhz7w\") pod \"multus-additional-cni-plugins-jstb5\" (UID: \"202dffc4-2ade-40df-96be-4f6d20b0d998\") " pod="openshift-multus/multus-additional-cni-plugins-jstb5"
Sep 08 15:17:29.662702 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662316    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/08200567-2c28-4a77-83fb-c6f82cd9f42e-konnectivity-ca\") pod \"konnectivity-agent-5kkjk\" (UID: \"08200567-2c28-4a77-83fb-c6f82cd9f42e\") " pod="kube-system/konnectivity-agent-5kkjk"
Sep 08 15:17:29.662702 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662338    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-var-lib-openvswitch\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.662702 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662361    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-etc-openvswitch\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.662702 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662387    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-ovnkube-config\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.662702 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662434    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/202dffc4-2ade-40df-96be-4f6d20b0d998-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jstb5\" (UID: \"202dffc4-2ade-40df-96be-4f6d20b0d998\") " pod="openshift-multus/multus-additional-cni-plugins-jstb5"
Sep 08 15:17:29.662702 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662472    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/762d10ee-64c5-40e1-896c-47e0cd55d613-serviceca\") pod \"node-ca-g7blt\" (UID: \"762d10ee-64c5-40e1-896c-47e0cd55d613\") " pod="openshift-image-registry/node-ca-g7blt"
Sep 08 15:17:29.662702 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662510    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-host-slash\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.662702 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662554    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-node-log\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.662702 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662579    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-host-run-ovn-kubernetes\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.662702 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662613    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-ovnkube-script-lib\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.662702 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662672    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/202dffc4-2ade-40df-96be-4f6d20b0d998-system-cni-dir\") pod \"multus-additional-cni-plugins-jstb5\" (UID: \"202dffc4-2ade-40df-96be-4f6d20b0d998\") " pod="openshift-multus/multus-additional-cni-plugins-jstb5"
Sep 08 15:17:29.663474 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662722    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/202dffc4-2ade-40df-96be-4f6d20b0d998-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jstb5\" (UID: \"202dffc4-2ade-40df-96be-4f6d20b0d998\") " pod="openshift-multus/multus-additional-cni-plugins-jstb5"
Sep 08 15:17:29.663474 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662745    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2a353d7a-6ed5-475c-b1eb-3624a6773ecb-host-slash\") pod \"iptables-alerter-n7qbx\" (UID: \"2a353d7a-6ed5-475c-b1eb-3624a6773ecb\") " pod="openshift-network-operator/iptables-alerter-n7qbx"
Sep 08 15:17:29.663474 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662760    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-run-systemd\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.663474 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662783    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-run-ovn\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.663474 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662801    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-ovn-node-metrics-cert\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.663474 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662815    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npmx5\" (UniqueName: \"kubernetes.io/projected/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-kube-api-access-npmx5\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.663474 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662832    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8mqc\" (UniqueName: \"kubernetes.io/projected/2a353d7a-6ed5-475c-b1eb-3624a6773ecb-kube-api-access-c8mqc\") pod \"iptables-alerter-n7qbx\" (UID: \"2a353d7a-6ed5-475c-b1eb-3624a6773ecb\") " pod="openshift-network-operator/iptables-alerter-n7qbx"
Sep 08 15:17:29.663474 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662862    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz9hv\" (UniqueName: \"kubernetes.io/projected/762d10ee-64c5-40e1-896c-47e0cd55d613-kube-api-access-vz9hv\") pod \"node-ca-g7blt\" (UID: \"762d10ee-64c5-40e1-896c-47e0cd55d613\") " pod="openshift-image-registry/node-ca-g7blt"
Sep 08 15:17:29.663474 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662901    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gk4dj\" (UniqueName: \"kubernetes.io/projected/31fb2a24-8166-40fc-b65f-4e3bbcdba36c-kube-api-access-gk4dj\") pod \"network-check-target-mb5jg\" (UID: \"31fb2a24-8166-40fc-b65f-4e3bbcdba36c\") " pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:17:29.663474 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662924    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2a353d7a-6ed5-475c-b1eb-3624a6773ecb-iptables-alerter-script\") pod \"iptables-alerter-n7qbx\" (UID: \"2a353d7a-6ed5-475c-b1eb-3624a6773ecb\") " pod="openshift-network-operator/iptables-alerter-n7qbx"
Sep 08 15:17:29.663474 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662930    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 08 15:17:29.663474 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662949    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-host-run-netns\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.663474 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.662983    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dvlfj"
Sep 08 15:17:29.663474 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.663196    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-67dtq\""
Sep 08 15:17:29.664676 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.664658    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 08 15:17:29.664888 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.664869    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 08 15:17:29.664978 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.664871    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 08 15:17:29.664978 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.664877    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-g4nlv\""
Sep 08 15:17:29.665677 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.665658    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.673951 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.673856    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 08 15:17:29.673951 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.673868    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7nkfn\""
Sep 08 15:17:29.674067 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.673960    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 08 15:17:29.691457 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.691439    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-tpmk4"
Sep 08 15:17:29.701833 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.701817    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tpmk4"
Sep 08 15:17:29.721559 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.721537    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 15:17:29.751965 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.750988    2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 08 15:17:29.764047 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764021    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npmx5\" (UniqueName: \"kubernetes.io/projected/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-kube-api-access-npmx5\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.764157 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764053    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a5785ee7-6851-45c1-aa4b-ecb22455837c-device-dir\") pod \"aws-ebs-csi-driver-node-pd26q\" (UID: \"a5785ee7-6851-45c1-aa4b-ecb22455837c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q"
Sep 08 15:17:29.764157 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764073    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/202dffc4-2ade-40df-96be-4f6d20b0d998-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jstb5\" (UID: \"202dffc4-2ade-40df-96be-4f6d20b0d998\") " pod="openshift-multus/multus-additional-cni-plugins-jstb5"
Sep 08 15:17:29.764157 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764105    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/762d10ee-64c5-40e1-896c-47e0cd55d613-serviceca\") pod \"node-ca-g7blt\" (UID: \"762d10ee-64c5-40e1-896c-47e0cd55d613\") " pod="openshift-image-registry/node-ca-g7blt"
Sep 08 15:17:29.764157 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764121    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-host-slash\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.764157 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764137    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-node-log\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.764157 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764152    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-host-var-lib-cni-multus\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.764461 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764169    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2a353d7a-6ed5-475c-b1eb-3624a6773ecb-host-slash\") pod \"iptables-alerter-n7qbx\" (UID: \"2a353d7a-6ed5-475c-b1eb-3624a6773ecb\") " pod="openshift-network-operator/iptables-alerter-n7qbx"
Sep 08 15:17:29.764461 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764212    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-run-systemd\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.764461 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764238    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-host-slash\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.764461 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764258    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-run-systemd\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.764461 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764261    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-ovn-node-metrics-cert\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.764461 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764260    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-node-log\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.764461 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764289    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2a353d7a-6ed5-475c-b1eb-3624a6773ecb-host-slash\") pod \"iptables-alerter-n7qbx\" (UID: \"2a353d7a-6ed5-475c-b1eb-3624a6773ecb\") " pod="openshift-network-operator/iptables-alerter-n7qbx"
Sep 08 15:17:29.764461 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764297    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a5785ee7-6851-45c1-aa4b-ecb22455837c-registration-dir\") pod \"aws-ebs-csi-driver-node-pd26q\" (UID: \"a5785ee7-6851-45c1-aa4b-ecb22455837c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q"
Sep 08 15:17:29.764461 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764352    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-hostroot\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.764461 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764384    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-etc-systemd\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.764461 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764421    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz9hv\" (UniqueName: \"kubernetes.io/projected/762d10ee-64c5-40e1-896c-47e0cd55d613-kube-api-access-vz9hv\") pod \"node-ca-g7blt\" (UID: \"762d10ee-64c5-40e1-896c-47e0cd55d613\") " pod="openshift-image-registry/node-ca-g7blt"
Sep 08 15:17:29.764461 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764458    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-host-run-netns\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.765060 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764513    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-host-run-netns\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.765060 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764588    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-run-openvswitch\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.765060 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764628    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-host-run-k8s-cni-cncf-io\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.765060 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764657    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-host-run-multus-certs\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.765060 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764657    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-run-openvswitch\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.765060 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764686    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/202dffc4-2ade-40df-96be-4f6d20b0d998-cnibin\") pod \"multus-additional-cni-plugins-jstb5\" (UID: \"202dffc4-2ade-40df-96be-4f6d20b0d998\") " pod="openshift-multus/multus-additional-cni-plugins-jstb5"
Sep 08 15:17:29.765060 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764711    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/762d10ee-64c5-40e1-896c-47e0cd55d613-host\") pod \"node-ca-g7blt\" (UID: \"762d10ee-64c5-40e1-896c-47e0cd55d613\") " pod="openshift-image-registry/node-ca-g7blt"
Sep 08 15:17:29.765060 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764739    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4b05229d-340e-424a-8533-87c77e6b97d5-dbus\") pod \"global-pull-secret-syncer-4wzj5\" (UID: \"4b05229d-340e-424a-8533-87c77e6b97d5\") " pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:29.765060 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764763    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84wdn\" (UniqueName: \"kubernetes.io/projected/4377c8d9-0724-4d65-8a61-49c2f643cb59-kube-api-access-84wdn\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.765060 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764766    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/762d10ee-64c5-40e1-896c-47e0cd55d613-serviceca\") pod \"node-ca-g7blt\" (UID: \"762d10ee-64c5-40e1-896c-47e0cd55d613\") " pod="openshift-image-registry/node-ca-g7blt"
Sep 08 15:17:29.765060 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764781    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e26148ba-c65b-4e39-bd74-2f74c1ba38e6-hosts-file\") pod \"node-resolver-dvlfj\" (UID: \"e26148ba-c65b-4e39-bd74-2f74c1ba38e6\") " pod="openshift-dns/node-resolver-dvlfj"
Sep 08 15:17:29.765060 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764776    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/762d10ee-64c5-40e1-896c-47e0cd55d613-host\") pod \"node-ca-g7blt\" (UID: \"762d10ee-64c5-40e1-896c-47e0cd55d613\") " pod="openshift-image-registry/node-ca-g7blt"
Sep 08 15:17:29.765060 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764779    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/202dffc4-2ade-40df-96be-4f6d20b0d998-cnibin\") pod \"multus-additional-cni-plugins-jstb5\" (UID: \"202dffc4-2ade-40df-96be-4f6d20b0d998\") " pod="openshift-multus/multus-additional-cni-plugins-jstb5"
Sep 08 15:17:29.765060 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764806    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkbt7\" (UniqueName: \"kubernetes.io/projected/e26148ba-c65b-4e39-bd74-2f74c1ba38e6-kube-api-access-rkbt7\") pod \"node-resolver-dvlfj\" (UID: \"e26148ba-c65b-4e39-bd74-2f74c1ba38e6\") " pod="openshift-dns/node-resolver-dvlfj"
Sep 08 15:17:29.765060 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764843    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-etc-kubernetes\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.765060 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764848    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/202dffc4-2ade-40df-96be-4f6d20b0d998-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jstb5\" (UID: \"202dffc4-2ade-40df-96be-4f6d20b0d998\") " pod="openshift-multus/multus-additional-cni-plugins-jstb5"
Sep 08 15:17:29.765060 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764881    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-run\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.765834 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764909    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/08200567-2c28-4a77-83fb-c6f82cd9f42e-agent-certs\") pod \"konnectivity-agent-5kkjk\" (UID: \"08200567-2c28-4a77-83fb-c6f82cd9f42e\") " pod="kube-system/konnectivity-agent-5kkjk"
Sep 08 15:17:29.765834 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764936    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-host-kubelet\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.765834 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764987    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-host-kubelet\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.765834 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.764988    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-systemd-units\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.765834 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765026    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-systemd-units\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.765834 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765028    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-host-cni-netd\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.765834 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765051    2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 08 15:17:29.765834 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765068    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b05229d-340e-424a-8533-87c77e6b97d5-original-pull-secret\") pod \"global-pull-secret-syncer-4wzj5\" (UID: \"4b05229d-340e-424a-8533-87c77e6b97d5\") " pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:29.765834 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765109    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-multus-socket-dir-parent\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.765834 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765126    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-host-run-netns\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.765834 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765142    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-host-var-lib-cni-bin\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.765834 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765173    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-multus-conf-dir\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.765834 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765074    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-host-cni-netd\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.765834 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765202    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-etc-modprobe-d\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.765834 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765223    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-tmp\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.765834 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765254    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhz7w\" (UniqueName: \"kubernetes.io/projected/202dffc4-2ade-40df-96be-4f6d20b0d998-kube-api-access-qhz7w\") pod \"multus-additional-cni-plugins-jstb5\" (UID: \"202dffc4-2ade-40df-96be-4f6d20b0d998\") " pod="openshift-multus/multus-additional-cni-plugins-jstb5"
Sep 08 15:17:29.765834 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765283    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/08200567-2c28-4a77-83fb-c6f82cd9f42e-konnectivity-ca\") pod \"konnectivity-agent-5kkjk\" (UID: \"08200567-2c28-4a77-83fb-c6f82cd9f42e\") " pod="kube-system/konnectivity-agent-5kkjk"
Sep 08 15:17:29.766539 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765305    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-var-lib-openvswitch\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.766539 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765330    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-etc-openvswitch\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.766539 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765357    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-ovnkube-config\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.766539 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765379    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-var-lib-openvswitch\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.766539 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765403    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-etc-openvswitch\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.766539 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765448    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-cnibin\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.766539 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765473    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-host-run-ovn-kubernetes\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.766539 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765501    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-ovnkube-script-lib\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.766539 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765528    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/202dffc4-2ade-40df-96be-4f6d20b0d998-system-cni-dir\") pod \"multus-additional-cni-plugins-jstb5\" (UID: \"202dffc4-2ade-40df-96be-4f6d20b0d998\") " pod="openshift-multus/multus-additional-cni-plugins-jstb5"
Sep 08 15:17:29.766539 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765529    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-host-run-ovn-kubernetes\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.766539 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765589    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/202dffc4-2ade-40df-96be-4f6d20b0d998-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jstb5\" (UID: \"202dffc4-2ade-40df-96be-4f6d20b0d998\") " pod="openshift-multus/multus-additional-cni-plugins-jstb5"
Sep 08 15:17:29.766539 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765598    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/202dffc4-2ade-40df-96be-4f6d20b0d998-system-cni-dir\") pod \"multus-additional-cni-plugins-jstb5\" (UID: \"202dffc4-2ade-40df-96be-4f6d20b0d998\") " pod="openshift-multus/multus-additional-cni-plugins-jstb5"
Sep 08 15:17:29.766539 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765620    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-run-ovn\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.766539 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765680    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-run-ovn\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.766539 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765715    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4b05229d-340e-424a-8533-87c77e6b97d5-kubelet-config\") pod \"global-pull-secret-syncer-4wzj5\" (UID: \"4b05229d-340e-424a-8533-87c77e6b97d5\") " pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:29.766539 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765757    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a5785ee7-6851-45c1-aa4b-ecb22455837c-sys-fs\") pod \"aws-ebs-csi-driver-node-pd26q\" (UID: \"a5785ee7-6851-45c1-aa4b-ecb22455837c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q"
Sep 08 15:17:29.766539 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765784    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-etc-tuned\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.767443 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765813    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8mqc\" (UniqueName: \"kubernetes.io/projected/2a353d7a-6ed5-475c-b1eb-3624a6773ecb-kube-api-access-c8mqc\") pod \"iptables-alerter-n7qbx\" (UID: \"2a353d7a-6ed5-475c-b1eb-3624a6773ecb\") " pod="openshift-network-operator/iptables-alerter-n7qbx"
Sep 08 15:17:29.767443 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765856    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-host-var-lib-kubelet\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.767443 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765883    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-etc-sysconfig\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.767443 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765892    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/08200567-2c28-4a77-83fb-c6f82cd9f42e-konnectivity-ca\") pod \"konnectivity-agent-5kkjk\" (UID: \"08200567-2c28-4a77-83fb-c6f82cd9f42e\") " pod="kube-system/konnectivity-agent-5kkjk"
Sep 08 15:17:29.767443 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765931    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-host\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.767443 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.765971    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk4dj\" (UniqueName: \"kubernetes.io/projected/31fb2a24-8166-40fc-b65f-4e3bbcdba36c-kube-api-access-gk4dj\") pod \"network-check-target-mb5jg\" (UID: \"31fb2a24-8166-40fc-b65f-4e3bbcdba36c\") " pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:17:29.767443 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766009    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2a353d7a-6ed5-475c-b1eb-3624a6773ecb-iptables-alerter-script\") pod \"iptables-alerter-n7qbx\" (UID: \"2a353d7a-6ed5-475c-b1eb-3624a6773ecb\") " pod="openshift-network-operator/iptables-alerter-n7qbx"
Sep 08 15:17:29.767443 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766033    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4377c8d9-0724-4d65-8a61-49c2f643cb59-cni-binary-copy\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.767443 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766055    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4377c8d9-0724-4d65-8a61-49c2f643cb59-multus-daemon-config\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.767443 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766080    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-ovnkube-config\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.767443 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766112    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-etc-kubernetes\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.767443 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766135    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-var-lib-kubelet\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.767443 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766209    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-ovnkube-script-lib\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.767443 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766220    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tk6sj\" (UniqueName: \"kubernetes.io/projected/443fd4ee-6833-4ac4-a15f-d4ebdf0925d5-kube-api-access-tk6sj\") pod \"network-metrics-daemon-dv2mt\" (UID: \"443fd4ee-6833-4ac4-a15f-d4ebdf0925d5\") " pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:17:29.767443 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766263    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-host-cni-bin\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.767443 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766290    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.767443 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766312    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-env-overrides\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.768366 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766334    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a5785ee7-6851-45c1-aa4b-ecb22455837c-socket-dir\") pod \"aws-ebs-csi-driver-node-pd26q\" (UID: \"a5785ee7-6851-45c1-aa4b-ecb22455837c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q"
Sep 08 15:17:29.768366 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766355    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-etc-sysctl-d\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.768366 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766336    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-host-cni-bin\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.768366 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766375    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-lib-modules\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.768366 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766418    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/202dffc4-2ade-40df-96be-4f6d20b0d998-cni-binary-copy\") pod \"multus-additional-cni-plugins-jstb5\" (UID: \"202dffc4-2ade-40df-96be-4f6d20b0d998\") " pod="openshift-multus/multus-additional-cni-plugins-jstb5"
Sep 08 15:17:29.768366 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766440    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/202dffc4-2ade-40df-96be-4f6d20b0d998-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jstb5\" (UID: \"202dffc4-2ade-40df-96be-4f6d20b0d998\") " pod="openshift-multus/multus-additional-cni-plugins-jstb5"
Sep 08 15:17:29.768366 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766454    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.768366 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766459    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/202dffc4-2ade-40df-96be-4f6d20b0d998-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jstb5\" (UID: \"202dffc4-2ade-40df-96be-4f6d20b0d998\") " pod="openshift-multus/multus-additional-cni-plugins-jstb5"
Sep 08 15:17:29.768366 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766463    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a5785ee7-6851-45c1-aa4b-ecb22455837c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pd26q\" (UID: \"a5785ee7-6851-45c1-aa4b-ecb22455837c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q"
Sep 08 15:17:29.768366 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766517    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbp69\" (UniqueName: \"kubernetes.io/projected/a5785ee7-6851-45c1-aa4b-ecb22455837c-kube-api-access-hbp69\") pod \"aws-ebs-csi-driver-node-pd26q\" (UID: \"a5785ee7-6851-45c1-aa4b-ecb22455837c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q"
Sep 08 15:17:29.768366 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766557    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/443fd4ee-6833-4ac4-a15f-d4ebdf0925d5-metrics-certs\") pod \"network-metrics-daemon-dv2mt\" (UID: \"443fd4ee-6833-4ac4-a15f-d4ebdf0925d5\") " pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:17:29.768366 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766583    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-log-socket\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.768366 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766606    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-multus-cni-dir\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.768366 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766632    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2a353d7a-6ed5-475c-b1eb-3624a6773ecb-iptables-alerter-script\") pod \"iptables-alerter-n7qbx\" (UID: \"2a353d7a-6ed5-475c-b1eb-3624a6773ecb\") " pod="openshift-network-operator/iptables-alerter-n7qbx"
Sep 08 15:17:29.768366 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766637    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/202dffc4-2ade-40df-96be-4f6d20b0d998-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jstb5\" (UID: \"202dffc4-2ade-40df-96be-4f6d20b0d998\") " pod="openshift-multus/multus-additional-cni-plugins-jstb5"
Sep 08 15:17:29.768366 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766631    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-etc-sysctl-conf\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.768366 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:29.766731    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:17:29.769076 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766793    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-log-socket\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.769076 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:29.766807    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/443fd4ee-6833-4ac4-a15f-d4ebdf0925d5-metrics-certs podName:443fd4ee-6833-4ac4-a15f-d4ebdf0925d5 nodeName:}" failed. No retries permitted until 2026-09-08 15:17:30.266765384 +0000 UTC m=+2.978707862 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/443fd4ee-6833-4ac4-a15f-d4ebdf0925d5-metrics-certs") pod "network-metrics-daemon-dv2mt" (UID: "443fd4ee-6833-4ac4-a15f-d4ebdf0925d5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:17:29.769076 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766824    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-sys\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.769076 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766849    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a5785ee7-6851-45c1-aa4b-ecb22455837c-etc-selinux\") pod \"aws-ebs-csi-driver-node-pd26q\" (UID: \"a5785ee7-6851-45c1-aa4b-ecb22455837c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q"
Sep 08 15:17:29.769076 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766871    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-system-cni-dir\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.769076 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766893    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e26148ba-c65b-4e39-bd74-2f74c1ba38e6-tmp-dir\") pod \"node-resolver-dvlfj\" (UID: \"e26148ba-c65b-4e39-bd74-2f74c1ba38e6\") " pod="openshift-dns/node-resolver-dvlfj"
Sep 08 15:17:29.769076 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766917    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79nlp\" (UniqueName: \"kubernetes.io/projected/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-kube-api-access-79nlp\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.769076 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.766924    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-env-overrides\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.769076 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.767038    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/202dffc4-2ade-40df-96be-4f6d20b0d998-cni-binary-copy\") pod \"multus-additional-cni-plugins-jstb5\" (UID: \"202dffc4-2ade-40df-96be-4f6d20b0d998\") " pod="openshift-multus/multus-additional-cni-plugins-jstb5"
Sep 08 15:17:29.769076 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.768864    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-ovn-node-metrics-cert\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.769076 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.769066    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/08200567-2c28-4a77-83fb-c6f82cd9f42e-agent-certs\") pod \"konnectivity-agent-5kkjk\" (UID: \"08200567-2c28-4a77-83fb-c6f82cd9f42e\") " pod="kube-system/konnectivity-agent-5kkjk"
Sep 08 15:17:29.773076 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.773050    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npmx5\" (UniqueName: \"kubernetes.io/projected/45ff8d65-b9ff-4741-91f8-d526b2ce65b3-kube-api-access-npmx5\") pod \"ovnkube-node-rgrvb\" (UID: \"45ff8d65-b9ff-4741-91f8-d526b2ce65b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.773211 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.773160    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz9hv\" (UniqueName: \"kubernetes.io/projected/762d10ee-64c5-40e1-896c-47e0cd55d613-kube-api-access-vz9hv\") pod \"node-ca-g7blt\" (UID: \"762d10ee-64c5-40e1-896c-47e0cd55d613\") " pod="openshift-image-registry/node-ca-g7blt"
Sep 08 15:17:29.773372 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.773353    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhz7w\" (UniqueName: \"kubernetes.io/projected/202dffc4-2ade-40df-96be-4f6d20b0d998-kube-api-access-qhz7w\") pod \"multus-additional-cni-plugins-jstb5\" (UID: \"202dffc4-2ade-40df-96be-4f6d20b0d998\") " pod="openshift-multus/multus-additional-cni-plugins-jstb5"
Sep 08 15:17:29.773840 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.773819    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8mqc\" (UniqueName: \"kubernetes.io/projected/2a353d7a-6ed5-475c-b1eb-3624a6773ecb-kube-api-access-c8mqc\") pod \"iptables-alerter-n7qbx\" (UID: \"2a353d7a-6ed5-475c-b1eb-3624a6773ecb\") " pod="openshift-network-operator/iptables-alerter-n7qbx"
Sep 08 15:17:29.774801 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:29.774784    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 15:17:29.774801 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:29.774803    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 15:17:29.774909 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:29.774813    2560 projected.go:196] Error preparing data for projected volume kube-api-access-gk4dj for pod openshift-network-diagnostics/network-check-target-mb5jg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:17:29.774909 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:29.774883    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31fb2a24-8166-40fc-b65f-4e3bbcdba36c-kube-api-access-gk4dj podName:31fb2a24-8166-40fc-b65f-4e3bbcdba36c nodeName:}" failed. No retries permitted until 2026-09-08 15:17:30.274868585 +0000 UTC m=+2.986811055 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-gk4dj" (UniqueName: "kubernetes.io/projected/31fb2a24-8166-40fc-b65f-4e3bbcdba36c-kube-api-access-gk4dj") pod "network-check-target-mb5jg" (UID: "31fb2a24-8166-40fc-b65f-4e3bbcdba36c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:17:29.780825 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.780803    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk6sj\" (UniqueName: \"kubernetes.io/projected/443fd4ee-6833-4ac4-a15f-d4ebdf0925d5-kube-api-access-tk6sj\") pod \"network-metrics-daemon-dv2mt\" (UID: \"443fd4ee-6833-4ac4-a15f-d4ebdf0925d5\") " pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:17:29.799526 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.799481    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-43.ec2.internal" event={"ID":"f7c1882d19ab40f24851c6d6b5b879ef","Type":"ContainerStarted","Data":"7a4d04ba7d86587cc73edb3d20abdb7226f69ccd9307997c9bf81edccd4963e1"}
Sep 08 15:17:29.800662 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.800639    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-43.ec2.internal" event={"ID":"613a137b636df8e8d8c322049cd80abe","Type":"ContainerStarted","Data":"3c890bbb3e46d68a2f607494d38131e8a89f2123a4d97e0bcf8081ed3b9b8f54"}
Sep 08 15:17:29.867309 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867268    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a5785ee7-6851-45c1-aa4b-ecb22455837c-etc-selinux\") pod \"aws-ebs-csi-driver-node-pd26q\" (UID: \"a5785ee7-6851-45c1-aa4b-ecb22455837c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q"
Sep 08 15:17:29.867309 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867305    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-system-cni-dir\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.867309 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867321    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e26148ba-c65b-4e39-bd74-2f74c1ba38e6-tmp-dir\") pod \"node-resolver-dvlfj\" (UID: \"e26148ba-c65b-4e39-bd74-2f74c1ba38e6\") " pod="openshift-dns/node-resolver-dvlfj"
Sep 08 15:17:29.867644 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867338    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79nlp\" (UniqueName: \"kubernetes.io/projected/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-kube-api-access-79nlp\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.867644 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867416    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a5785ee7-6851-45c1-aa4b-ecb22455837c-device-dir\") pod \"aws-ebs-csi-driver-node-pd26q\" (UID: \"a5785ee7-6851-45c1-aa4b-ecb22455837c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q"
Sep 08 15:17:29.867644 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867461    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-host-var-lib-cni-multus\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.867644 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867466    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a5785ee7-6851-45c1-aa4b-ecb22455837c-etc-selinux\") pod \"aws-ebs-csi-driver-node-pd26q\" (UID: \"a5785ee7-6851-45c1-aa4b-ecb22455837c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q"
Sep 08 15:17:29.867644 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867466    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-system-cni-dir\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.867644 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867492    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a5785ee7-6851-45c1-aa4b-ecb22455837c-registration-dir\") pod \"aws-ebs-csi-driver-node-pd26q\" (UID: \"a5785ee7-6851-45c1-aa4b-ecb22455837c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q"
Sep 08 15:17:29.867644 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867522    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-host-var-lib-cni-multus\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.867644 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867553    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a5785ee7-6851-45c1-aa4b-ecb22455837c-device-dir\") pod \"aws-ebs-csi-driver-node-pd26q\" (UID: \"a5785ee7-6851-45c1-aa4b-ecb22455837c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q"
Sep 08 15:17:29.867644 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867588    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-hostroot\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.867644 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867565    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a5785ee7-6851-45c1-aa4b-ecb22455837c-registration-dir\") pod \"aws-ebs-csi-driver-node-pd26q\" (UID: \"a5785ee7-6851-45c1-aa4b-ecb22455837c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q"
Sep 08 15:17:29.867644 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867549    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-hostroot\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.867644 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867635    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-etc-systemd\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.868234 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867662    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-host-run-k8s-cni-cncf-io\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.868234 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867675    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e26148ba-c65b-4e39-bd74-2f74c1ba38e6-tmp-dir\") pod \"node-resolver-dvlfj\" (UID: \"e26148ba-c65b-4e39-bd74-2f74c1ba38e6\") " pod="openshift-dns/node-resolver-dvlfj"
Sep 08 15:17:29.868234 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867689    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-host-run-multus-certs\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.868234 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867720    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4b05229d-340e-424a-8533-87c77e6b97d5-dbus\") pod \"global-pull-secret-syncer-4wzj5\" (UID: \"4b05229d-340e-424a-8533-87c77e6b97d5\") " pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:29.868234 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867711    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-etc-systemd\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.868234 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867744    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-host-run-k8s-cni-cncf-io\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.868234 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867746    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84wdn\" (UniqueName: \"kubernetes.io/projected/4377c8d9-0724-4d65-8a61-49c2f643cb59-kube-api-access-84wdn\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.868234 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867791    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e26148ba-c65b-4e39-bd74-2f74c1ba38e6-hosts-file\") pod \"node-resolver-dvlfj\" (UID: \"e26148ba-c65b-4e39-bd74-2f74c1ba38e6\") " pod="openshift-dns/node-resolver-dvlfj"
Sep 08 15:17:29.868234 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867796    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4b05229d-340e-424a-8533-87c77e6b97d5-dbus\") pod \"global-pull-secret-syncer-4wzj5\" (UID: \"4b05229d-340e-424a-8533-87c77e6b97d5\") " pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:29.868234 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867758    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-host-run-multus-certs\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.868234 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867838    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkbt7\" (UniqueName: \"kubernetes.io/projected/e26148ba-c65b-4e39-bd74-2f74c1ba38e6-kube-api-access-rkbt7\") pod \"node-resolver-dvlfj\" (UID: \"e26148ba-c65b-4e39-bd74-2f74c1ba38e6\") " pod="openshift-dns/node-resolver-dvlfj"
Sep 08 15:17:29.868234 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867871    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-etc-kubernetes\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.868234 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867898    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-run\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.868234 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867929    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b05229d-340e-424a-8533-87c77e6b97d5-original-pull-secret\") pod \"global-pull-secret-syncer-4wzj5\" (UID: \"4b05229d-340e-424a-8533-87c77e6b97d5\") " pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:29.868234 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867943    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-etc-kubernetes\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.868234 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867954    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-multus-socket-dir-parent\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.868234 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867843    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e26148ba-c65b-4e39-bd74-2f74c1ba38e6-hosts-file\") pod \"node-resolver-dvlfj\" (UID: \"e26148ba-c65b-4e39-bd74-2f74c1ba38e6\") " pod="openshift-dns/node-resolver-dvlfj"
Sep 08 15:17:29.868234 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867960    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-run\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.869102 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.867985    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-host-run-netns\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.869102 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868008    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-multus-socket-dir-parent\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.869102 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868010    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-host-var-lib-cni-bin\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.869102 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868035    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-host-run-netns\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.869102 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868047    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-host-var-lib-cni-bin\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.869102 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868050    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-multus-conf-dir\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.869102 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:29.868045    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 15:17:29.869102 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868075    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-etc-modprobe-d\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.869102 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868105    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-multus-conf-dir\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.869102 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:29.868124    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b05229d-340e-424a-8533-87c77e6b97d5-original-pull-secret podName:4b05229d-340e-424a-8533-87c77e6b97d5 nodeName:}" failed. No retries permitted until 2026-09-08 15:17:30.368108206 +0000 UTC m=+3.080050682 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4b05229d-340e-424a-8533-87c77e6b97d5-original-pull-secret") pod "global-pull-secret-syncer-4wzj5" (UID: "4b05229d-340e-424a-8533-87c77e6b97d5") : object "kube-system"/"original-pull-secret" not registered
Sep 08 15:17:29.869102 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868110    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-tmp\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.869102 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868172    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-cnibin\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.869102 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868204    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4b05229d-340e-424a-8533-87c77e6b97d5-kubelet-config\") pod \"global-pull-secret-syncer-4wzj5\" (UID: \"4b05229d-340e-424a-8533-87c77e6b97d5\") " pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:29.869102 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868231    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a5785ee7-6851-45c1-aa4b-ecb22455837c-sys-fs\") pod \"aws-ebs-csi-driver-node-pd26q\" (UID: \"a5785ee7-6851-45c1-aa4b-ecb22455837c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q"
Sep 08 15:17:29.869102 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868244    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-etc-modprobe-d\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.869102 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868256    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-etc-tuned\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.869102 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868299    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-host-var-lib-kubelet\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.869102 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868304    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4b05229d-340e-424a-8533-87c77e6b97d5-kubelet-config\") pod \"global-pull-secret-syncer-4wzj5\" (UID: \"4b05229d-340e-424a-8533-87c77e6b97d5\") " pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:29.869732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868323    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-etc-sysconfig\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.869732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868302    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-cnibin\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.869732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868346    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a5785ee7-6851-45c1-aa4b-ecb22455837c-sys-fs\") pod \"aws-ebs-csi-driver-node-pd26q\" (UID: \"a5785ee7-6851-45c1-aa4b-ecb22455837c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q"
Sep 08 15:17:29.869732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868354    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-host-var-lib-kubelet\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.869732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868360    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-host\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.869732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868398    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-etc-sysconfig\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.869732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868400    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4377c8d9-0724-4d65-8a61-49c2f643cb59-cni-binary-copy\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.869732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868457    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-host\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.869732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868445    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4377c8d9-0724-4d65-8a61-49c2f643cb59-multus-daemon-config\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.869732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868506    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-etc-kubernetes\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.869732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868530    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-var-lib-kubelet\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.869732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868548    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-etc-kubernetes\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.869732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868555    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a5785ee7-6851-45c1-aa4b-ecb22455837c-socket-dir\") pod \"aws-ebs-csi-driver-node-pd26q\" (UID: \"a5785ee7-6851-45c1-aa4b-ecb22455837c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q"
Sep 08 15:17:29.869732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868579    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-etc-sysctl-d\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.869732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868602    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-lib-modules\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.869732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868628    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a5785ee7-6851-45c1-aa4b-ecb22455837c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pd26q\" (UID: \"a5785ee7-6851-45c1-aa4b-ecb22455837c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q"
Sep 08 15:17:29.869732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868659    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-var-lib-kubelet\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.869732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868666    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbp69\" (UniqueName: \"kubernetes.io/projected/a5785ee7-6851-45c1-aa4b-ecb22455837c-kube-api-access-hbp69\") pod \"aws-ebs-csi-driver-node-pd26q\" (UID: \"a5785ee7-6851-45c1-aa4b-ecb22455837c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q"
Sep 08 15:17:29.870285 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868722    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-multus-cni-dir\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.870285 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868750    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-etc-sysctl-conf\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.870285 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868775    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-sys\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.870285 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868859    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-sys\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.870285 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868895    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a5785ee7-6851-45c1-aa4b-ecb22455837c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pd26q\" (UID: \"a5785ee7-6851-45c1-aa4b-ecb22455837c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q"
Sep 08 15:17:29.870285 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868916    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4377c8d9-0724-4d65-8a61-49c2f643cb59-multus-cni-dir\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.870285 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.868926    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-etc-sysctl-d\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.870285 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.869007    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-lib-modules\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.870285 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.869055    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-etc-sysctl-conf\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.870285 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.869097    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4377c8d9-0724-4d65-8a61-49c2f643cb59-multus-daemon-config\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.870285 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.869325    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4377c8d9-0724-4d65-8a61-49c2f643cb59-cni-binary-copy\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.870285 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.869318    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a5785ee7-6851-45c1-aa4b-ecb22455837c-socket-dir\") pod \"aws-ebs-csi-driver-node-pd26q\" (UID: \"a5785ee7-6851-45c1-aa4b-ecb22455837c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q"
Sep 08 15:17:29.870659 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.870517    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-tmp\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.870696 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.870673    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-etc-tuned\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.889749 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.889677    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbp69\" (UniqueName: \"kubernetes.io/projected/a5785ee7-6851-45c1-aa4b-ecb22455837c-kube-api-access-hbp69\") pod \"aws-ebs-csi-driver-node-pd26q\" (UID: \"a5785ee7-6851-45c1-aa4b-ecb22455837c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q"
Sep 08 15:17:29.889855 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.889762    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84wdn\" (UniqueName: \"kubernetes.io/projected/4377c8d9-0724-4d65-8a61-49c2f643cb59-kube-api-access-84wdn\") pod \"multus-d8449\" (UID: \"4377c8d9-0724-4d65-8a61-49c2f643cb59\") " pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.889855 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.889815    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79nlp\" (UniqueName: \"kubernetes.io/projected/51a7da58-d825-4c5d-9cf8-16a7a1b99cc4-kube-api-access-79nlp\") pod \"tuned-b66wt\" (UID: \"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4\") " pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:29.889945 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.889921    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkbt7\" (UniqueName: \"kubernetes.io/projected/e26148ba-c65b-4e39-bd74-2f74c1ba38e6-kube-api-access-rkbt7\") pod \"node-resolver-dvlfj\" (UID: \"e26148ba-c65b-4e39-bd74-2f74c1ba38e6\") " pod="openshift-dns/node-resolver-dvlfj"
Sep 08 15:17:29.949980 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.949931    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-n7qbx"
Sep 08 15:17:29.957075 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.957052    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5kkjk"
Sep 08 15:17:29.960960 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:29.960933    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a353d7a_6ed5_475c_b1eb_3624a6773ecb.slice/crio-eba90fe540fe7f3f0c2166544bb283badf24fd17746595c71235baec95c58f75 WatchSource:0}: Error finding container eba90fe540fe7f3f0c2166544bb283badf24fd17746595c71235baec95c58f75: Status 404 returned error can't find the container with id eba90fe540fe7f3f0c2166544bb283badf24fd17746595c71235baec95c58f75
Sep 08 15:17:29.964997 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.964976    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:29.966347 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:29.966325    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08200567_2c28_4a77_83fb_c6f82cd9f42e.slice/crio-52008ac6b9eae260eae3f3605b3c45461d9f14a2138efb699110641cb4c8d37f WatchSource:0}: Error finding container 52008ac6b9eae260eae3f3605b3c45461d9f14a2138efb699110641cb4c8d37f: Status 404 returned error can't find the container with id 52008ac6b9eae260eae3f3605b3c45461d9f14a2138efb699110641cb4c8d37f
Sep 08 15:17:29.970977 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.970956    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jstb5"
Sep 08 15:17:29.975691 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:29.975672    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45ff8d65_b9ff_4741_91f8_d526b2ce65b3.slice/crio-9db5b998c709738717d4ed3d0c0783d0969fa32c109865a22a1f6ce55cfbafb3 WatchSource:0}: Error finding container 9db5b998c709738717d4ed3d0c0783d0969fa32c109865a22a1f6ce55cfbafb3: Status 404 returned error can't find the container with id 9db5b998c709738717d4ed3d0c0783d0969fa32c109865a22a1f6ce55cfbafb3
Sep 08 15:17:29.977867 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.976774    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g7blt"
Sep 08 15:17:29.983247 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.983227    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q"
Sep 08 15:17:29.983842 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:29.983654    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod202dffc4_2ade_40df_96be_4f6d20b0d998.slice/crio-c1291e9c710193a611384c650c92ab1cb3e7be689cbb2e9e73bb5e5a0e9acf94 WatchSource:0}: Error finding container c1291e9c710193a611384c650c92ab1cb3e7be689cbb2e9e73bb5e5a0e9acf94: Status 404 returned error can't find the container with id c1291e9c710193a611384c650c92ab1cb3e7be689cbb2e9e73bb5e5a0e9acf94
Sep 08 15:17:29.986927 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:29.986904    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod762d10ee_64c5_40e1_896c_47e0cd55d613.slice/crio-e17abb8726d79266026474aa539143583d7f1ab11a2510997f5b7be2e5e81565 WatchSource:0}: Error finding container e17abb8726d79266026474aa539143583d7f1ab11a2510997f5b7be2e5e81565: Status 404 returned error can't find the container with id e17abb8726d79266026474aa539143583d7f1ab11a2510997f5b7be2e5e81565
Sep 08 15:17:29.989098 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.989066    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d8449"
Sep 08 15:17:29.992120 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:29.992026    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5785ee7_6851_45c1_aa4b_ecb22455837c.slice/crio-713b89a5124ee532549dee4a9e72dcf85610b06473e8d03089dda1a29211da27 WatchSource:0}: Error finding container 713b89a5124ee532549dee4a9e72dcf85610b06473e8d03089dda1a29211da27: Status 404 returned error can't find the container with id 713b89a5124ee532549dee4a9e72dcf85610b06473e8d03089dda1a29211da27
Sep 08 15:17:29.995112 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:29.995087    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dvlfj"
Sep 08 15:17:29.997959 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:29.997937    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4377c8d9_0724_4d65_8a61_49c2f643cb59.slice/crio-27792d5f48b4dabaf21ece41871a53cef3d08b7618e8d48c1b51a56ef5023df9 WatchSource:0}: Error finding container 27792d5f48b4dabaf21ece41871a53cef3d08b7618e8d48c1b51a56ef5023df9: Status 404 returned error can't find the container with id 27792d5f48b4dabaf21ece41871a53cef3d08b7618e8d48c1b51a56ef5023df9
Sep 08 15:17:30.000289 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:30.000267    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b66wt"
Sep 08 15:17:30.005836 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:30.005817    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode26148ba_c65b_4e39_bd74_2f74c1ba38e6.slice/crio-9e5c6b3b1eff50a7fbf38146cb1f3481a2ec6baa5351d75478da731c3dbcec2b WatchSource:0}: Error finding container 9e5c6b3b1eff50a7fbf38146cb1f3481a2ec6baa5351d75478da731c3dbcec2b: Status 404 returned error can't find the container with id 9e5c6b3b1eff50a7fbf38146cb1f3481a2ec6baa5351d75478da731c3dbcec2b
Sep 08 15:17:30.010117 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:17:30.010096    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51a7da58_d825_4c5d_9cf8_16a7a1b99cc4.slice/crio-e4ecf394ee01036082b0a45f6c5e47b4fea70e3fff54fdb54f6ade93cbcd2854 WatchSource:0}: Error finding container e4ecf394ee01036082b0a45f6c5e47b4fea70e3fff54fdb54f6ade93cbcd2854: Status 404 returned error can't find the container with id e4ecf394ee01036082b0a45f6c5e47b4fea70e3fff54fdb54f6ade93cbcd2854
Sep 08 15:17:30.271688 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:30.271595    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/443fd4ee-6833-4ac4-a15f-d4ebdf0925d5-metrics-certs\") pod \"network-metrics-daemon-dv2mt\" (UID: \"443fd4ee-6833-4ac4-a15f-d4ebdf0925d5\") " pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:17:30.271845 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:30.271725    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:17:30.271845 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:30.271795    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/443fd4ee-6833-4ac4-a15f-d4ebdf0925d5-metrics-certs podName:443fd4ee-6833-4ac4-a15f-d4ebdf0925d5 nodeName:}" failed. No retries permitted until 2026-09-08 15:17:31.271774805 +0000 UTC m=+3.983717282 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/443fd4ee-6833-4ac4-a15f-d4ebdf0925d5-metrics-certs") pod "network-metrics-daemon-dv2mt" (UID: "443fd4ee-6833-4ac4-a15f-d4ebdf0925d5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:17:30.372366 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:30.372321    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b05229d-340e-424a-8533-87c77e6b97d5-original-pull-secret\") pod \"global-pull-secret-syncer-4wzj5\" (UID: \"4b05229d-340e-424a-8533-87c77e6b97d5\") " pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:30.372545 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:30.372385    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk4dj\" (UniqueName: \"kubernetes.io/projected/31fb2a24-8166-40fc-b65f-4e3bbcdba36c-kube-api-access-gk4dj\") pod \"network-check-target-mb5jg\" (UID: \"31fb2a24-8166-40fc-b65f-4e3bbcdba36c\") " pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:17:30.372545 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:30.372505    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 15:17:30.372545 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:30.372537    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 15:17:30.372716 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:30.372561    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 15:17:30.372716 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:30.372575    2560 projected.go:196] Error preparing data for projected volume kube-api-access-gk4dj for pod openshift-network-diagnostics/network-check-target-mb5jg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:17:30.372716 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:30.372585    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b05229d-340e-424a-8533-87c77e6b97d5-original-pull-secret podName:4b05229d-340e-424a-8533-87c77e6b97d5 nodeName:}" failed. No retries permitted until 2026-09-08 15:17:31.372568112 +0000 UTC m=+4.084510582 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4b05229d-340e-424a-8533-87c77e6b97d5-original-pull-secret") pod "global-pull-secret-syncer-4wzj5" (UID: "4b05229d-340e-424a-8533-87c77e6b97d5") : object "kube-system"/"original-pull-secret" not registered
Sep 08 15:17:30.372716 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:30.372635    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31fb2a24-8166-40fc-b65f-4e3bbcdba36c-kube-api-access-gk4dj podName:31fb2a24-8166-40fc-b65f-4e3bbcdba36c nodeName:}" failed. No retries permitted until 2026-09-08 15:17:31.372616319 +0000 UTC m=+4.084558808 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-gk4dj" (UniqueName: "kubernetes.io/projected/31fb2a24-8166-40fc-b65f-4e3bbcdba36c-kube-api-access-gk4dj") pod "network-check-target-mb5jg" (UID: "31fb2a24-8166-40fc-b65f-4e3bbcdba36c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:17:30.705201 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:30.705051    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 15:12:30 +0000 UTC" deadline="2028-05-09 04:46:46.984229888 +0000 UTC"
Sep 08 15:17:30.705201 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:30.705136    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14605h29m16.279098629s"
Sep 08 15:17:30.794879 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:30.794344    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:17:30.794879 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:30.794493    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dv2mt" podUID="443fd4ee-6833-4ac4-a15f-d4ebdf0925d5"
Sep 08 15:17:30.806945 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:30.806911    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb" event={"ID":"45ff8d65-b9ff-4741-91f8-d526b2ce65b3","Type":"ContainerStarted","Data":"9db5b998c709738717d4ed3d0c0783d0969fa32c109865a22a1f6ce55cfbafb3"}
Sep 08 15:17:30.809236 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:30.809203    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5kkjk" event={"ID":"08200567-2c28-4a77-83fb-c6f82cd9f42e","Type":"ContainerStarted","Data":"52008ac6b9eae260eae3f3605b3c45461d9f14a2138efb699110641cb4c8d37f"}
Sep 08 15:17:30.811801 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:30.811754    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dvlfj" event={"ID":"e26148ba-c65b-4e39-bd74-2f74c1ba38e6","Type":"ContainerStarted","Data":"9e5c6b3b1eff50a7fbf38146cb1f3481a2ec6baa5351d75478da731c3dbcec2b"}
Sep 08 15:17:30.813828 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:30.813803    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d8449" event={"ID":"4377c8d9-0724-4d65-8a61-49c2f643cb59","Type":"ContainerStarted","Data":"27792d5f48b4dabaf21ece41871a53cef3d08b7618e8d48c1b51a56ef5023df9"}
Sep 08 15:17:30.817262 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:30.817209    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q" event={"ID":"a5785ee7-6851-45c1-aa4b-ecb22455837c","Type":"ContainerStarted","Data":"713b89a5124ee532549dee4a9e72dcf85610b06473e8d03089dda1a29211da27"}
Sep 08 15:17:30.818719 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:30.818657    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g7blt" event={"ID":"762d10ee-64c5-40e1-896c-47e0cd55d613","Type":"ContainerStarted","Data":"e17abb8726d79266026474aa539143583d7f1ab11a2510997f5b7be2e5e81565"}
Sep 08 15:17:30.820342 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:30.820203    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-n7qbx" event={"ID":"2a353d7a-6ed5-475c-b1eb-3624a6773ecb","Type":"ContainerStarted","Data":"eba90fe540fe7f3f0c2166544bb283badf24fd17746595c71235baec95c58f75"}
Sep 08 15:17:30.822285 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:30.822224    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b66wt" event={"ID":"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4","Type":"ContainerStarted","Data":"e4ecf394ee01036082b0a45f6c5e47b4fea70e3fff54fdb54f6ade93cbcd2854"}
Sep 08 15:17:30.824366 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:30.824342    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jstb5" event={"ID":"202dffc4-2ade-40df-96be-4f6d20b0d998","Type":"ContainerStarted","Data":"c1291e9c710193a611384c650c92ab1cb3e7be689cbb2e9e73bb5e5a0e9acf94"}
Sep 08 15:17:31.279980 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:31.279912    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/443fd4ee-6833-4ac4-a15f-d4ebdf0925d5-metrics-certs\") pod \"network-metrics-daemon-dv2mt\" (UID: \"443fd4ee-6833-4ac4-a15f-d4ebdf0925d5\") " pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:17:31.280194 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:31.280170    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:17:31.280278 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:31.280249    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/443fd4ee-6833-4ac4-a15f-d4ebdf0925d5-metrics-certs podName:443fd4ee-6833-4ac4-a15f-d4ebdf0925d5 nodeName:}" failed. No retries permitted until 2026-09-08 15:17:33.280228586 +0000 UTC m=+5.992171063 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/443fd4ee-6833-4ac4-a15f-d4ebdf0925d5-metrics-certs") pod "network-metrics-daemon-dv2mt" (UID: "443fd4ee-6833-4ac4-a15f-d4ebdf0925d5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:17:31.380396 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:31.380356    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk4dj\" (UniqueName: \"kubernetes.io/projected/31fb2a24-8166-40fc-b65f-4e3bbcdba36c-kube-api-access-gk4dj\") pod \"network-check-target-mb5jg\" (UID: \"31fb2a24-8166-40fc-b65f-4e3bbcdba36c\") " pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:17:31.380590 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:31.380458    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b05229d-340e-424a-8533-87c77e6b97d5-original-pull-secret\") pod \"global-pull-secret-syncer-4wzj5\" (UID: \"4b05229d-340e-424a-8533-87c77e6b97d5\") " pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:31.380670 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:31.380603    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 15:17:31.380670 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:31.380667    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b05229d-340e-424a-8533-87c77e6b97d5-original-pull-secret podName:4b05229d-340e-424a-8533-87c77e6b97d5 nodeName:}" failed. No retries permitted until 2026-09-08 15:17:33.380647509 +0000 UTC m=+6.092590002 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4b05229d-340e-424a-8533-87c77e6b97d5-original-pull-secret") pod "global-pull-secret-syncer-4wzj5" (UID: "4b05229d-340e-424a-8533-87c77e6b97d5") : object "kube-system"/"original-pull-secret" not registered
Sep 08 15:17:31.380789 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:31.380759    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 15:17:31.380789 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:31.380775    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 15:17:31.380789 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:31.380788    2560 projected.go:196] Error preparing data for projected volume kube-api-access-gk4dj for pod openshift-network-diagnostics/network-check-target-mb5jg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:17:31.380951 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:31.380825    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31fb2a24-8166-40fc-b65f-4e3bbcdba36c-kube-api-access-gk4dj podName:31fb2a24-8166-40fc-b65f-4e3bbcdba36c nodeName:}" failed. No retries permitted until 2026-09-08 15:17:33.380813681 +0000 UTC m=+6.092756167 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-gk4dj" (UniqueName: "kubernetes.io/projected/31fb2a24-8166-40fc-b65f-4e3bbcdba36c-kube-api-access-gk4dj") pod "network-check-target-mb5jg" (UID: "31fb2a24-8166-40fc-b65f-4e3bbcdba36c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:17:31.705803 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:31.705703    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 15:12:30 +0000 UTC" deadline="2028-06-02 14:01:37.580370262 +0000 UTC"
Sep 08 15:17:31.705803 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:31.705746    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15190h44m5.874629137s"
Sep 08 15:17:31.794536 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:31.794505    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:31.794701 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:31.794665    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wzj5" podUID="4b05229d-340e-424a-8533-87c77e6b97d5"
Sep 08 15:17:31.794787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:31.794710    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:17:31.794893 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:31.794852    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mb5jg" podUID="31fb2a24-8166-40fc-b65f-4e3bbcdba36c"
Sep 08 15:17:32.795472 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:32.794542    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:17:32.795472 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:32.794698    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dv2mt" podUID="443fd4ee-6833-4ac4-a15f-d4ebdf0925d5"
Sep 08 15:17:33.299737 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:33.299700    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/443fd4ee-6833-4ac4-a15f-d4ebdf0925d5-metrics-certs\") pod \"network-metrics-daemon-dv2mt\" (UID: \"443fd4ee-6833-4ac4-a15f-d4ebdf0925d5\") " pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:17:33.299930 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:33.299918    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:17:33.299994 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:33.299987    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/443fd4ee-6833-4ac4-a15f-d4ebdf0925d5-metrics-certs podName:443fd4ee-6833-4ac4-a15f-d4ebdf0925d5 nodeName:}" failed. No retries permitted until 2026-09-08 15:17:37.299966326 +0000 UTC m=+10.011908829 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/443fd4ee-6833-4ac4-a15f-d4ebdf0925d5-metrics-certs") pod "network-metrics-daemon-dv2mt" (UID: "443fd4ee-6833-4ac4-a15f-d4ebdf0925d5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:17:33.401687 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:33.400843    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk4dj\" (UniqueName: \"kubernetes.io/projected/31fb2a24-8166-40fc-b65f-4e3bbcdba36c-kube-api-access-gk4dj\") pod \"network-check-target-mb5jg\" (UID: \"31fb2a24-8166-40fc-b65f-4e3bbcdba36c\") " pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:17:33.401687 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:33.400940    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b05229d-340e-424a-8533-87c77e6b97d5-original-pull-secret\") pod \"global-pull-secret-syncer-4wzj5\" (UID: \"4b05229d-340e-424a-8533-87c77e6b97d5\") " pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:33.401687 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:33.401088    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 15:17:33.401687 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:33.401170    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b05229d-340e-424a-8533-87c77e6b97d5-original-pull-secret podName:4b05229d-340e-424a-8533-87c77e6b97d5 nodeName:}" failed. No retries permitted until 2026-09-08 15:17:37.401149628 +0000 UTC m=+10.113092123 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4b05229d-340e-424a-8533-87c77e6b97d5-original-pull-secret") pod "global-pull-secret-syncer-4wzj5" (UID: "4b05229d-340e-424a-8533-87c77e6b97d5") : object "kube-system"/"original-pull-secret" not registered
Sep 08 15:17:33.401687 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:33.401262    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 15:17:33.401687 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:33.401276    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 15:17:33.401687 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:33.401290    2560 projected.go:196] Error preparing data for projected volume kube-api-access-gk4dj for pod openshift-network-diagnostics/network-check-target-mb5jg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:17:33.401687 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:33.401326    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31fb2a24-8166-40fc-b65f-4e3bbcdba36c-kube-api-access-gk4dj podName:31fb2a24-8166-40fc-b65f-4e3bbcdba36c nodeName:}" failed. No retries permitted until 2026-09-08 15:17:37.401314553 +0000 UTC m=+10.113257046 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-gk4dj" (UniqueName: "kubernetes.io/projected/31fb2a24-8166-40fc-b65f-4e3bbcdba36c-kube-api-access-gk4dj") pod "network-check-target-mb5jg" (UID: "31fb2a24-8166-40fc-b65f-4e3bbcdba36c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:17:33.797024 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:33.796433    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:17:33.797024 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:33.796573    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mb5jg" podUID="31fb2a24-8166-40fc-b65f-4e3bbcdba36c"
Sep 08 15:17:33.797024 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:33.796965    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:33.797628 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:33.797082    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wzj5" podUID="4b05229d-340e-424a-8533-87c77e6b97d5"
Sep 08 15:17:34.793802 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:34.793762    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:17:34.793979 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:34.793914    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dv2mt" podUID="443fd4ee-6833-4ac4-a15f-d4ebdf0925d5"
Sep 08 15:17:35.793948 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:35.793914    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:17:35.794461 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:35.794042    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mb5jg" podUID="31fb2a24-8166-40fc-b65f-4e3bbcdba36c"
Sep 08 15:17:35.794537 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:35.794491    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:35.794598 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:35.794577    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wzj5" podUID="4b05229d-340e-424a-8533-87c77e6b97d5"
Sep 08 15:17:36.794624 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:36.794544    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:17:36.795156 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:36.794690    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dv2mt" podUID="443fd4ee-6833-4ac4-a15f-d4ebdf0925d5"
Sep 08 15:17:37.337950 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:37.337912    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/443fd4ee-6833-4ac4-a15f-d4ebdf0925d5-metrics-certs\") pod \"network-metrics-daemon-dv2mt\" (UID: \"443fd4ee-6833-4ac4-a15f-d4ebdf0925d5\") " pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:17:37.338166 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:37.338107    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:17:37.338245 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:37.338178    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/443fd4ee-6833-4ac4-a15f-d4ebdf0925d5-metrics-certs podName:443fd4ee-6833-4ac4-a15f-d4ebdf0925d5 nodeName:}" failed. No retries permitted until 2026-09-08 15:17:45.338155805 +0000 UTC m=+18.050098290 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/443fd4ee-6833-4ac4-a15f-d4ebdf0925d5-metrics-certs") pod "network-metrics-daemon-dv2mt" (UID: "443fd4ee-6833-4ac4-a15f-d4ebdf0925d5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:17:37.439084 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:37.439034    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b05229d-340e-424a-8533-87c77e6b97d5-original-pull-secret\") pod \"global-pull-secret-syncer-4wzj5\" (UID: \"4b05229d-340e-424a-8533-87c77e6b97d5\") " pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:37.439252 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:37.439112    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk4dj\" (UniqueName: \"kubernetes.io/projected/31fb2a24-8166-40fc-b65f-4e3bbcdba36c-kube-api-access-gk4dj\") pod \"network-check-target-mb5jg\" (UID: \"31fb2a24-8166-40fc-b65f-4e3bbcdba36c\") " pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:17:37.439252 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:37.439206    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 15:17:37.439377 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:37.439275    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b05229d-340e-424a-8533-87c77e6b97d5-original-pull-secret podName:4b05229d-340e-424a-8533-87c77e6b97d5 nodeName:}" failed. No retries permitted until 2026-09-08 15:17:45.439255253 +0000 UTC m=+18.151197725 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4b05229d-340e-424a-8533-87c77e6b97d5-original-pull-secret") pod "global-pull-secret-syncer-4wzj5" (UID: "4b05229d-340e-424a-8533-87c77e6b97d5") : object "kube-system"/"original-pull-secret" not registered
Sep 08 15:17:37.439377 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:37.439295    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 15:17:37.439377 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:37.439319    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 15:17:37.439377 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:37.439334    2560 projected.go:196] Error preparing data for projected volume kube-api-access-gk4dj for pod openshift-network-diagnostics/network-check-target-mb5jg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:17:37.439604 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:37.439386    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31fb2a24-8166-40fc-b65f-4e3bbcdba36c-kube-api-access-gk4dj podName:31fb2a24-8166-40fc-b65f-4e3bbcdba36c nodeName:}" failed. No retries permitted until 2026-09-08 15:17:45.439367287 +0000 UTC m=+18.151309773 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-gk4dj" (UniqueName: "kubernetes.io/projected/31fb2a24-8166-40fc-b65f-4e3bbcdba36c-kube-api-access-gk4dj") pod "network-check-target-mb5jg" (UID: "31fb2a24-8166-40fc-b65f-4e3bbcdba36c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:17:37.795694 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:37.794859    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:37.795694 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:37.795016    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wzj5" podUID="4b05229d-340e-424a-8533-87c77e6b97d5"
Sep 08 15:17:37.795694 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:37.795491    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:17:37.795694 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:37.795591    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mb5jg" podUID="31fb2a24-8166-40fc-b65f-4e3bbcdba36c"
Sep 08 15:17:38.794420 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:38.794378    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:17:38.794608 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:38.794545    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dv2mt" podUID="443fd4ee-6833-4ac4-a15f-d4ebdf0925d5"
Sep 08 15:17:39.794241 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:39.794206    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:17:39.794701 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:39.794337    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mb5jg" podUID="31fb2a24-8166-40fc-b65f-4e3bbcdba36c"
Sep 08 15:17:39.794984 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:39.794804    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:39.794984 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:39.794918    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wzj5" podUID="4b05229d-340e-424a-8533-87c77e6b97d5"
Sep 08 15:17:40.793695 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:40.793663    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:17:40.793877 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:40.793798    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dv2mt" podUID="443fd4ee-6833-4ac4-a15f-d4ebdf0925d5"
Sep 08 15:17:41.794144 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:41.794002    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:41.794144 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:41.794016    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:17:41.794626 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:41.794141    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wzj5" podUID="4b05229d-340e-424a-8533-87c77e6b97d5"
Sep 08 15:17:41.794626 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:41.794218    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mb5jg" podUID="31fb2a24-8166-40fc-b65f-4e3bbcdba36c"
Sep 08 15:17:42.794118 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:42.794085    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:17:42.794312 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:42.794212    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dv2mt" podUID="443fd4ee-6833-4ac4-a15f-d4ebdf0925d5"
Sep 08 15:17:43.793831 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:43.793796    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:43.794003 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:43.793929    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wzj5" podUID="4b05229d-340e-424a-8533-87c77e6b97d5"
Sep 08 15:17:43.794003 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:43.793951    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:17:43.794155 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:43.794030    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mb5jg" podUID="31fb2a24-8166-40fc-b65f-4e3bbcdba36c"
Sep 08 15:17:44.793785 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:44.793754    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:17:44.794244 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:44.793854    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dv2mt" podUID="443fd4ee-6833-4ac4-a15f-d4ebdf0925d5"
Sep 08 15:17:45.399547 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:45.399509    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/443fd4ee-6833-4ac4-a15f-d4ebdf0925d5-metrics-certs\") pod \"network-metrics-daemon-dv2mt\" (UID: \"443fd4ee-6833-4ac4-a15f-d4ebdf0925d5\") " pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:17:45.399707 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:45.399685    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:17:45.399772 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:45.399761    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/443fd4ee-6833-4ac4-a15f-d4ebdf0925d5-metrics-certs podName:443fd4ee-6833-4ac4-a15f-d4ebdf0925d5 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:01.399740712 +0000 UTC m=+34.111683204 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/443fd4ee-6833-4ac4-a15f-d4ebdf0925d5-metrics-certs") pod "network-metrics-daemon-dv2mt" (UID: "443fd4ee-6833-4ac4-a15f-d4ebdf0925d5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:17:45.500716 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:45.500680    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b05229d-340e-424a-8533-87c77e6b97d5-original-pull-secret\") pod \"global-pull-secret-syncer-4wzj5\" (UID: \"4b05229d-340e-424a-8533-87c77e6b97d5\") " pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:45.500716 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:45.500731    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk4dj\" (UniqueName: \"kubernetes.io/projected/31fb2a24-8166-40fc-b65f-4e3bbcdba36c-kube-api-access-gk4dj\") pod \"network-check-target-mb5jg\" (UID: \"31fb2a24-8166-40fc-b65f-4e3bbcdba36c\") " pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:17:45.500940 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:45.500846    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 15:17:45.500940 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:45.500923    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b05229d-340e-424a-8533-87c77e6b97d5-original-pull-secret podName:4b05229d-340e-424a-8533-87c77e6b97d5 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:01.500901455 +0000 UTC m=+34.212843946 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4b05229d-340e-424a-8533-87c77e6b97d5-original-pull-secret") pod "global-pull-secret-syncer-4wzj5" (UID: "4b05229d-340e-424a-8533-87c77e6b97d5") : object "kube-system"/"original-pull-secret" not registered
Sep 08 15:17:45.500940 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:45.500855    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 15:17:45.501143 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:45.500960    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 15:17:45.501143 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:45.500974    2560 projected.go:196] Error preparing data for projected volume kube-api-access-gk4dj for pod openshift-network-diagnostics/network-check-target-mb5jg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:17:45.501143 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:45.501039    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31fb2a24-8166-40fc-b65f-4e3bbcdba36c-kube-api-access-gk4dj podName:31fb2a24-8166-40fc-b65f-4e3bbcdba36c nodeName:}" failed. No retries permitted until 2026-09-08 15:18:01.501024551 +0000 UTC m=+34.212967020 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-gk4dj" (UniqueName: "kubernetes.io/projected/31fb2a24-8166-40fc-b65f-4e3bbcdba36c-kube-api-access-gk4dj") pod "network-check-target-mb5jg" (UID: "31fb2a24-8166-40fc-b65f-4e3bbcdba36c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:17:45.794365 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:45.794334    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:17:45.794883 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:45.794457    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mb5jg" podUID="31fb2a24-8166-40fc-b65f-4e3bbcdba36c"
Sep 08 15:17:45.794883 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:45.794481    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:45.794883 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:45.794602    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wzj5" podUID="4b05229d-340e-424a-8533-87c77e6b97d5"
Sep 08 15:17:46.793901 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:46.793868    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:17:46.794092 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:46.793999    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dv2mt" podUID="443fd4ee-6833-4ac4-a15f-d4ebdf0925d5"
Sep 08 15:17:47.795010 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:47.794970    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:17:47.795462 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:47.795097    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:47.795462 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:47.795111    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mb5jg" podUID="31fb2a24-8166-40fc-b65f-4e3bbcdba36c"
Sep 08 15:17:47.795462 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:47.795206    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wzj5" podUID="4b05229d-340e-424a-8533-87c77e6b97d5"
Sep 08 15:17:48.793882 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:48.793860    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:17:48.794014 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:48.793991    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dv2mt" podUID="443fd4ee-6833-4ac4-a15f-d4ebdf0925d5"
Sep 08 15:17:48.861438 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:48.861224    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d8449" event={"ID":"4377c8d9-0724-4d65-8a61-49c2f643cb59","Type":"ContainerStarted","Data":"f83f11169043885cbd31ca2b1ff84162f977a6758db7823f608cd5abddfc9173"}
Sep 08 15:17:48.865793 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:48.864460    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-43.ec2.internal" event={"ID":"f7c1882d19ab40f24851c6d6b5b879ef","Type":"ContainerStarted","Data":"bd51a8910c65f4ed2e87397d70822ad19254569daa64dcfd83671a5199a23a5a"}
Sep 08 15:17:48.868729 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:48.867260    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b66wt" event={"ID":"51a7da58-d825-4c5d-9cf8-16a7a1b99cc4","Type":"ContainerStarted","Data":"af8123a723bdae35e5ad04b4346740abd43f3de783a1a3f94583c166a36637b3"}
Sep 08 15:17:48.870203 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:48.870178    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb" event={"ID":"45ff8d65-b9ff-4741-91f8-d526b2ce65b3","Type":"ContainerStarted","Data":"98c2a58d465077f091f4abc2b75df3b4cfb9e49b7eac9ff311c69aec26c6d7c8"}
Sep 08 15:17:48.870296 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:48.870258    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb" event={"ID":"45ff8d65-b9ff-4741-91f8-d526b2ce65b3","Type":"ContainerStarted","Data":"52a85930040aec9eda661ffe11a7bfc7a5699d885ee22bb5bf38563886fe6f88"}
Sep 08 15:17:48.870352 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:48.870312    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb" event={"ID":"45ff8d65-b9ff-4741-91f8-d526b2ce65b3","Type":"ContainerStarted","Data":"06b5ad41281bfd6d5feb95aecbc2422d23fcd2cc128c1b1625835f4570906c15"}
Sep 08 15:17:48.870402 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:48.870362    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb" event={"ID":"45ff8d65-b9ff-4741-91f8-d526b2ce65b3","Type":"ContainerStarted","Data":"de115f9b647eab656e1bcc228f358869d1e1ce5e875c472619d13008a2bc48af"}
Sep 08 15:17:48.939355 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:48.939304    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-b66wt" podStartSLOduration=5.12003277 podStartE2EDuration="19.939286806s" totalImagesPullingTime="14.819254036s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:17:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:17:48.939253882 +0000 UTC m=+21.651196373" watchObservedRunningTime="2026-09-08 15:17:48.939286806 +0000 UTC m=+21.651229299"
Sep 08 15:17:48.939699 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:48.939672    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-d8449" podStartSLOduration=1.4240997819999999 podStartE2EDuration="19.939664915s" totalImagesPullingTime="18.515565133s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:17:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:17:48.907541154 +0000 UTC m=+21.619483645" watchObservedRunningTime="2026-09-08 15:17:48.939664915 +0000 UTC m=+21.651607407"
Sep 08 15:17:49.794757 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:49.794712    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:17:49.794967 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:49.794713    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:49.794967 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:49.794851    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mb5jg" podUID="31fb2a24-8166-40fc-b65f-4e3bbcdba36c"
Sep 08 15:17:49.794967 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:49.794902    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wzj5" podUID="4b05229d-340e-424a-8533-87c77e6b97d5"
Sep 08 15:17:49.873012 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:49.872975    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dvlfj" event={"ID":"e26148ba-c65b-4e39-bd74-2f74c1ba38e6","Type":"ContainerStarted","Data":"19f12ee3311d3b3bb0caedb11e8d118dacc54691fc11d5a2861a2aa9bf7095be"}
Sep 08 15:17:49.874394 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:49.874372    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q" event={"ID":"a5785ee7-6851-45c1-aa4b-ecb22455837c","Type":"ContainerStarted","Data":"787293f00d3620fca5fb235fb0871731a234ea370ce25d681bc6be6dedb288e7"}
Sep 08 15:17:49.875665 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:49.875642    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g7blt" event={"ID":"762d10ee-64c5-40e1-896c-47e0cd55d613","Type":"ContainerStarted","Data":"82e48d131332d044d106c2ad462d5128a0ae3b4a09ef5f0f317fe1a36b0cfb9e"}
Sep 08 15:17:49.876902 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:49.876880    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-n7qbx" event={"ID":"2a353d7a-6ed5-475c-b1eb-3624a6773ecb","Type":"ContainerStarted","Data":"44bb7ee14e2b1a7e2817256f228ef8d8effa7529f3201343534af5f62984a583"}
Sep 08 15:17:49.878520 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:49.878495    2560 generic.go:356] "Generic (PLEG): container finished" podID="202dffc4-2ade-40df-96be-4f6d20b0d998" containerID="2749a1e0f7b2f380acd6bbe5634dba3bb6115ad1ab83dc5e7f0c04f8ea1dcac2" exitCode=0
Sep 08 15:17:49.878618 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:49.878566    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jstb5" event={"ID":"202dffc4-2ade-40df-96be-4f6d20b0d998","Type":"ContainerDied","Data":"2749a1e0f7b2f380acd6bbe5634dba3bb6115ad1ab83dc5e7f0c04f8ea1dcac2"}
Sep 08 15:17:49.882089 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:49.882055    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb" event={"ID":"45ff8d65-b9ff-4741-91f8-d526b2ce65b3","Type":"ContainerStarted","Data":"d0c80bc0f75635526cf6712a38ccb80a34ee081b5d38991ddcb3fc671789613c"}
Sep 08 15:17:49.882167 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:49.882126    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb" event={"ID":"45ff8d65-b9ff-4741-91f8-d526b2ce65b3","Type":"ContainerStarted","Data":"0e3f76df5e200082e0a518567c2f8f76f96053b6d89da7c458cb0c230f7bb612"}
Sep 08 15:17:49.883715 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:49.883688    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5kkjk" event={"ID":"08200567-2c28-4a77-83fb-c6f82cd9f42e","Type":"ContainerStarted","Data":"6d5d7853b49405b865f058fb98db285e98dc180ad0971d26b14262672e89d281"}
Sep 08 15:17:49.885385 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:49.885363    2560 generic.go:356] "Generic (PLEG): container finished" podID="613a137b636df8e8d8c322049cd80abe" containerID="4915f912a06c1435f87ed2084fbe0a221236ea27f73c8e487734f41fca958942" exitCode=0
Sep 08 15:17:49.885474 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:49.885447    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-43.ec2.internal" event={"ID":"613a137b636df8e8d8c322049cd80abe","Type":"ContainerDied","Data":"4915f912a06c1435f87ed2084fbe0a221236ea27f73c8e487734f41fca958942"}
Sep 08 15:17:49.905427 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:49.905366    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-43.ec2.internal" podStartSLOduration=20.905344482 podStartE2EDuration="20.905344482s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:17:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:17:48.973989936 +0000 UTC m=+21.685932428" watchObservedRunningTime="2026-09-08 15:17:49.905344482 +0000 UTC m=+22.617286976"
Sep 08 15:17:49.956297 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:49.956243    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-dvlfj" podStartSLOduration=2.497933206 podStartE2EDuration="20.956220661s" totalImagesPullingTime="18.458287455s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:17:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:17:49.905839431 +0000 UTC m=+22.617781923" watchObservedRunningTime="2026-09-08 15:17:49.956220661 +0000 UTC m=+22.668163153"
Sep 08 15:17:49.984203 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:49.983684    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-g7blt" podStartSLOduration=10.552066509 podStartE2EDuration="21.983668499s" totalImagesPullingTime="11.43160199s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:17:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:17:49.983636932 +0000 UTC m=+22.695579423" watchObservedRunningTime="2026-09-08 15:17:49.983668499 +0000 UTC m=+22.695611002"
Sep 08 15:17:50.011084 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:50.010982    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-n7qbx" podStartSLOduration=3.508640523 podStartE2EDuration="22.010961363s" totalImagesPullingTime="18.50232084s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:17:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:17:50.010491065 +0000 UTC m=+22.722433566" watchObservedRunningTime="2026-09-08 15:17:50.010961363 +0000 UTC m=+22.722903853"
Sep 08 15:17:50.030344 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:50.030284    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5kkjk" podStartSLOduration=3.532846327 podStartE2EDuration="22.030262716s" totalImagesPullingTime="18.497416389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:17:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:17:50.029566557 +0000 UTC m=+22.741509048" watchObservedRunningTime="2026-09-08 15:17:50.030262716 +0000 UTC m=+22.742205208"
Sep 08 15:17:50.260736 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:50.260713    2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 08 15:17:50.404235 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:50.404154    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5kkjk"
Sep 08 15:17:50.404760 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:50.404745    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5kkjk"
Sep 08 15:17:50.712910 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:50.712761    2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-08T15:17:50.260729549Z","UUID":"3bff9f24-1bea-45ba-bb81-08f8ba5854c0","Handler":null,"Name":"","Endpoint":""}
Sep 08 15:17:50.714109 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:50.714093    2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 08 15:17:50.714186 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:50.714121    2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 08 15:17:50.794433 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:50.794403    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:17:50.794586 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:50.794492    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dv2mt" podUID="443fd4ee-6833-4ac4-a15f-d4ebdf0925d5"
Sep 08 15:17:50.889112 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:50.889065    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q" event={"ID":"a5785ee7-6851-45c1-aa4b-ecb22455837c","Type":"ContainerStarted","Data":"89c0ed15ae118f1e0160e982527ee456fa66ae42b2500bb9c5372033fa332b30"}
Sep 08 15:17:50.890826 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:50.890799    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-43.ec2.internal" event={"ID":"613a137b636df8e8d8c322049cd80abe","Type":"ContainerStarted","Data":"05c54df734203abf9078e0e2b5a2e8294b36e333d68776818b8088d0c7b37b6e"}
Sep 08 15:17:50.891397 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:50.891363    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5kkjk"
Sep 08 15:17:50.891674 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:50.891659    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5kkjk"
Sep 08 15:17:50.946891 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:50.946848    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-43.ec2.internal" podStartSLOduration=21.946833335 podStartE2EDuration="21.946833335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:17:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:17:50.919334238 +0000 UTC m=+23.631276741" watchObservedRunningTime="2026-09-08 15:17:50.946833335 +0000 UTC m=+23.658775825"
Sep 08 15:17:51.794084 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:51.794041    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:51.794213 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:51.794097    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:17:51.794213 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:51.794195    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wzj5" podUID="4b05229d-340e-424a-8533-87c77e6b97d5"
Sep 08 15:17:51.794348 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:51.794324    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mb5jg" podUID="31fb2a24-8166-40fc-b65f-4e3bbcdba36c"
Sep 08 15:17:51.896242 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:51.896205    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q" event={"ID":"a5785ee7-6851-45c1-aa4b-ecb22455837c","Type":"ContainerStarted","Data":"bed9e290293123b8c90e0ad64a9621a3d7e151e5fe2621b5d1b40f4890112da8"}
Sep 08 15:17:51.900426 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:51.900390    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb" event={"ID":"45ff8d65-b9ff-4741-91f8-d526b2ce65b3","Type":"ContainerStarted","Data":"1933295ac095e8131376bd6f3c3724ed1dbf3c8835227293958684a381ee8fba"}
Sep 08 15:17:51.917341 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:51.917280    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pd26q" podStartSLOduration=1.740789075 podStartE2EDuration="22.917257538s" totalImagesPullingTime="21.176468463s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:17:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:17:51.916703875 +0000 UTC m=+24.628646365" watchObservedRunningTime="2026-09-08 15:17:51.917257538 +0000 UTC m=+24.629200027"
Sep 08 15:17:52.794233 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:52.794203    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:17:52.794449 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:52.794334    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dv2mt" podUID="443fd4ee-6833-4ac4-a15f-d4ebdf0925d5"
Sep 08 15:17:53.793858 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:53.793824    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:53.794426 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:53.793867    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:17:53.794426 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:53.793959    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wzj5" podUID="4b05229d-340e-424a-8533-87c77e6b97d5"
Sep 08 15:17:53.794426 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:53.794101    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mb5jg" podUID="31fb2a24-8166-40fc-b65f-4e3bbcdba36c"
Sep 08 15:17:53.910261 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:53.910221    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb" event={"ID":"45ff8d65-b9ff-4741-91f8-d526b2ce65b3","Type":"ContainerStarted","Data":"33ecdd9d9d01ba22047e179235215e394590a27f1409a16484afa2faf5a81f48"}
Sep 08 15:17:53.910593 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:53.910507    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:53.927642 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:53.927561    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:53.958170 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:53.958120    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb" podStartSLOduration=7.423858081 podStartE2EDuration="25.958103214s" totalImagesPullingTime="18.534245133s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:17:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:17:53.957821959 +0000 UTC m=+26.669764475" watchObservedRunningTime="2026-09-08 15:17:53.958103214 +0000 UTC m=+26.670045707"
Sep 08 15:17:54.794813 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:54.794615    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:17:54.795243 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:54.794919    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dv2mt" podUID="443fd4ee-6833-4ac4-a15f-d4ebdf0925d5"
Sep 08 15:17:54.913756 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:54.913721    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:54.913921 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:54.913817    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:54.930359 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:54.930332    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:17:55.375386 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:55.375342    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4wzj5"]
Sep 08 15:17:55.375576 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:55.375455    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:55.375576 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:55.375563    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wzj5" podUID="4b05229d-340e-424a-8533-87c77e6b97d5"
Sep 08 15:17:55.377685 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:55.377627    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dv2mt"]
Sep 08 15:17:55.377806 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:55.377724    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:17:55.377860 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:55.377820    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dv2mt" podUID="443fd4ee-6833-4ac4-a15f-d4ebdf0925d5"
Sep 08 15:17:55.395366 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:55.395339    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mb5jg"]
Sep 08 15:17:55.395500 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:55.395427    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:17:55.395557 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:55.395520    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mb5jg" podUID="31fb2a24-8166-40fc-b65f-4e3bbcdba36c"
Sep 08 15:17:56.793889 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:56.793858    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:56.794387 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:56.793965    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wzj5" podUID="4b05229d-340e-424a-8533-87c77e6b97d5"
Sep 08 15:17:56.919216 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:56.919184    2560 generic.go:356] "Generic (PLEG): container finished" podID="202dffc4-2ade-40df-96be-4f6d20b0d998" containerID="6d4bc1838ddb47e7615722521d7d1fe721ed57847dc9219317fa8db3aa2859d9" exitCode=0
Sep 08 15:17:56.919380 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:56.919270    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jstb5" event={"ID":"202dffc4-2ade-40df-96be-4f6d20b0d998","Type":"ContainerDied","Data":"6d4bc1838ddb47e7615722521d7d1fe721ed57847dc9219317fa8db3aa2859d9"}
Sep 08 15:17:57.794937 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:57.794906    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:17:57.795353 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:57.794989    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dv2mt" podUID="443fd4ee-6833-4ac4-a15f-d4ebdf0925d5"
Sep 08 15:17:57.795353 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:57.795029    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:17:57.795353 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:57.795093    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mb5jg" podUID="31fb2a24-8166-40fc-b65f-4e3bbcdba36c"
Sep 08 15:17:57.926313 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:57.926279    2560 generic.go:356] "Generic (PLEG): container finished" podID="202dffc4-2ade-40df-96be-4f6d20b0d998" containerID="3f221822ead1822902aa3c03df69a101a730f31f500d41c29017d991220693a6" exitCode=0
Sep 08 15:17:57.926474 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:57.926331    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jstb5" event={"ID":"202dffc4-2ade-40df-96be-4f6d20b0d998","Type":"ContainerDied","Data":"3f221822ead1822902aa3c03df69a101a730f31f500d41c29017d991220693a6"}
Sep 08 15:17:58.794270 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:58.794235    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:17:58.794409 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:58.794342    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wzj5" podUID="4b05229d-340e-424a-8533-87c77e6b97d5"
Sep 08 15:17:59.794178 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:59.794145    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:17:59.794642 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:59.794150    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:17:59.794642 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:59.794259    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dv2mt" podUID="443fd4ee-6833-4ac4-a15f-d4ebdf0925d5"
Sep 08 15:17:59.794642 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:17:59.794320    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mb5jg" podUID="31fb2a24-8166-40fc-b65f-4e3bbcdba36c"
Sep 08 15:17:59.933659 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:59.933623    2560 generic.go:356] "Generic (PLEG): container finished" podID="202dffc4-2ade-40df-96be-4f6d20b0d998" containerID="a2b9176684509da321a917236bb6cb2014457094a1048edf4209c0fa03819763" exitCode=0
Sep 08 15:17:59.933814 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:17:59.933663    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jstb5" event={"ID":"202dffc4-2ade-40df-96be-4f6d20b0d998","Type":"ContainerDied","Data":"a2b9176684509da321a917236bb6cb2014457094a1048edf4209c0fa03819763"}
Sep 08 15:18:00.793663 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:00.793632    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:18:00.793819 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:00.793797    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wzj5" podUID="4b05229d-340e-424a-8533-87c77e6b97d5"
Sep 08 15:18:01.414679 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.414639    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/443fd4ee-6833-4ac4-a15f-d4ebdf0925d5-metrics-certs\") pod \"network-metrics-daemon-dv2mt\" (UID: \"443fd4ee-6833-4ac4-a15f-d4ebdf0925d5\") " pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:18:01.415056 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:01.414827    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:18:01.415056 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:01.414913    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/443fd4ee-6833-4ac4-a15f-d4ebdf0925d5-metrics-certs podName:443fd4ee-6833-4ac4-a15f-d4ebdf0925d5 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:33.414892186 +0000 UTC m=+66.126834661 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/443fd4ee-6833-4ac4-a15f-d4ebdf0925d5-metrics-certs") pod "network-metrics-daemon-dv2mt" (UID: "443fd4ee-6833-4ac4-a15f-d4ebdf0925d5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:18:01.515638 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.515541    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b05229d-340e-424a-8533-87c77e6b97d5-original-pull-secret\") pod \"global-pull-secret-syncer-4wzj5\" (UID: \"4b05229d-340e-424a-8533-87c77e6b97d5\") " pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:18:01.515638 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.515614    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk4dj\" (UniqueName: \"kubernetes.io/projected/31fb2a24-8166-40fc-b65f-4e3bbcdba36c-kube-api-access-gk4dj\") pod \"network-check-target-mb5jg\" (UID: \"31fb2a24-8166-40fc-b65f-4e3bbcdba36c\") " pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:18:01.515855 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:01.515710    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 15:18:01.515855 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:01.515777    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b05229d-340e-424a-8533-87c77e6b97d5-original-pull-secret podName:4b05229d-340e-424a-8533-87c77e6b97d5 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:33.515762432 +0000 UTC m=+66.227704901 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4b05229d-340e-424a-8533-87c77e6b97d5-original-pull-secret") pod "global-pull-secret-syncer-4wzj5" (UID: "4b05229d-340e-424a-8533-87c77e6b97d5") : object "kube-system"/"original-pull-secret" not registered
Sep 08 15:18:01.515855 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:01.515813    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 15:18:01.515855 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:01.515841    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 15:18:01.515855 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:01.515856    2560 projected.go:196] Error preparing data for projected volume kube-api-access-gk4dj for pod openshift-network-diagnostics/network-check-target-mb5jg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:18:01.516089 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:01.515913    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31fb2a24-8166-40fc-b65f-4e3bbcdba36c-kube-api-access-gk4dj podName:31fb2a24-8166-40fc-b65f-4e3bbcdba36c nodeName:}" failed. No retries permitted until 2026-09-08 15:18:33.515896746 +0000 UTC m=+66.227839232 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-gk4dj" (UniqueName: "kubernetes.io/projected/31fb2a24-8166-40fc-b65f-4e3bbcdba36c-kube-api-access-gk4dj") pod "network-check-target-mb5jg" (UID: "31fb2a24-8166-40fc-b65f-4e3bbcdba36c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:18:01.541369 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.541337    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-43.ec2.internal" event="NodeReady"
Sep 08 15:18:01.541554 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.541497    2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 08 15:18:01.598450 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.598200    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-64f7cfd7bb-88t2j"]
Sep 08 15:18:01.601370 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.601330    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-z6gr5"]
Sep 08 15:18:01.601523 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.601500    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:01.603845 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.603763    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-km8ph"]
Sep 08 15:18:01.604016 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.603995    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-z6gr5"
Sep 08 15:18:01.605420 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.605396    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8dlwl"]
Sep 08 15:18:01.605584 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.605559    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-km8ph"
Sep 08 15:18:01.607555 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.607538    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-vc5n9"]
Sep 08 15:18:01.607663 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.607652    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8dlwl"
Sep 08 15:18:01.610909 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.610849    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-857nk"]
Sep 08 15:18:01.610985 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.610970    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-vc5n9"
Sep 08 15:18:01.613578 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.613559    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-dkwzw"]
Sep 08 15:18:01.613762 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.613731    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-857nk"
Sep 08 15:18:01.616960 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.616943    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-88497859b-8nkdj"]
Sep 08 15:18:01.617418 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.617097    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-dkwzw"
Sep 08 15:18:01.619705 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.619675    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:01.623989 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.623629    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 08 15:18:01.623989 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.623772    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 08 15:18:01.623989 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.623822    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 08 15:18:01.623989 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.623852    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 08 15:18:01.623989 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.623896    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 08 15:18:01.623989 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.623773    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 08 15:18:01.624648 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.624518    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 08 15:18:01.624648 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.624614    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 08 15:18:01.624784 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.624670    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-smd6z\""
Sep 08 15:18:01.624837 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.624784    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 08 15:18:01.624837 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.624795    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 08 15:18:01.625772 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.625146    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-s5fmp\""
Sep 08 15:18:01.625772 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.625170    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 08 15:18:01.625772 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.625195    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-nxz7s\""
Sep 08 15:18:01.625772 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.625359    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 08 15:18:01.625772 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.625388    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 08 15:18:01.625772 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.625441    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 08 15:18:01.625772 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.625494    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-dx6gs\""
Sep 08 15:18:01.625772 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.625519    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-2qq58\""
Sep 08 15:18:01.625772 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.625577    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 08 15:18:01.625772 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.625626    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 08 15:18:01.625772 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.625640    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 08 15:18:01.625772 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.625690    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 08 15:18:01.625772 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.625737    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 08 15:18:01.626654 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.625832    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 08 15:18:01.626654 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.626016    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 08 15:18:01.626654 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.626543    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-htbkw\""
Sep 08 15:18:01.626897 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.626705    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 08 15:18:01.626897 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.626750    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 08 15:18:01.626999 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.626959    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-m7sfd\""
Sep 08 15:18:01.630553 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.630535    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-pvz7x"]
Sep 08 15:18:01.631173 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.631153    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 08 15:18:01.635538 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.635379    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-f67sn"]
Sep 08 15:18:01.638632 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.638613    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pvz7x"
Sep 08 15:18:01.639862 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.639838    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 08 15:18:01.640301 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.640062    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 08 15:18:01.642593 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.642559    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64f7cfd7bb-88t2j"]
Sep 08 15:18:01.642700 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.642668    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-f67sn"
Sep 08 15:18:01.644439 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.644421    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 08 15:18:01.644522 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.644457    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 08 15:18:01.644588 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.644575    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 08 15:18:01.644715 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.644697    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 08 15:18:01.644715 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.644709    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 08 15:18:01.644842 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.644827    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-tmffq\""
Sep 08 15:18:01.644954 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.644933    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 08 15:18:01.647487 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.647468    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 08 15:18:01.649212 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.649192    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-542kt\""
Sep 08 15:18:01.650087 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.649911    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 08 15:18:01.650846 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.650598    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 08 15:18:01.658739 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.658717    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-z6gr5"]
Sep 08 15:18:01.665725 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.665707    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 08 15:18:01.665882 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.665864    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 08 15:18:01.666215 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.665813    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 08 15:18:01.667188 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.667168    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-km8ph"]
Sep 08 15:18:01.668042 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.668024    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-857nk"]
Sep 08 15:18:01.668829 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.668807    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-vc5n9"]
Sep 08 15:18:01.669137 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.669118    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 08 15:18:01.669531 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.669509    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-z2r89"]
Sep 08 15:18:01.673343 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.673327    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z2r89"
Sep 08 15:18:01.675780 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.675762    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-pvz7x"]
Sep 08 15:18:01.676679 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.676659    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-88497859b-8nkdj"]
Sep 08 15:18:01.679809 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.679788    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 08 15:18:01.680122 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.680103    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 08 15:18:01.680357 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.680336    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-dkwzw"]
Sep 08 15:18:01.682347 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.682330    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 08 15:18:01.682460 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.682439    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9zz4p\""
Sep 08 15:18:01.682661 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.682644    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-6k88g\""
Sep 08 15:18:01.683319 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.683298    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ct4vs"]
Sep 08 15:18:01.687579 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.687387    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ct4vs"
Sep 08 15:18:01.689898 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.689862    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-65z9v\""
Sep 08 15:18:01.690258 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.690201    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-f67sn"]
Sep 08 15:18:01.690724 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.690521    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 08 15:18:01.690724 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.690666    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 08 15:18:01.692130 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.692113    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8dlwl"]
Sep 08 15:18:01.708691 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.708667    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z2r89"]
Sep 08 15:18:01.709561 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.709537    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ct4vs"]
Sep 08 15:18:01.717130 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.717103    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5551e120-e4db-4d93-8d2a-9e3626db076e-metrics-certs\") pod \"router-default-88497859b-8nkdj\" (UID: \"5551e120-e4db-4d93-8d2a-9e3626db076e\") " pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:01.717254 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.717147    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn7c9\" (UniqueName: \"kubernetes.io/projected/17e30810-dfd0-4e6f-9204-731d1a6479a3-kube-api-access-sn7c9\") pod \"console-operator-85b454874b-vc5n9\" (UID: \"17e30810-dfd0-4e6f-9204-731d1a6479a3\") " pod="openshift-console-operator/console-operator-85b454874b-vc5n9"
Sep 08 15:18:01.717254 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.717173    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxv49\" (UniqueName: \"kubernetes.io/projected/5551e120-e4db-4d93-8d2a-9e3626db076e-kube-api-access-kxv49\") pod \"router-default-88497859b-8nkdj\" (UID: \"5551e120-e4db-4d93-8d2a-9e3626db076e\") " pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:01.717254 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.717234    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d4a10483-c29e-4a1c-9eb8-79d3197c1e35-serving-cert\") pod \"insights-operator-76bcd6ccf5-dkwzw\" (UID: \"d4a10483-c29e-4a1c-9eb8-79d3197c1e35\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dkwzw"
Sep 08 15:18:01.717400 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.717298    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wksks\" (UniqueName: \"kubernetes.io/projected/590ddb7c-a30e-4ad5-9803-7ae4b2267d66-kube-api-access-wksks\") pod \"cluster-samples-operator-7cdcfcf76c-z6gr5\" (UID: \"590ddb7c-a30e-4ad5-9803-7ae4b2267d66\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-z6gr5"
Sep 08 15:18:01.717400 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.717336    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4a10483-c29e-4a1c-9eb8-79d3197c1e35-tmp\") pod \"insights-operator-76bcd6ccf5-dkwzw\" (UID: \"d4a10483-c29e-4a1c-9eb8-79d3197c1e35\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dkwzw"
Sep 08 15:18:01.717400 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.717388    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/17e30810-dfd0-4e6f-9204-731d1a6479a3-trusted-ca\") pod \"console-operator-85b454874b-vc5n9\" (UID: \"17e30810-dfd0-4e6f-9204-731d1a6479a3\") " pod="openshift-console-operator/console-operator-85b454874b-vc5n9"
Sep 08 15:18:01.717511 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.717417    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/590ddb7c-a30e-4ad5-9803-7ae4b2267d66-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-z6gr5\" (UID: \"590ddb7c-a30e-4ad5-9803-7ae4b2267d66\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-z6gr5"
Sep 08 15:18:01.717511 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.717443    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/17e30810-dfd0-4e6f-9204-731d1a6479a3-config\") pod \"console-operator-85b454874b-vc5n9\" (UID: \"17e30810-dfd0-4e6f-9204-731d1a6479a3\") " pod="openshift-console-operator/console-operator-85b454874b-vc5n9"
Sep 08 15:18:01.717511 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.717485    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-bound-sa-token\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:01.717616 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.717540    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2c9c\" (UniqueName: \"kubernetes.io/projected/c64984d1-f8b2-414b-8f55-0a4d68e867cf-kube-api-access-c2c9c\") pod \"cluster-monitoring-operator-6cdcc96965-8dlwl\" (UID: \"c64984d1-f8b2-414b-8f55-0a4d68e867cf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8dlwl"
Sep 08 15:18:01.717616 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.717570    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-image-registry-private-configuration\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:01.717616 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.717595    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qfmp\" (UniqueName: \"kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-kube-api-access-7qfmp\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:01.717754 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.717625    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c64984d1-f8b2-414b-8f55-0a4d68e867cf-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-8dlwl\" (UID: \"c64984d1-f8b2-414b-8f55-0a4d68e867cf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8dlwl"
Sep 08 15:18:01.717754 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.717668    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5551e120-e4db-4d93-8d2a-9e3626db076e-default-certificate\") pod \"router-default-88497859b-8nkdj\" (UID: \"5551e120-e4db-4d93-8d2a-9e3626db076e\") " pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:01.717754 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.717698    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-trusted-ca\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:01.717754 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.717724    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhfnf\" (UniqueName: \"kubernetes.io/projected/d4a10483-c29e-4a1c-9eb8-79d3197c1e35-kube-api-access-fhfnf\") pod \"insights-operator-76bcd6ccf5-dkwzw\" (UID: \"d4a10483-c29e-4a1c-9eb8-79d3197c1e35\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dkwzw"
Sep 08 15:18:01.717754 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.717749    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/21604064-e43d-4ec1-b6e9-9af32f6208ed-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-857nk\" (UID: \"21604064-e43d-4ec1-b6e9-9af32f6208ed\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-857nk"
Sep 08 15:18:01.717993 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.717781    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/17e30810-dfd0-4e6f-9204-731d1a6479a3-serving-cert\") pod \"console-operator-85b454874b-vc5n9\" (UID: \"17e30810-dfd0-4e6f-9204-731d1a6479a3\") " pod="openshift-console-operator/console-operator-85b454874b-vc5n9"
Sep 08 15:18:01.717993 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.717794    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d4a10483-c29e-4a1c-9eb8-79d3197c1e35-snapshots\") pod \"insights-operator-76bcd6ccf5-dkwzw\" (UID: \"d4a10483-c29e-4a1c-9eb8-79d3197c1e35\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dkwzw"
Sep 08 15:18:01.717993 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.717822    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxwx6\" (UniqueName: \"kubernetes.io/projected/149a58aa-2b85-4704-b93b-1056c13054df-kube-api-access-qxwx6\") pod \"service-ca-operator-859f94c4b-pvz7x\" (UID: \"149a58aa-2b85-4704-b93b-1056c13054df\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pvz7x"
Sep 08 15:18:01.717993 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.717876    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-certificates\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:01.717993 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.717938    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c64984d1-f8b2-414b-8f55-0a4d68e867cf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-8dlwl\" (UID: \"c64984d1-f8b2-414b-8f55-0a4d68e867cf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8dlwl"
Sep 08 15:18:01.717993 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.717964    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5551e120-e4db-4d93-8d2a-9e3626db076e-service-ca-bundle\") pod \"router-default-88497859b-8nkdj\" (UID: \"5551e120-e4db-4d93-8d2a-9e3626db076e\") " pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:01.717993 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.717989    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/149a58aa-2b85-4704-b93b-1056c13054df-serving-cert\") pod \"service-ca-operator-859f94c4b-pvz7x\" (UID: \"149a58aa-2b85-4704-b93b-1056c13054df\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pvz7x"
Sep 08 15:18:01.718342 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.718018    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pvxm\" (UniqueName: \"kubernetes.io/projected/21604064-e43d-4ec1-b6e9-9af32f6208ed-kube-api-access-8pvxm\") pod \"kube-storage-version-migrator-operator-8d7dd4675-857nk\" (UID: \"21604064-e43d-4ec1-b6e9-9af32f6208ed\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-857nk"
Sep 08 15:18:01.718342 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.718054    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5551e120-e4db-4d93-8d2a-9e3626db076e-stats-auth\") pod \"router-default-88497859b-8nkdj\" (UID: \"5551e120-e4db-4d93-8d2a-9e3626db076e\") " pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:01.718342 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.718094    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/149a58aa-2b85-4704-b93b-1056c13054df-config\") pod \"service-ca-operator-859f94c4b-pvz7x\" (UID: \"149a58aa-2b85-4704-b93b-1056c13054df\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pvz7x"
Sep 08 15:18:01.718342 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.718124    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-installation-pull-secrets\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:01.718342 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.718147    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4a10483-c29e-4a1c-9eb8-79d3197c1e35-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-dkwzw\" (UID: \"d4a10483-c29e-4a1c-9eb8-79d3197c1e35\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dkwzw"
Sep 08 15:18:01.718342 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.718172    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-tls\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:01.718342 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.718195    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-ca-trust-extracted\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:01.718342 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.718223    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4a10483-c29e-4a1c-9eb8-79d3197c1e35-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-dkwzw\" (UID: \"d4a10483-c29e-4a1c-9eb8-79d3197c1e35\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dkwzw"
Sep 08 15:18:01.718342 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.718266    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzxmk\" (UniqueName: \"kubernetes.io/projected/7fc9b652-fc89-43ee-801e-440e67f2bad3-kube-api-access-vzxmk\") pod \"volume-data-source-validator-695c895bd6-km8ph\" (UID: \"7fc9b652-fc89-43ee-801e-440e67f2bad3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-km8ph"
Sep 08 15:18:01.718342 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.718291    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/21604064-e43d-4ec1-b6e9-9af32f6208ed-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-857nk\" (UID: \"21604064-e43d-4ec1-b6e9-9af32f6208ed\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-857nk"
Sep 08 15:18:01.793865 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.793777    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:18:01.794016 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.793777    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:18:01.795795 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.795761    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-np4gb\""
Sep 08 15:18:01.795919 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.795880    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 08 15:18:01.796136 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.796118    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-76z9f\""
Sep 08 15:18:01.819540 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.819513    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pvxm\" (UniqueName: \"kubernetes.io/projected/21604064-e43d-4ec1-b6e9-9af32f6208ed-kube-api-access-8pvxm\") pod \"kube-storage-version-migrator-operator-8d7dd4675-857nk\" (UID: \"21604064-e43d-4ec1-b6e9-9af32f6208ed\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-857nk"
Sep 08 15:18:01.819667 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.819551    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5551e120-e4db-4d93-8d2a-9e3626db076e-stats-auth\") pod \"router-default-88497859b-8nkdj\" (UID: \"5551e120-e4db-4d93-8d2a-9e3626db076e\") " pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:01.819667 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.819566    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/149a58aa-2b85-4704-b93b-1056c13054df-config\") pod \"service-ca-operator-859f94c4b-pvz7x\" (UID: \"149a58aa-2b85-4704-b93b-1056c13054df\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pvz7x"
Sep 08 15:18:01.819667 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.819589    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2df26fe4-655b-4248-a176-8028835f7379-tmp-dir\") pod \"dns-default-ct4vs\" (UID: \"2df26fe4-655b-4248-a176-8028835f7379\") " pod="openshift-dns/dns-default-ct4vs"
Sep 08 15:18:01.819667 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.819615    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-installation-pull-secrets\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:01.819924 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.819676    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4a10483-c29e-4a1c-9eb8-79d3197c1e35-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-dkwzw\" (UID: \"d4a10483-c29e-4a1c-9eb8-79d3197c1e35\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dkwzw"
Sep 08 15:18:01.819924 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.819719    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-tls\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:01.819924 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.819747    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-ca-trust-extracted\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:01.819924 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.819770    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4a10483-c29e-4a1c-9eb8-79d3197c1e35-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-dkwzw\" (UID: \"d4a10483-c29e-4a1c-9eb8-79d3197c1e35\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dkwzw"
Sep 08 15:18:01.819924 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.819800    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzxmk\" (UniqueName: \"kubernetes.io/projected/7fc9b652-fc89-43ee-801e-440e67f2bad3-kube-api-access-vzxmk\") pod \"volume-data-source-validator-695c895bd6-km8ph\" (UID: \"7fc9b652-fc89-43ee-801e-440e67f2bad3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-km8ph"
Sep 08 15:18:01.819924 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.819828    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/21604064-e43d-4ec1-b6e9-9af32f6208ed-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-857nk\" (UID: \"21604064-e43d-4ec1-b6e9-9af32f6208ed\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-857nk"
Sep 08 15:18:01.819924 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.819858    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5551e120-e4db-4d93-8d2a-9e3626db076e-metrics-certs\") pod \"router-default-88497859b-8nkdj\" (UID: \"5551e120-e4db-4d93-8d2a-9e3626db076e\") " pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:01.819924 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.819905    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn7c9\" (UniqueName: \"kubernetes.io/projected/17e30810-dfd0-4e6f-9204-731d1a6479a3-kube-api-access-sn7c9\") pod \"console-operator-85b454874b-vc5n9\" (UID: \"17e30810-dfd0-4e6f-9204-731d1a6479a3\") " pod="openshift-console-operator/console-operator-85b454874b-vc5n9"
Sep 08 15:18:01.820345 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.819939    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxv49\" (UniqueName: \"kubernetes.io/projected/5551e120-e4db-4d93-8d2a-9e3626db076e-kube-api-access-kxv49\") pod \"router-default-88497859b-8nkdj\" (UID: \"5551e120-e4db-4d93-8d2a-9e3626db076e\") " pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:01.820345 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:01.819949    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 15:18:01.820345 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.819968    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d4a10483-c29e-4a1c-9eb8-79d3197c1e35-serving-cert\") pod \"insights-operator-76bcd6ccf5-dkwzw\" (UID: \"d4a10483-c29e-4a1c-9eb8-79d3197c1e35\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dkwzw"
Sep 08 15:18:01.820345 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.820006    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wksks\" (UniqueName: \"kubernetes.io/projected/590ddb7c-a30e-4ad5-9803-7ae4b2267d66-kube-api-access-wksks\") pod \"cluster-samples-operator-7cdcfcf76c-z6gr5\" (UID: \"590ddb7c-a30e-4ad5-9803-7ae4b2267d66\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-z6gr5"
Sep 08 15:18:01.820345 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:01.820027    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5551e120-e4db-4d93-8d2a-9e3626db076e-metrics-certs podName:5551e120-e4db-4d93-8d2a-9e3626db076e nodeName:}" failed. No retries permitted until 2026-09-08 15:18:02.320008922 +0000 UTC m=+35.031951394 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5551e120-e4db-4d93-8d2a-9e3626db076e-metrics-certs") pod "router-default-88497859b-8nkdj" (UID: "5551e120-e4db-4d93-8d2a-9e3626db076e") : secret "router-metrics-certs-default" not found
Sep 08 15:18:01.820345 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.820058    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4a10483-c29e-4a1c-9eb8-79d3197c1e35-tmp\") pod \"insights-operator-76bcd6ccf5-dkwzw\" (UID: \"d4a10483-c29e-4a1c-9eb8-79d3197c1e35\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dkwzw"
Sep 08 15:18:01.820345 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.820115    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/17e30810-dfd0-4e6f-9204-731d1a6479a3-trusted-ca\") pod \"console-operator-85b454874b-vc5n9\" (UID: \"17e30810-dfd0-4e6f-9204-731d1a6479a3\") " pod="openshift-console-operator/console-operator-85b454874b-vc5n9"
Sep 08 15:18:01.820345 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.820147    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/590ddb7c-a30e-4ad5-9803-7ae4b2267d66-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-z6gr5\" (UID: \"590ddb7c-a30e-4ad5-9803-7ae4b2267d66\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-z6gr5"
Sep 08 15:18:01.820345 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.820176    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/12943199-b315-4e6a-ae18-cae8dc762efe-cert\") pod \"ingress-canary-z2r89\" (UID: \"12943199-b315-4e6a-ae18-cae8dc762efe\") " pod="openshift-ingress-canary/ingress-canary-z2r89"
Sep 08 15:18:01.820345 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.820203    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/17e30810-dfd0-4e6f-9204-731d1a6479a3-config\") pod \"console-operator-85b454874b-vc5n9\" (UID: \"17e30810-dfd0-4e6f-9204-731d1a6479a3\") " pod="openshift-console-operator/console-operator-85b454874b-vc5n9"
Sep 08 15:18:01.820345 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.820235    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-bound-sa-token\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:01.820345 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.820236    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/149a58aa-2b85-4704-b93b-1056c13054df-config\") pod \"service-ca-operator-859f94c4b-pvz7x\" (UID: \"149a58aa-2b85-4704-b93b-1056c13054df\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pvz7x"
Sep 08 15:18:01.820345 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.820298    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2c9c\" (UniqueName: \"kubernetes.io/projected/c64984d1-f8b2-414b-8f55-0a4d68e867cf-kube-api-access-c2c9c\") pod \"cluster-monitoring-operator-6cdcc96965-8dlwl\" (UID: \"c64984d1-f8b2-414b-8f55-0a4d68e867cf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8dlwl"
Sep 08 15:18:01.820345 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.820327    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-image-registry-private-configuration\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:01.821019 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.820356    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qfmp\" (UniqueName: \"kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-kube-api-access-7qfmp\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:01.821019 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.820426    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7zxn\" (UniqueName: \"kubernetes.io/projected/12943199-b315-4e6a-ae18-cae8dc762efe-kube-api-access-h7zxn\") pod \"ingress-canary-z2r89\" (UID: \"12943199-b315-4e6a-ae18-cae8dc762efe\") " pod="openshift-ingress-canary/ingress-canary-z2r89"
Sep 08 15:18:01.821019 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.820458    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c64984d1-f8b2-414b-8f55-0a4d68e867cf-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-8dlwl\" (UID: \"c64984d1-f8b2-414b-8f55-0a4d68e867cf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8dlwl"
Sep 08 15:18:01.821019 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.820484    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czcm6\" (UniqueName: \"kubernetes.io/projected/2df26fe4-655b-4248-a176-8028835f7379-kube-api-access-czcm6\") pod \"dns-default-ct4vs\" (UID: \"2df26fe4-655b-4248-a176-8028835f7379\") " pod="openshift-dns/dns-default-ct4vs"
Sep 08 15:18:01.821019 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.820529    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5551e120-e4db-4d93-8d2a-9e3626db076e-default-certificate\") pod \"router-default-88497859b-8nkdj\" (UID: \"5551e120-e4db-4d93-8d2a-9e3626db076e\") " pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:01.821019 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.820560    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-trusted-ca\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:01.821019 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.820588    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhfnf\" (UniqueName: \"kubernetes.io/projected/d4a10483-c29e-4a1c-9eb8-79d3197c1e35-kube-api-access-fhfnf\") pod \"insights-operator-76bcd6ccf5-dkwzw\" (UID: \"d4a10483-c29e-4a1c-9eb8-79d3197c1e35\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dkwzw"
Sep 08 15:18:01.821019 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.820620    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/21604064-e43d-4ec1-b6e9-9af32f6208ed-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-857nk\" (UID: \"21604064-e43d-4ec1-b6e9-9af32f6208ed\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-857nk"
Sep 08 15:18:01.821019 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:01.820663    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 15:18:01.821019 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:01.820678    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64f7cfd7bb-88t2j: secret "image-registry-tls" not found
Sep 08 15:18:01.821019 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.820664    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/17e30810-dfd0-4e6f-9204-731d1a6479a3-serving-cert\") pod \"console-operator-85b454874b-vc5n9\" (UID: \"17e30810-dfd0-4e6f-9204-731d1a6479a3\") " pod="openshift-console-operator/console-operator-85b454874b-vc5n9"
Sep 08 15:18:01.821019 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.820749    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d4a10483-c29e-4a1c-9eb8-79d3197c1e35-snapshots\") pod \"insights-operator-76bcd6ccf5-dkwzw\" (UID: \"d4a10483-c29e-4a1c-9eb8-79d3197c1e35\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dkwzw"
Sep 08 15:18:01.821019 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.820780    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxwx6\" (UniqueName: \"kubernetes.io/projected/149a58aa-2b85-4704-b93b-1056c13054df-kube-api-access-qxwx6\") pod \"service-ca-operator-859f94c4b-pvz7x\" (UID: \"149a58aa-2b85-4704-b93b-1056c13054df\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pvz7x"
Sep 08 15:18:01.821019 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.820783    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4a10483-c29e-4a1c-9eb8-79d3197c1e35-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-dkwzw\" (UID: \"d4a10483-c29e-4a1c-9eb8-79d3197c1e35\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dkwzw"
Sep 08 15:18:01.821019 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.820812    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-certificates\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:01.821019 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:01.820833    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-tls podName:75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa nodeName:}" failed. No retries permitted until 2026-09-08 15:18:02.320817616 +0000 UTC m=+35.032760094 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-tls") pod "image-registry-64f7cfd7bb-88t2j" (UID: "75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa") : secret "image-registry-tls" not found
Sep 08 15:18:01.821735 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.820869    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2df26fe4-655b-4248-a176-8028835f7379-metrics-tls\") pod \"dns-default-ct4vs\" (UID: \"2df26fe4-655b-4248-a176-8028835f7379\") " pod="openshift-dns/dns-default-ct4vs"
Sep 08 15:18:01.821735 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.820929    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c64984d1-f8b2-414b-8f55-0a4d68e867cf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-8dlwl\" (UID: \"c64984d1-f8b2-414b-8f55-0a4d68e867cf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8dlwl"
Sep 08 15:18:01.821735 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.820956    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5551e120-e4db-4d93-8d2a-9e3626db076e-service-ca-bundle\") pod \"router-default-88497859b-8nkdj\" (UID: \"5551e120-e4db-4d93-8d2a-9e3626db076e\") " pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:01.821735 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.820981    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/149a58aa-2b85-4704-b93b-1056c13054df-serving-cert\") pod \"service-ca-operator-859f94c4b-pvz7x\" (UID: \"149a58aa-2b85-4704-b93b-1056c13054df\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pvz7x"
Sep 08 15:18:01.821735 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.821007    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2df26fe4-655b-4248-a176-8028835f7379-config-volume\") pod \"dns-default-ct4vs\" (UID: \"2df26fe4-655b-4248-a176-8028835f7379\") " pod="openshift-dns/dns-default-ct4vs"
Sep 08 15:18:01.821735 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.821036    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkmdl\" (UniqueName: \"kubernetes.io/projected/e073fcbf-44cc-445d-9f6f-669385f1d09d-kube-api-access-nkmdl\") pod \"network-check-source-685456655f-f67sn\" (UID: \"e073fcbf-44cc-445d-9f6f-669385f1d09d\") " pod="openshift-network-diagnostics/network-check-source-685456655f-f67sn"
Sep 08 15:18:01.821735 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.821054    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4a10483-c29e-4a1c-9eb8-79d3197c1e35-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-dkwzw\" (UID: \"d4a10483-c29e-4a1c-9eb8-79d3197c1e35\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dkwzw"
Sep 08 15:18:01.821735 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.821332    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-certificates\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:01.821735 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.821548    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4a10483-c29e-4a1c-9eb8-79d3197c1e35-tmp\") pod \"insights-operator-76bcd6ccf5-dkwzw\" (UID: \"d4a10483-c29e-4a1c-9eb8-79d3197c1e35\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dkwzw"
Sep 08 15:18:01.821735 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.821579    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d4a10483-c29e-4a1c-9eb8-79d3197c1e35-snapshots\") pod \"insights-operator-76bcd6ccf5-dkwzw\" (UID: \"d4a10483-c29e-4a1c-9eb8-79d3197c1e35\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dkwzw"
Sep 08 15:18:01.822395 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:01.821883    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 15:18:01.822395 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:01.821942    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/590ddb7c-a30e-4ad5-9803-7ae4b2267d66-samples-operator-tls podName:590ddb7c-a30e-4ad5-9803-7ae4b2267d66 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:02.321925864 +0000 UTC m=+35.033868350 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/590ddb7c-a30e-4ad5-9803-7ae4b2267d66-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-z6gr5" (UID: "590ddb7c-a30e-4ad5-9803-7ae4b2267d66") : secret "samples-operator-tls" not found
Sep 08 15:18:01.822395 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.822111    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/17e30810-dfd0-4e6f-9204-731d1a6479a3-config\") pod \"console-operator-85b454874b-vc5n9\" (UID: \"17e30810-dfd0-4e6f-9204-731d1a6479a3\") " pod="openshift-console-operator/console-operator-85b454874b-vc5n9"
Sep 08 15:18:01.822395 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.822168    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/21604064-e43d-4ec1-b6e9-9af32f6208ed-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-857nk\" (UID: \"21604064-e43d-4ec1-b6e9-9af32f6208ed\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-857nk"
Sep 08 15:18:01.822395 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:01.822214    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5551e120-e4db-4d93-8d2a-9e3626db076e-service-ca-bundle podName:5551e120-e4db-4d93-8d2a-9e3626db076e nodeName:}" failed. No retries permitted until 2026-09-08 15:18:02.322197919 +0000 UTC m=+35.034140395 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5551e120-e4db-4d93-8d2a-9e3626db076e-service-ca-bundle") pod "router-default-88497859b-8nkdj" (UID: "5551e120-e4db-4d93-8d2a-9e3626db076e") : configmap references non-existent config key: service-ca.crt
Sep 08 15:18:01.822716 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.822479    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-ca-trust-extracted\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:01.822870 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.822824    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c64984d1-f8b2-414b-8f55-0a4d68e867cf-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-8dlwl\" (UID: \"c64984d1-f8b2-414b-8f55-0a4d68e867cf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8dlwl"
Sep 08 15:18:01.822976 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.822866    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/17e30810-dfd0-4e6f-9204-731d1a6479a3-trusted-ca\") pod \"console-operator-85b454874b-vc5n9\" (UID: \"17e30810-dfd0-4e6f-9204-731d1a6479a3\") " pod="openshift-console-operator/console-operator-85b454874b-vc5n9"
Sep 08 15:18:01.822976 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:01.822940    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 15:18:01.823111 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:01.822985    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c64984d1-f8b2-414b-8f55-0a4d68e867cf-cluster-monitoring-operator-tls podName:c64984d1-f8b2-414b-8f55-0a4d68e867cf nodeName:}" failed. No retries permitted until 2026-09-08 15:18:02.322970411 +0000 UTC m=+35.034912886 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c64984d1-f8b2-414b-8f55-0a4d68e867cf-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-8dlwl" (UID: "c64984d1-f8b2-414b-8f55-0a4d68e867cf") : secret "cluster-monitoring-operator-tls" not found
Sep 08 15:18:01.824377 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.823814    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-trusted-ca\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:01.825414 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.825128    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/17e30810-dfd0-4e6f-9204-731d1a6479a3-serving-cert\") pod \"console-operator-85b454874b-vc5n9\" (UID: \"17e30810-dfd0-4e6f-9204-731d1a6479a3\") " pod="openshift-console-operator/console-operator-85b454874b-vc5n9"
Sep 08 15:18:01.825414 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.825238    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-installation-pull-secrets\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:01.825414 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.825299    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/149a58aa-2b85-4704-b93b-1056c13054df-serving-cert\") pod \"service-ca-operator-859f94c4b-pvz7x\" (UID: \"149a58aa-2b85-4704-b93b-1056c13054df\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pvz7x"
Sep 08 15:18:01.825982 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.825959    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5551e120-e4db-4d93-8d2a-9e3626db076e-default-certificate\") pod \"router-default-88497859b-8nkdj\" (UID: \"5551e120-e4db-4d93-8d2a-9e3626db076e\") " pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:01.826592 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.826551    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/21604064-e43d-4ec1-b6e9-9af32f6208ed-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-857nk\" (UID: \"21604064-e43d-4ec1-b6e9-9af32f6208ed\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-857nk"
Sep 08 15:18:01.826592 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.826567    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5551e120-e4db-4d93-8d2a-9e3626db076e-stats-auth\") pod \"router-default-88497859b-8nkdj\" (UID: \"5551e120-e4db-4d93-8d2a-9e3626db076e\") " pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:01.827513 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.827458    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d4a10483-c29e-4a1c-9eb8-79d3197c1e35-serving-cert\") pod \"insights-operator-76bcd6ccf5-dkwzw\" (UID: \"d4a10483-c29e-4a1c-9eb8-79d3197c1e35\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dkwzw"
Sep 08 15:18:01.827870 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.827832    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-image-registry-private-configuration\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:01.832746 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.832711    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wksks\" (UniqueName: \"kubernetes.io/projected/590ddb7c-a30e-4ad5-9803-7ae4b2267d66-kube-api-access-wksks\") pod \"cluster-samples-operator-7cdcfcf76c-z6gr5\" (UID: \"590ddb7c-a30e-4ad5-9803-7ae4b2267d66\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-z6gr5"
Sep 08 15:18:01.833061 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.833016    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pvxm\" (UniqueName: \"kubernetes.io/projected/21604064-e43d-4ec1-b6e9-9af32f6208ed-kube-api-access-8pvxm\") pod \"kube-storage-version-migrator-operator-8d7dd4675-857nk\" (UID: \"21604064-e43d-4ec1-b6e9-9af32f6208ed\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-857nk"
Sep 08 15:18:01.836256 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.836225    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzxmk\" (UniqueName: \"kubernetes.io/projected/7fc9b652-fc89-43ee-801e-440e67f2bad3-kube-api-access-vzxmk\") pod \"volume-data-source-validator-695c895bd6-km8ph\" (UID: \"7fc9b652-fc89-43ee-801e-440e67f2bad3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-km8ph"
Sep 08 15:18:01.836459 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.836412    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-bound-sa-token\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:01.837318 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.837277    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxv49\" (UniqueName: \"kubernetes.io/projected/5551e120-e4db-4d93-8d2a-9e3626db076e-kube-api-access-kxv49\") pod \"router-default-88497859b-8nkdj\" (UID: \"5551e120-e4db-4d93-8d2a-9e3626db076e\") " pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:01.837457 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.837421    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn7c9\" (UniqueName: \"kubernetes.io/projected/17e30810-dfd0-4e6f-9204-731d1a6479a3-kube-api-access-sn7c9\") pod \"console-operator-85b454874b-vc5n9\" (UID: \"17e30810-dfd0-4e6f-9204-731d1a6479a3\") " pod="openshift-console-operator/console-operator-85b454874b-vc5n9"
Sep 08 15:18:01.837740 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.837716    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2c9c\" (UniqueName: \"kubernetes.io/projected/c64984d1-f8b2-414b-8f55-0a4d68e867cf-kube-api-access-c2c9c\") pod \"cluster-monitoring-operator-6cdcc96965-8dlwl\" (UID: \"c64984d1-f8b2-414b-8f55-0a4d68e867cf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8dlwl"
Sep 08 15:18:01.837952 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.837934    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhfnf\" (UniqueName: \"kubernetes.io/projected/d4a10483-c29e-4a1c-9eb8-79d3197c1e35-kube-api-access-fhfnf\") pod \"insights-operator-76bcd6ccf5-dkwzw\" (UID: \"d4a10483-c29e-4a1c-9eb8-79d3197c1e35\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dkwzw"
Sep 08 15:18:01.838566 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.838548    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxwx6\" (UniqueName: \"kubernetes.io/projected/149a58aa-2b85-4704-b93b-1056c13054df-kube-api-access-qxwx6\") pod \"service-ca-operator-859f94c4b-pvz7x\" (UID: \"149a58aa-2b85-4704-b93b-1056c13054df\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pvz7x"
Sep 08 15:18:01.841154 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.841135    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qfmp\" (UniqueName: \"kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-kube-api-access-7qfmp\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:01.869536 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.869496    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:01.922329 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.922293    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/12943199-b315-4e6a-ae18-cae8dc762efe-cert\") pod \"ingress-canary-z2r89\" (UID: \"12943199-b315-4e6a-ae18-cae8dc762efe\") " pod="openshift-ingress-canary/ingress-canary-z2r89"
Sep 08 15:18:01.922329 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.922336    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7zxn\" (UniqueName: \"kubernetes.io/projected/12943199-b315-4e6a-ae18-cae8dc762efe-kube-api-access-h7zxn\") pod \"ingress-canary-z2r89\" (UID: \"12943199-b315-4e6a-ae18-cae8dc762efe\") " pod="openshift-ingress-canary/ingress-canary-z2r89"
Sep 08 15:18:01.922564 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.922364    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czcm6\" (UniqueName: \"kubernetes.io/projected/2df26fe4-655b-4248-a176-8028835f7379-kube-api-access-czcm6\") pod \"dns-default-ct4vs\" (UID: \"2df26fe4-655b-4248-a176-8028835f7379\") " pod="openshift-dns/dns-default-ct4vs"
Sep 08 15:18:01.922564 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.922514    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2df26fe4-655b-4248-a176-8028835f7379-metrics-tls\") pod \"dns-default-ct4vs\" (UID: \"2df26fe4-655b-4248-a176-8028835f7379\") " pod="openshift-dns/dns-default-ct4vs"
Sep 08 15:18:01.922669 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.922597    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2df26fe4-655b-4248-a176-8028835f7379-config-volume\") pod \"dns-default-ct4vs\" (UID: \"2df26fe4-655b-4248-a176-8028835f7379\") " pod="openshift-dns/dns-default-ct4vs"
Sep 08 15:18:01.922669 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.922624    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkmdl\" (UniqueName: \"kubernetes.io/projected/e073fcbf-44cc-445d-9f6f-669385f1d09d-kube-api-access-nkmdl\") pod \"network-check-source-685456655f-f67sn\" (UID: \"e073fcbf-44cc-445d-9f6f-669385f1d09d\") " pod="openshift-network-diagnostics/network-check-source-685456655f-f67sn"
Sep 08 15:18:01.922669 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.922663    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2df26fe4-655b-4248-a176-8028835f7379-tmp-dir\") pod \"dns-default-ct4vs\" (UID: \"2df26fe4-655b-4248-a176-8028835f7379\") " pod="openshift-dns/dns-default-ct4vs"
Sep 08 15:18:01.922809 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:01.922766    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 15:18:01.922854 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:01.922833    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2df26fe4-655b-4248-a176-8028835f7379-metrics-tls podName:2df26fe4-655b-4248-a176-8028835f7379 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:02.422809422 +0000 UTC m=+35.134751890 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2df26fe4-655b-4248-a176-8028835f7379-metrics-tls") pod "dns-default-ct4vs" (UID: "2df26fe4-655b-4248-a176-8028835f7379") : secret "dns-default-metrics-tls" not found
Sep 08 15:18:01.923147 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.923090    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2df26fe4-655b-4248-a176-8028835f7379-tmp-dir\") pod \"dns-default-ct4vs\" (UID: \"2df26fe4-655b-4248-a176-8028835f7379\") " pod="openshift-dns/dns-default-ct4vs"
Sep 08 15:18:01.923297 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.923279    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2df26fe4-655b-4248-a176-8028835f7379-config-volume\") pod \"dns-default-ct4vs\" (UID: \"2df26fe4-655b-4248-a176-8028835f7379\") " pod="openshift-dns/dns-default-ct4vs"
Sep 08 15:18:01.925187 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.925162    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/12943199-b315-4e6a-ae18-cae8dc762efe-cert\") pod \"ingress-canary-z2r89\" (UID: \"12943199-b315-4e6a-ae18-cae8dc762efe\") " pod="openshift-ingress-canary/ingress-canary-z2r89"
Sep 08 15:18:01.930899 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.930880    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-km8ph"
Sep 08 15:18:01.949952 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.949926    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czcm6\" (UniqueName: \"kubernetes.io/projected/2df26fe4-655b-4248-a176-8028835f7379-kube-api-access-czcm6\") pod \"dns-default-ct4vs\" (UID: \"2df26fe4-655b-4248-a176-8028835f7379\") " pod="openshift-dns/dns-default-ct4vs"
Sep 08 15:18:01.950542 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.950523    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkmdl\" (UniqueName: \"kubernetes.io/projected/e073fcbf-44cc-445d-9f6f-669385f1d09d-kube-api-access-nkmdl\") pod \"network-check-source-685456655f-f67sn\" (UID: \"e073fcbf-44cc-445d-9f6f-669385f1d09d\") " pod="openshift-network-diagnostics/network-check-source-685456655f-f67sn"
Sep 08 15:18:01.950632 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.950620    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-vc5n9"
Sep 08 15:18:01.956845 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.956798    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7zxn\" (UniqueName: \"kubernetes.io/projected/12943199-b315-4e6a-ae18-cae8dc762efe-kube-api-access-h7zxn\") pod \"ingress-canary-z2r89\" (UID: \"12943199-b315-4e6a-ae18-cae8dc762efe\") " pod="openshift-ingress-canary/ingress-canary-z2r89"
Sep 08 15:18:01.959448 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.959429    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-857nk"
Sep 08 15:18:01.968806 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.968780    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-dkwzw"
Sep 08 15:18:01.985795 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.985768    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pvz7x"
Sep 08 15:18:01.994446 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:01.994368    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-f67sn"
Sep 08 15:18:02.000138 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:02.000060    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:02.000474 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:02.000267    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z2r89"
Sep 08 15:18:02.257697 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:02.257417    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-km8ph"]
Sep 08 15:18:02.258322 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:02.258263    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-vc5n9"]
Sep 08 15:18:02.261827 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:18:02.261805    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fc9b652_fc89_43ee_801e_440e67f2bad3.slice/crio-d0e9bcf82e8e7f48577df51b128fc9398c9aefe8a40ae1e61d6ba66bbe3ae087 WatchSource:0}: Error finding container d0e9bcf82e8e7f48577df51b128fc9398c9aefe8a40ae1e61d6ba66bbe3ae087: Status 404 returned error can't find the container with id d0e9bcf82e8e7f48577df51b128fc9398c9aefe8a40ae1e61d6ba66bbe3ae087
Sep 08 15:18:02.262872 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:18:02.262837    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17e30810_dfd0_4e6f_9204_731d1a6479a3.slice/crio-bbd96308854926d25e6eb564a0c8f9fa1e950fb466be51e5c805ce056b40a971 WatchSource:0}: Error finding container bbd96308854926d25e6eb564a0c8f9fa1e950fb466be51e5c805ce056b40a971: Status 404 returned error can't find the container with id bbd96308854926d25e6eb564a0c8f9fa1e950fb466be51e5c805ce056b40a971
Sep 08 15:18:02.283774 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:02.283748    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-857nk"]
Sep 08 15:18:02.287484 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:18:02.287459    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21604064_e43d_4ec1_b6e9_9af32f6208ed.slice/crio-5c8dc20670eb39b103ea168aeb1aa38b13be0b7894d77c1e2a5854d9b18cb1a2 WatchSource:0}: Error finding container 5c8dc20670eb39b103ea168aeb1aa38b13be0b7894d77c1e2a5854d9b18cb1a2: Status 404 returned error can't find the container with id 5c8dc20670eb39b103ea168aeb1aa38b13be0b7894d77c1e2a5854d9b18cb1a2
Sep 08 15:18:02.290854 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:02.290600    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-dkwzw"]
Sep 08 15:18:02.294343 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:18:02.294322    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4a10483_c29e_4a1c_9eb8_79d3197c1e35.slice/crio-4b4a70e55ca38ee22270dd807661da1e96c0bc9b06ccd9ab83f3c7f6ea4cb7e2 WatchSource:0}: Error finding container 4b4a70e55ca38ee22270dd807661da1e96c0bc9b06ccd9ab83f3c7f6ea4cb7e2: Status 404 returned error can't find the container with id 4b4a70e55ca38ee22270dd807661da1e96c0bc9b06ccd9ab83f3c7f6ea4cb7e2
Sep 08 15:18:02.325149 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:02.325046    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-pvz7x"]
Sep 08 15:18:02.328420 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:18:02.328351    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod149a58aa_2b85_4704_b93b_1056c13054df.slice/crio-b678281ce234dbb5811808bb6a23174c42982f9310919272b6161a6a7cec9ce7 WatchSource:0}: Error finding container b678281ce234dbb5811808bb6a23174c42982f9310919272b6161a6a7cec9ce7: Status 404 returned error can't find the container with id b678281ce234dbb5811808bb6a23174c42982f9310919272b6161a6a7cec9ce7
Sep 08 15:18:02.330395 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:02.330374    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c64984d1-f8b2-414b-8f55-0a4d68e867cf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-8dlwl\" (UID: \"c64984d1-f8b2-414b-8f55-0a4d68e867cf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8dlwl"
Sep 08 15:18:02.330450 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:02.330408    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5551e120-e4db-4d93-8d2a-9e3626db076e-service-ca-bundle\") pod \"router-default-88497859b-8nkdj\" (UID: \"5551e120-e4db-4d93-8d2a-9e3626db076e\") " pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:02.330450 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:02.330439    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-tls\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:02.330650 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:02.330469    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5551e120-e4db-4d93-8d2a-9e3626db076e-metrics-certs\") pod \"router-default-88497859b-8nkdj\" (UID: \"5551e120-e4db-4d93-8d2a-9e3626db076e\") " pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:02.330650 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:02.330530    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/590ddb7c-a30e-4ad5-9803-7ae4b2267d66-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-z6gr5\" (UID: \"590ddb7c-a30e-4ad5-9803-7ae4b2267d66\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-z6gr5"
Sep 08 15:18:02.330650 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:02.330531    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 15:18:02.330650 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:02.330592    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 15:18:02.330842 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:02.330657    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 15:18:02.330842 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:02.330675    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64f7cfd7bb-88t2j: secret "image-registry-tls" not found
Sep 08 15:18:02.330842 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:02.330637    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5551e120-e4db-4d93-8d2a-9e3626db076e-service-ca-bundle podName:5551e120-e4db-4d93-8d2a-9e3626db076e nodeName:}" failed. No retries permitted until 2026-09-08 15:18:03.330615765 +0000 UTC m=+36.042558253 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5551e120-e4db-4d93-8d2a-9e3626db076e-service-ca-bundle") pod "router-default-88497859b-8nkdj" (UID: "5551e120-e4db-4d93-8d2a-9e3626db076e") : configmap references non-existent config key: service-ca.crt
Sep 08 15:18:02.330842 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:02.330728    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c64984d1-f8b2-414b-8f55-0a4d68e867cf-cluster-monitoring-operator-tls podName:c64984d1-f8b2-414b-8f55-0a4d68e867cf nodeName:}" failed. No retries permitted until 2026-09-08 15:18:03.330710973 +0000 UTC m=+36.042653444 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c64984d1-f8b2-414b-8f55-0a4d68e867cf-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-8dlwl" (UID: "c64984d1-f8b2-414b-8f55-0a4d68e867cf") : secret "cluster-monitoring-operator-tls" not found
Sep 08 15:18:02.330842 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:02.330746    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5551e120-e4db-4d93-8d2a-9e3626db076e-metrics-certs podName:5551e120-e4db-4d93-8d2a-9e3626db076e nodeName:}" failed. No retries permitted until 2026-09-08 15:18:03.330736459 +0000 UTC m=+36.042678936 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5551e120-e4db-4d93-8d2a-9e3626db076e-metrics-certs") pod "router-default-88497859b-8nkdj" (UID: "5551e120-e4db-4d93-8d2a-9e3626db076e") : secret "router-metrics-certs-default" not found
Sep 08 15:18:02.330842 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:02.330747    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 15:18:02.330842 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:02.330764    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-tls podName:75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa nodeName:}" failed. No retries permitted until 2026-09-08 15:18:03.330755833 +0000 UTC m=+36.042698307 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-tls") pod "image-registry-64f7cfd7bb-88t2j" (UID: "75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa") : secret "image-registry-tls" not found
Sep 08 15:18:02.330842 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:02.330788    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/590ddb7c-a30e-4ad5-9803-7ae4b2267d66-samples-operator-tls podName:590ddb7c-a30e-4ad5-9803-7ae4b2267d66 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:03.330773872 +0000 UTC m=+36.042716358 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/590ddb7c-a30e-4ad5-9803-7ae4b2267d66-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-z6gr5" (UID: "590ddb7c-a30e-4ad5-9803-7ae4b2267d66") : secret "samples-operator-tls" not found
Sep 08 15:18:02.355404 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:02.355374    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-f67sn"]
Sep 08 15:18:02.360524 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:18:02.360020    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode073fcbf_44cc_445d_9f6f_669385f1d09d.slice/crio-ce87d636417387dc8bdf73b10b92c0e17cdbff695f30f87dcafe928e23ef4bf4 WatchSource:0}: Error finding container ce87d636417387dc8bdf73b10b92c0e17cdbff695f30f87dcafe928e23ef4bf4: Status 404 returned error can't find the container with id ce87d636417387dc8bdf73b10b92c0e17cdbff695f30f87dcafe928e23ef4bf4
Sep 08 15:18:02.363914 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:02.363886    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z2r89"]
Sep 08 15:18:02.367039 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:18:02.367012    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12943199_b315_4e6a_ae18_cae8dc762efe.slice/crio-c3a119052b0ee576505fc1ed835e83507698762ae4e88181283707b7ad295b75 WatchSource:0}: Error finding container c3a119052b0ee576505fc1ed835e83507698762ae4e88181283707b7ad295b75: Status 404 returned error can't find the container with id c3a119052b0ee576505fc1ed835e83507698762ae4e88181283707b7ad295b75
Sep 08 15:18:02.431565 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:02.431530    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2df26fe4-655b-4248-a176-8028835f7379-metrics-tls\") pod \"dns-default-ct4vs\" (UID: \"2df26fe4-655b-4248-a176-8028835f7379\") " pod="openshift-dns/dns-default-ct4vs"
Sep 08 15:18:02.431941 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:02.431649    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 15:18:02.431941 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:02.431712    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2df26fe4-655b-4248-a176-8028835f7379-metrics-tls podName:2df26fe4-655b-4248-a176-8028835f7379 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:03.431694182 +0000 UTC m=+36.143636663 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2df26fe4-655b-4248-a176-8028835f7379-metrics-tls") pod "dns-default-ct4vs" (UID: "2df26fe4-655b-4248-a176-8028835f7379") : secret "dns-default-metrics-tls" not found
Sep 08 15:18:02.794189 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:02.794151    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:18:02.796449 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:02.796418    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 08 15:18:02.943153 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:02.943111    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pvz7x" event={"ID":"149a58aa-2b85-4704-b93b-1056c13054df","Type":"ContainerStarted","Data":"b678281ce234dbb5811808bb6a23174c42982f9310919272b6161a6a7cec9ce7"}
Sep 08 15:18:02.944621 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:02.944570    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-vc5n9" event={"ID":"17e30810-dfd0-4e6f-9204-731d1a6479a3","Type":"ContainerStarted","Data":"bbd96308854926d25e6eb564a0c8f9fa1e950fb466be51e5c805ce056b40a971"}
Sep 08 15:18:02.946002 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:02.945974    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-f67sn" event={"ID":"e073fcbf-44cc-445d-9f6f-669385f1d09d","Type":"ContainerStarted","Data":"ce87d636417387dc8bdf73b10b92c0e17cdbff695f30f87dcafe928e23ef4bf4"}
Sep 08 15:18:02.947480 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:02.947450    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-dkwzw" event={"ID":"d4a10483-c29e-4a1c-9eb8-79d3197c1e35","Type":"ContainerStarted","Data":"4b4a70e55ca38ee22270dd807661da1e96c0bc9b06ccd9ab83f3c7f6ea4cb7e2"}
Sep 08 15:18:02.948903 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:02.948854    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-km8ph" event={"ID":"7fc9b652-fc89-43ee-801e-440e67f2bad3","Type":"ContainerStarted","Data":"d0e9bcf82e8e7f48577df51b128fc9398c9aefe8a40ae1e61d6ba66bbe3ae087"}
Sep 08 15:18:02.950210 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:02.950180    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z2r89" event={"ID":"12943199-b315-4e6a-ae18-cae8dc762efe","Type":"ContainerStarted","Data":"c3a119052b0ee576505fc1ed835e83507698762ae4e88181283707b7ad295b75"}
Sep 08 15:18:02.951585 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:02.951561    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-857nk" event={"ID":"21604064-e43d-4ec1-b6e9-9af32f6208ed","Type":"ContainerStarted","Data":"5c8dc20670eb39b103ea168aeb1aa38b13be0b7894d77c1e2a5854d9b18cb1a2"}
Sep 08 15:18:03.339028 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:03.338990    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/590ddb7c-a30e-4ad5-9803-7ae4b2267d66-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-z6gr5\" (UID: \"590ddb7c-a30e-4ad5-9803-7ae4b2267d66\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-z6gr5"
Sep 08 15:18:03.339247 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:03.339123    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c64984d1-f8b2-414b-8f55-0a4d68e867cf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-8dlwl\" (UID: \"c64984d1-f8b2-414b-8f55-0a4d68e867cf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8dlwl"
Sep 08 15:18:03.339247 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:03.339154    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5551e120-e4db-4d93-8d2a-9e3626db076e-service-ca-bundle\") pod \"router-default-88497859b-8nkdj\" (UID: \"5551e120-e4db-4d93-8d2a-9e3626db076e\") " pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:03.339247 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:03.339178    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 15:18:03.339247 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:03.339195    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-tls\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:03.339247 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:03.339227    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5551e120-e4db-4d93-8d2a-9e3626db076e-metrics-certs\") pod \"router-default-88497859b-8nkdj\" (UID: \"5551e120-e4db-4d93-8d2a-9e3626db076e\") " pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:03.339492 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:03.339255    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/590ddb7c-a30e-4ad5-9803-7ae4b2267d66-samples-operator-tls podName:590ddb7c-a30e-4ad5-9803-7ae4b2267d66 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:05.339239037 +0000 UTC m=+38.051181528 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/590ddb7c-a30e-4ad5-9803-7ae4b2267d66-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-z6gr5" (UID: "590ddb7c-a30e-4ad5-9803-7ae4b2267d66") : secret "samples-operator-tls" not found
Sep 08 15:18:03.339492 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:03.339311    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 15:18:03.339492 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:03.339339    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 15:18:03.339492 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:03.339355    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64f7cfd7bb-88t2j: secret "image-registry-tls" not found
Sep 08 15:18:03.339492 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:03.339315    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5551e120-e4db-4d93-8d2a-9e3626db076e-service-ca-bundle podName:5551e120-e4db-4d93-8d2a-9e3626db076e nodeName:}" failed. No retries permitted until 2026-09-08 15:18:05.339296295 +0000 UTC m=+38.051238779 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5551e120-e4db-4d93-8d2a-9e3626db076e-service-ca-bundle") pod "router-default-88497859b-8nkdj" (UID: "5551e120-e4db-4d93-8d2a-9e3626db076e") : configmap references non-existent config key: service-ca.crt
Sep 08 15:18:03.339492 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:03.339378    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 15:18:03.339492 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:03.339380    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5551e120-e4db-4d93-8d2a-9e3626db076e-metrics-certs podName:5551e120-e4db-4d93-8d2a-9e3626db076e nodeName:}" failed. No retries permitted until 2026-09-08 15:18:05.339365967 +0000 UTC m=+38.051308438 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5551e120-e4db-4d93-8d2a-9e3626db076e-metrics-certs") pod "router-default-88497859b-8nkdj" (UID: "5551e120-e4db-4d93-8d2a-9e3626db076e") : secret "router-metrics-certs-default" not found
Sep 08 15:18:03.339492 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:03.339441    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-tls podName:75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa nodeName:}" failed. No retries permitted until 2026-09-08 15:18:05.339428895 +0000 UTC m=+38.051371369 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-tls") pod "image-registry-64f7cfd7bb-88t2j" (UID: "75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa") : secret "image-registry-tls" not found
Sep 08 15:18:03.339492 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:03.339457    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c64984d1-f8b2-414b-8f55-0a4d68e867cf-cluster-monitoring-operator-tls podName:c64984d1-f8b2-414b-8f55-0a4d68e867cf nodeName:}" failed. No retries permitted until 2026-09-08 15:18:05.339447505 +0000 UTC m=+38.051389979 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c64984d1-f8b2-414b-8f55-0a4d68e867cf-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-8dlwl" (UID: "c64984d1-f8b2-414b-8f55-0a4d68e867cf") : secret "cluster-monitoring-operator-tls" not found
Sep 08 15:18:03.440363 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:03.440329    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2df26fe4-655b-4248-a176-8028835f7379-metrics-tls\") pod \"dns-default-ct4vs\" (UID: \"2df26fe4-655b-4248-a176-8028835f7379\") " pod="openshift-dns/dns-default-ct4vs"
Sep 08 15:18:03.440789 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:03.440499    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 15:18:03.440789 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:03.440599    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2df26fe4-655b-4248-a176-8028835f7379-metrics-tls podName:2df26fe4-655b-4248-a176-8028835f7379 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:05.440574775 +0000 UTC m=+38.152517247 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2df26fe4-655b-4248-a176-8028835f7379-metrics-tls") pod "dns-default-ct4vs" (UID: "2df26fe4-655b-4248-a176-8028835f7379") : secret "dns-default-metrics-tls" not found
Sep 08 15:18:04.963560 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:04.963524    2560 generic.go:356] "Generic (PLEG): container finished" podID="202dffc4-2ade-40df-96be-4f6d20b0d998" containerID="acdd35a4e4220f671ba53f90e874c25d3efdbd1cdf656c4410e50b79858ee621" exitCode=0
Sep 08 15:18:04.964324 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:04.963815    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jstb5" event={"ID":"202dffc4-2ade-40df-96be-4f6d20b0d998","Type":"ContainerDied","Data":"acdd35a4e4220f671ba53f90e874c25d3efdbd1cdf656c4410e50b79858ee621"}
Sep 08 15:18:05.359465 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:05.358217    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-tls\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:05.359465 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:05.358278    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5551e120-e4db-4d93-8d2a-9e3626db076e-metrics-certs\") pod \"router-default-88497859b-8nkdj\" (UID: \"5551e120-e4db-4d93-8d2a-9e3626db076e\") " pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:05.359465 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:05.358321    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/590ddb7c-a30e-4ad5-9803-7ae4b2267d66-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-z6gr5\" (UID: \"590ddb7c-a30e-4ad5-9803-7ae4b2267d66\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-z6gr5"
Sep 08 15:18:05.359465 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:05.358441    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c64984d1-f8b2-414b-8f55-0a4d68e867cf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-8dlwl\" (UID: \"c64984d1-f8b2-414b-8f55-0a4d68e867cf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8dlwl"
Sep 08 15:18:05.359465 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:05.358469    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5551e120-e4db-4d93-8d2a-9e3626db076e-service-ca-bundle\") pod \"router-default-88497859b-8nkdj\" (UID: \"5551e120-e4db-4d93-8d2a-9e3626db076e\") " pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:05.359465 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:05.358617    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5551e120-e4db-4d93-8d2a-9e3626db076e-service-ca-bundle podName:5551e120-e4db-4d93-8d2a-9e3626db076e nodeName:}" failed. No retries permitted until 2026-09-08 15:18:09.358597511 +0000 UTC m=+42.070540004 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5551e120-e4db-4d93-8d2a-9e3626db076e-service-ca-bundle") pod "router-default-88497859b-8nkdj" (UID: "5551e120-e4db-4d93-8d2a-9e3626db076e") : configmap references non-existent config key: service-ca.crt
Sep 08 15:18:05.359465 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:05.359031    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 15:18:05.359465 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:05.359050    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64f7cfd7bb-88t2j: secret "image-registry-tls" not found
Sep 08 15:18:05.359465 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:05.359111    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-tls podName:75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa nodeName:}" failed. No retries permitted until 2026-09-08 15:18:09.359094856 +0000 UTC m=+42.071037339 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-tls") pod "image-registry-64f7cfd7bb-88t2j" (UID: "75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa") : secret "image-registry-tls" not found
Sep 08 15:18:05.359465 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:05.359172    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 15:18:05.359465 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:05.359204    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5551e120-e4db-4d93-8d2a-9e3626db076e-metrics-certs podName:5551e120-e4db-4d93-8d2a-9e3626db076e nodeName:}" failed. No retries permitted until 2026-09-08 15:18:09.359193873 +0000 UTC m=+42.071136347 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5551e120-e4db-4d93-8d2a-9e3626db076e-metrics-certs") pod "router-default-88497859b-8nkdj" (UID: "5551e120-e4db-4d93-8d2a-9e3626db076e") : secret "router-metrics-certs-default" not found
Sep 08 15:18:05.359465 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:05.359255    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 15:18:05.359465 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:05.359285    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/590ddb7c-a30e-4ad5-9803-7ae4b2267d66-samples-operator-tls podName:590ddb7c-a30e-4ad5-9803-7ae4b2267d66 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:09.359274966 +0000 UTC m=+42.071217450 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/590ddb7c-a30e-4ad5-9803-7ae4b2267d66-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-z6gr5" (UID: "590ddb7c-a30e-4ad5-9803-7ae4b2267d66") : secret "samples-operator-tls" not found
Sep 08 15:18:05.359465 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:05.359336    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 15:18:05.359465 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:05.359380    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c64984d1-f8b2-414b-8f55-0a4d68e867cf-cluster-monitoring-operator-tls podName:c64984d1-f8b2-414b-8f55-0a4d68e867cf nodeName:}" failed. No retries permitted until 2026-09-08 15:18:09.359355858 +0000 UTC m=+42.071298344 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c64984d1-f8b2-414b-8f55-0a4d68e867cf-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-8dlwl" (UID: "c64984d1-f8b2-414b-8f55-0a4d68e867cf") : secret "cluster-monitoring-operator-tls" not found
Sep 08 15:18:05.459844 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:05.459158    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2df26fe4-655b-4248-a176-8028835f7379-metrics-tls\") pod \"dns-default-ct4vs\" (UID: \"2df26fe4-655b-4248-a176-8028835f7379\") " pod="openshift-dns/dns-default-ct4vs"
Sep 08 15:18:05.459844 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:05.459413    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 15:18:05.459844 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:05.459475    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2df26fe4-655b-4248-a176-8028835f7379-metrics-tls podName:2df26fe4-655b-4248-a176-8028835f7379 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:09.459457499 +0000 UTC m=+42.171399982 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2df26fe4-655b-4248-a176-8028835f7379-metrics-tls") pod "dns-default-ct4vs" (UID: "2df26fe4-655b-4248-a176-8028835f7379") : secret "dns-default-metrics-tls" not found
Sep 08 15:18:05.981392 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:05.981358    2560 generic.go:356] "Generic (PLEG): container finished" podID="202dffc4-2ade-40df-96be-4f6d20b0d998" containerID="89d6fac2e24d694bd5146cf7b3a5fd70fa8801d95d0fd19f3dc89dc1a3b0d0b5" exitCode=0
Sep 08 15:18:05.981927 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:05.981408    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jstb5" event={"ID":"202dffc4-2ade-40df-96be-4f6d20b0d998","Type":"ContainerDied","Data":"89d6fac2e24d694bd5146cf7b3a5fd70fa8801d95d0fd19f3dc89dc1a3b0d0b5"}
Sep 08 15:18:06.201248 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:06.201210    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:06.275064 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:06.275030    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:06.661027 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:06.660907    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:06.744574 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:06.744539    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:06.847878 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:06.847782    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:06.947588 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:06.947556    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:06.986916 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:06.986878    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-km8ph" event={"ID":"7fc9b652-fc89-43ee-801e-440e67f2bad3","Type":"ContainerStarted","Data":"9e72f595242d3cd52a9b8e2a1aa21fe99327eb2fc27af4f18628b40275dcb7be"}
Sep 08 15:18:07.003270 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:07.003207    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-km8ph" podStartSLOduration=26.4210121 podStartE2EDuration="30.003182239s" totalImagesPullingTime="3.582170139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:17:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:07.002128546 +0000 UTC m=+39.714071038" watchObservedRunningTime="2026-09-08 15:18:07.003182239 +0000 UTC m=+39.715124767"
Sep 08 15:18:08.716039 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:08.715997    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:08.807255 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:08.807201    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:09.400006 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:09.399961    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/590ddb7c-a30e-4ad5-9803-7ae4b2267d66-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-z6gr5\" (UID: \"590ddb7c-a30e-4ad5-9803-7ae4b2267d66\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-z6gr5"
Sep 08 15:18:09.400228 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:09.400112    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c64984d1-f8b2-414b-8f55-0a4d68e867cf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-8dlwl\" (UID: \"c64984d1-f8b2-414b-8f55-0a4d68e867cf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8dlwl"
Sep 08 15:18:09.400228 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:09.400140    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 15:18:09.400228 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:09.400145    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5551e120-e4db-4d93-8d2a-9e3626db076e-service-ca-bundle\") pod \"router-default-88497859b-8nkdj\" (UID: \"5551e120-e4db-4d93-8d2a-9e3626db076e\") " pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:09.400228 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:09.400213    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/590ddb7c-a30e-4ad5-9803-7ae4b2267d66-samples-operator-tls podName:590ddb7c-a30e-4ad5-9803-7ae4b2267d66 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:17.400193044 +0000 UTC m=+50.112135530 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/590ddb7c-a30e-4ad5-9803-7ae4b2267d66-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-z6gr5" (UID: "590ddb7c-a30e-4ad5-9803-7ae4b2267d66") : secret "samples-operator-tls" not found
Sep 08 15:18:09.400458 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:09.400242    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5551e120-e4db-4d93-8d2a-9e3626db076e-service-ca-bundle podName:5551e120-e4db-4d93-8d2a-9e3626db076e nodeName:}" failed. No retries permitted until 2026-09-08 15:18:17.400231923 +0000 UTC m=+50.112174392 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5551e120-e4db-4d93-8d2a-9e3626db076e-service-ca-bundle") pod "router-default-88497859b-8nkdj" (UID: "5551e120-e4db-4d93-8d2a-9e3626db076e") : configmap references non-existent config key: service-ca.crt
Sep 08 15:18:09.400458 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:09.400269    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 15:18:09.400458 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:09.400296    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-tls\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:09.400458 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:09.400330    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c64984d1-f8b2-414b-8f55-0a4d68e867cf-cluster-monitoring-operator-tls podName:c64984d1-f8b2-414b-8f55-0a4d68e867cf nodeName:}" failed. No retries permitted until 2026-09-08 15:18:17.40031428 +0000 UTC m=+50.112256766 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c64984d1-f8b2-414b-8f55-0a4d68e867cf-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-8dlwl" (UID: "c64984d1-f8b2-414b-8f55-0a4d68e867cf") : secret "cluster-monitoring-operator-tls" not found
Sep 08 15:18:09.400458 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:09.400368    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5551e120-e4db-4d93-8d2a-9e3626db076e-metrics-certs\") pod \"router-default-88497859b-8nkdj\" (UID: \"5551e120-e4db-4d93-8d2a-9e3626db076e\") " pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:09.400458 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:09.400376    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 15:18:09.400458 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:09.400389    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64f7cfd7bb-88t2j: secret "image-registry-tls" not found
Sep 08 15:18:09.400458 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:09.400426    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-tls podName:75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa nodeName:}" failed. No retries permitted until 2026-09-08 15:18:17.400415455 +0000 UTC m=+50.112357936 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-tls") pod "image-registry-64f7cfd7bb-88t2j" (UID: "75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa") : secret "image-registry-tls" not found
Sep 08 15:18:09.400458 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:09.400461    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 15:18:09.400787 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:09.400495    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5551e120-e4db-4d93-8d2a-9e3626db076e-metrics-certs podName:5551e120-e4db-4d93-8d2a-9e3626db076e nodeName:}" failed. No retries permitted until 2026-09-08 15:18:17.400484781 +0000 UTC m=+50.112427253 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5551e120-e4db-4d93-8d2a-9e3626db076e-metrics-certs") pod "router-default-88497859b-8nkdj" (UID: "5551e120-e4db-4d93-8d2a-9e3626db076e") : secret "router-metrics-certs-default" not found
Sep 08 15:18:09.501133 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:09.501100    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2df26fe4-655b-4248-a176-8028835f7379-metrics-tls\") pod \"dns-default-ct4vs\" (UID: \"2df26fe4-655b-4248-a176-8028835f7379\") " pod="openshift-dns/dns-default-ct4vs"
Sep 08 15:18:09.501310 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:09.501245    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 15:18:09.501310 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:09.501307    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2df26fe4-655b-4248-a176-8028835f7379-metrics-tls podName:2df26fe4-655b-4248-a176-8028835f7379 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:17.501293149 +0000 UTC m=+50.213235618 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2df26fe4-655b-4248-a176-8028835f7379-metrics-tls") pod "dns-default-ct4vs" (UID: "2df26fe4-655b-4248-a176-8028835f7379") : secret "dns-default-metrics-tls" not found
Sep 08 15:18:10.290490 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:10.290422    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:11.498456 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:11.498432    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:11.598458 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:11.598433    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:12.005272 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:12.004955    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jstb5" event={"ID":"202dffc4-2ade-40df-96be-4f6d20b0d998","Type":"ContainerStarted","Data":"6de0eedf47f9b9a3dcfa40cde49f1d3e9f30e09733c047767a4365eb4c5d5554"}
Sep 08 15:18:12.006804 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:12.006777    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pvz7x" event={"ID":"149a58aa-2b85-4704-b93b-1056c13054df","Type":"ContainerStarted","Data":"d029b29b48918eafb91765b8d00fe68e33bf7e27c012964076b531fa649fba16"}
Sep 08 15:18:12.009238 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:12.009214    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vc5n9_17e30810-dfd0-4e6f-9204-731d1a6479a3/console-operator/0.log"
Sep 08 15:18:12.009349 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:12.009251    2560 generic.go:356] "Generic (PLEG): container finished" podID="17e30810-dfd0-4e6f-9204-731d1a6479a3" containerID="5f8431dd29ccebc179f478e233155667fa0f88ce43694c38ee9129781a66a983" exitCode=255
Sep 08 15:18:12.009406 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:12.009343    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-vc5n9" event={"ID":"17e30810-dfd0-4e6f-9204-731d1a6479a3","Type":"ContainerDied","Data":"5f8431dd29ccebc179f478e233155667fa0f88ce43694c38ee9129781a66a983"}
Sep 08 15:18:12.009648 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:12.009631    2560 scope.go:117] "RemoveContainer" containerID="5f8431dd29ccebc179f478e233155667fa0f88ce43694c38ee9129781a66a983"
Sep 08 15:18:12.011094 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:12.011053    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-f67sn" event={"ID":"e073fcbf-44cc-445d-9f6f-669385f1d09d","Type":"ContainerStarted","Data":"5aad665d4ef18fcd7dac2263234783bed136ffeeedfef0483fe6b6832d91f786"}
Sep 08 15:18:12.012997 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:12.012969    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-dkwzw" event={"ID":"d4a10483-c29e-4a1c-9eb8-79d3197c1e35","Type":"ContainerStarted","Data":"82117695d7f282d676404e994bb47a03206ccb3a001a02fbd1b2dba9533b3c75"}
Sep 08 15:18:12.014714 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:12.014683    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z2r89" event={"ID":"12943199-b315-4e6a-ae18-cae8dc762efe","Type":"ContainerStarted","Data":"e627e0489f59ca5bf02cc0228e966a4765b83dcd8d927ccd540ad9a8c763c618"}
Sep 08 15:18:12.016251 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:12.016230    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-857nk" event={"ID":"21604064-e43d-4ec1-b6e9-9af32f6208ed","Type":"ContainerStarted","Data":"d35b466dc72d1837328ef6bf7e76a750a2e63deeaa0f6511c98cc769dd769452"}
Sep 08 15:18:12.027147 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:12.026875    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jstb5" podStartSLOduration=22.221741186 podStartE2EDuration="44.026858473s" totalImagesPullingTime="21.805117287s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:17:28 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:12.025577057 +0000 UTC m=+44.737519549" watchObservedRunningTime="2026-09-08 15:18:12.026858473 +0000 UTC m=+44.738800967"
Sep 08 15:18:12.042348 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:12.042303    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pvz7x" podStartSLOduration=18.910618329 podStartE2EDuration="28.04228588s" totalImagesPullingTime="9.131667551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:17:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:12.041120516 +0000 UTC m=+44.753063008" watchObservedRunningTime="2026-09-08 15:18:12.04228588 +0000 UTC m=+44.754228367"
Sep 08 15:18:12.058046 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:12.057996    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-z2r89" podStartSLOduration=1.962932421 podStartE2EDuration="11.057976871s" totalImagesPullingTime="9.09504445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:18:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:12.056378626 +0000 UTC m=+44.768321119" watchObservedRunningTime="2026-09-08 15:18:12.057976871 +0000 UTC m=+44.769919363"
Sep 08 15:18:12.073543 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:12.073418    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-857nk" podStartSLOduration=20.900386988 podStartE2EDuration="30.073388153s" totalImagesPullingTime="9.173001165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:17:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:12.072143686 +0000 UTC m=+44.784086178" watchObservedRunningTime="2026-09-08 15:18:12.073388153 +0000 UTC m=+44.785330644"
Sep 08 15:18:12.104602 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:12.104538    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-dkwzw" podStartSLOduration=26.97551561 podStartE2EDuration="36.10451901s" totalImagesPullingTime="9.1290034s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:12.093146885 +0000 UTC m=+44.805089378" watchObservedRunningTime="2026-09-08 15:18:12.10451901 +0000 UTC m=+44.816461501"
Sep 08 15:18:12.121434 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:12.121375    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-f67sn" podStartSLOduration=19.012496066 podStartE2EDuration="28.121356289s" totalImagesPullingTime="9.108860223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:17:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:12.119971052 +0000 UTC m=+44.831913544" watchObservedRunningTime="2026-09-08 15:18:12.121356289 +0000 UTC m=+44.833298783"
Sep 08 15:18:12.145930 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:12.145835    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:12.216654 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:12.216620    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:12.276547 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:12.276476    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:12.932115 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:12.932064    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pzl9w"]
Sep 08 15:18:12.935578 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:12.935552    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:12.948446 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:12.948422    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pzl9w"
Sep 08 15:18:12.949271 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:12.949249    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pzl9w"]
Sep 08 15:18:12.953023 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:12.952998    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wln7f\""
Sep 08 15:18:12.953023 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:12.952977    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 08 15:18:12.953316 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:12.953296    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 08 15:18:13.021343 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:13.021319    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vc5n9_17e30810-dfd0-4e6f-9204-731d1a6479a3/console-operator/1.log"
Sep 08 15:18:13.021790 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:13.021772    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vc5n9_17e30810-dfd0-4e6f-9204-731d1a6479a3/console-operator/0.log"
Sep 08 15:18:13.021881 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:13.021814    2560 generic.go:356] "Generic (PLEG): container finished" podID="17e30810-dfd0-4e6f-9204-731d1a6479a3" containerID="175a45ea2d12aa811f5642656db344e2743da4656cd091ce84e66db05aee5876" exitCode=255
Sep 08 15:18:13.021938 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:13.021917    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-vc5n9" event={"ID":"17e30810-dfd0-4e6f-9204-731d1a6479a3","Type":"ContainerDied","Data":"175a45ea2d12aa811f5642656db344e2743da4656cd091ce84e66db05aee5876"}
Sep 08 15:18:13.021991 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:13.021969    2560 scope.go:117] "RemoveContainer" containerID="5f8431dd29ccebc179f478e233155667fa0f88ce43694c38ee9129781a66a983"
Sep 08 15:18:13.022495 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:13.022305    2560 scope.go:117] "RemoveContainer" containerID="175a45ea2d12aa811f5642656db344e2743da4656cd091ce84e66db05aee5876"
Sep 08 15:18:13.022495 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:13.022477    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-vc5n9_openshift-console-operator(17e30810-dfd0-4e6f-9204-731d1a6479a3)\"" pod="openshift-console-operator/console-operator-85b454874b-vc5n9" podUID="17e30810-dfd0-4e6f-9204-731d1a6479a3"
Sep 08 15:18:13.033278 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:13.033259    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-data-volume\") pod \"insights-runtime-extractor-pzl9w\" (UID: \"dbacbcb2-59f9-4a4e-95a2-f42e2653eec4\") " pod="openshift-insights/insights-runtime-extractor-pzl9w"
Sep 08 15:18:13.033374 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:13.033298    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sbhj\" (UniqueName: \"kubernetes.io/projected/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-kube-api-access-6sbhj\") pod \"insights-runtime-extractor-pzl9w\" (UID: \"dbacbcb2-59f9-4a4e-95a2-f42e2653eec4\") " pod="openshift-insights/insights-runtime-extractor-pzl9w"
Sep 08 15:18:13.033374 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:13.033314    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-crio-socket\") pod \"insights-runtime-extractor-pzl9w\" (UID: \"dbacbcb2-59f9-4a4e-95a2-f42e2653eec4\") " pod="openshift-insights/insights-runtime-extractor-pzl9w"
Sep 08 15:18:13.033374 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:13.033371    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pzl9w\" (UID: \"dbacbcb2-59f9-4a4e-95a2-f42e2653eec4\") " pod="openshift-insights/insights-runtime-extractor-pzl9w"
Sep 08 15:18:13.033524 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:13.033500    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pzl9w\" (UID: \"dbacbcb2-59f9-4a4e-95a2-f42e2653eec4\") " pod="openshift-insights/insights-runtime-extractor-pzl9w"
Sep 08 15:18:13.051695 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:13.051670    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:13.134414 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:13.134376    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pzl9w\" (UID: \"dbacbcb2-59f9-4a4e-95a2-f42e2653eec4\") " pod="openshift-insights/insights-runtime-extractor-pzl9w"
Sep 08 15:18:13.134599 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:13.134504    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 15:18:13.134599 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:13.134553    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-insights-runtime-extractor-tls podName:dbacbcb2-59f9-4a4e-95a2-f42e2653eec4 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:13.634538975 +0000 UTC m=+46.346481466 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pzl9w" (UID: "dbacbcb2-59f9-4a4e-95a2-f42e2653eec4") : secret "insights-runtime-extractor-tls" not found
Sep 08 15:18:13.134933 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:13.134721    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-data-volume\") pod \"insights-runtime-extractor-pzl9w\" (UID: \"dbacbcb2-59f9-4a4e-95a2-f42e2653eec4\") " pod="openshift-insights/insights-runtime-extractor-pzl9w"
Sep 08 15:18:13.134933 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:13.134882    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sbhj\" (UniqueName: \"kubernetes.io/projected/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-kube-api-access-6sbhj\") pod \"insights-runtime-extractor-pzl9w\" (UID: \"dbacbcb2-59f9-4a4e-95a2-f42e2653eec4\") " pod="openshift-insights/insights-runtime-extractor-pzl9w"
Sep 08 15:18:13.134933 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:13.134920    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-crio-socket\") pod \"insights-runtime-extractor-pzl9w\" (UID: \"dbacbcb2-59f9-4a4e-95a2-f42e2653eec4\") " pod="openshift-insights/insights-runtime-extractor-pzl9w"
Sep 08 15:18:13.135190 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:13.135037    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pzl9w\" (UID: \"dbacbcb2-59f9-4a4e-95a2-f42e2653eec4\") " pod="openshift-insights/insights-runtime-extractor-pzl9w"
Sep 08 15:18:13.135190 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:13.135153    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-data-volume\") pod \"insights-runtime-extractor-pzl9w\" (UID: \"dbacbcb2-59f9-4a4e-95a2-f42e2653eec4\") " pod="openshift-insights/insights-runtime-extractor-pzl9w"
Sep 08 15:18:13.135308 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:13.135198    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-crio-socket\") pod \"insights-runtime-extractor-pzl9w\" (UID: \"dbacbcb2-59f9-4a4e-95a2-f42e2653eec4\") " pod="openshift-insights/insights-runtime-extractor-pzl9w"
Sep 08 15:18:13.135308 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:13.135270    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 15:18:13.135426 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:13.135321    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-kube-rbac-proxy-cm podName:dbacbcb2-59f9-4a4e-95a2-f42e2653eec4 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:13.635303627 +0000 UTC m=+46.347246103 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pzl9w" (UID: "dbacbcb2-59f9-4a4e-95a2-f42e2653eec4") : configmap "kube-rbac-proxy" not found
Sep 08 15:18:13.143272 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:13.143252    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sbhj\" (UniqueName: \"kubernetes.io/projected/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-kube-api-access-6sbhj\") pod \"insights-runtime-extractor-pzl9w\" (UID: \"dbacbcb2-59f9-4a4e-95a2-f42e2653eec4\") " pod="openshift-insights/insights-runtime-extractor-pzl9w"
Sep 08 15:18:13.291292 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:13.291260    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:13.639371 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:13.639290    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pzl9w\" (UID: \"dbacbcb2-59f9-4a4e-95a2-f42e2653eec4\") " pod="openshift-insights/insights-runtime-extractor-pzl9w"
Sep 08 15:18:13.639556 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:13.639394    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pzl9w\" (UID: \"dbacbcb2-59f9-4a4e-95a2-f42e2653eec4\") " pod="openshift-insights/insights-runtime-extractor-pzl9w"
Sep 08 15:18:13.639556 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:13.639481    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 15:18:13.639556 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:13.639513    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 15:18:13.639720 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:13.639565    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-insights-runtime-extractor-tls podName:dbacbcb2-59f9-4a4e-95a2-f42e2653eec4 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:14.639544658 +0000 UTC m=+47.351487144 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pzl9w" (UID: "dbacbcb2-59f9-4a4e-95a2-f42e2653eec4") : secret "insights-runtime-extractor-tls" not found
Sep 08 15:18:13.639720 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:13.639585    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-kube-rbac-proxy-cm podName:dbacbcb2-59f9-4a4e-95a2-f42e2653eec4 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:14.639576467 +0000 UTC m=+47.351518936 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pzl9w" (UID: "dbacbcb2-59f9-4a4e-95a2-f42e2653eec4") : configmap "kube-rbac-proxy" not found
Sep 08 15:18:14.026481 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:14.026453    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vc5n9_17e30810-dfd0-4e6f-9204-731d1a6479a3/console-operator/1.log"
Sep 08 15:18:14.026870 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:14.026777    2560 scope.go:117] "RemoveContainer" containerID="175a45ea2d12aa811f5642656db344e2743da4656cd091ce84e66db05aee5876"
Sep 08 15:18:14.026937 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:14.026917    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-vc5n9_openshift-console-operator(17e30810-dfd0-4e6f-9204-731d1a6479a3)\"" pod="openshift-console-operator/console-operator-85b454874b-vc5n9" podUID="17e30810-dfd0-4e6f-9204-731d1a6479a3"
Sep 08 15:18:14.585339 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:14.585307    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dvlfj_e26148ba-c65b-4e39-bd74-2f74c1ba38e6/dns-node-resolver/0.log"
Sep 08 15:18:14.647940 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:14.647903    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pzl9w\" (UID: \"dbacbcb2-59f9-4a4e-95a2-f42e2653eec4\") " pod="openshift-insights/insights-runtime-extractor-pzl9w"
Sep 08 15:18:14.648161 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:14.647993    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pzl9w\" (UID: \"dbacbcb2-59f9-4a4e-95a2-f42e2653eec4\") " pod="openshift-insights/insights-runtime-extractor-pzl9w"
Sep 08 15:18:14.648161 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:14.648027    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 15:18:14.648161 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:14.648109    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-kube-rbac-proxy-cm podName:dbacbcb2-59f9-4a4e-95a2-f42e2653eec4 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:16.648089826 +0000 UTC m=+49.360032303 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pzl9w" (UID: "dbacbcb2-59f9-4a4e-95a2-f42e2653eec4") : configmap "kube-rbac-proxy" not found
Sep 08 15:18:14.648161 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:14.648146    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 15:18:14.648379 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:14.648199    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-insights-runtime-extractor-tls podName:dbacbcb2-59f9-4a4e-95a2-f42e2653eec4 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:16.648185821 +0000 UTC m=+49.360128302 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pzl9w" (UID: "dbacbcb2-59f9-4a4e-95a2-f42e2653eec4") : secret "insights-runtime-extractor-tls" not found
Sep 08 15:18:14.968630 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:14.968577    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:15.032175 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:15.032128    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:15.393001 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:15.392915    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g7blt_762d10ee-64c5-40e1-896c-47e0cd55d613/node-ca/0.log"
Sep 08 15:18:15.592447 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:15.592414    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:15.651346 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:15.651273    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:16.385305 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:16.385277    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z2r89_12943199-b315-4e6a-ae18-cae8dc762efe/serve-healthcheck-canary/0.log"
Sep 08 15:18:16.597363 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:16.597331    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:16.664752 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:16.664669    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pzl9w\" (UID: \"dbacbcb2-59f9-4a4e-95a2-f42e2653eec4\") " pod="openshift-insights/insights-runtime-extractor-pzl9w"
Sep 08 15:18:16.664894 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:16.664802    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pzl9w\" (UID: \"dbacbcb2-59f9-4a4e-95a2-f42e2653eec4\") " pod="openshift-insights/insights-runtime-extractor-pzl9w"
Sep 08 15:18:16.664894 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:16.664811    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 15:18:16.665001 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:16.664839    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 15:18:16.665001 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:16.664943    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-insights-runtime-extractor-tls podName:dbacbcb2-59f9-4a4e-95a2-f42e2653eec4 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:20.664923579 +0000 UTC m=+53.376866065 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pzl9w" (UID: "dbacbcb2-59f9-4a4e-95a2-f42e2653eec4") : secret "insights-runtime-extractor-tls" not found
Sep 08 15:18:16.665001 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:16.664983    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-kube-rbac-proxy-cm podName:dbacbcb2-59f9-4a4e-95a2-f42e2653eec4 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:20.664966438 +0000 UTC m=+53.376908912 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pzl9w" (UID: "dbacbcb2-59f9-4a4e-95a2-f42e2653eec4") : configmap "kube-rbac-proxy" not found
Sep 08 15:18:17.472723 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:17.472686    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5551e120-e4db-4d93-8d2a-9e3626db076e-metrics-certs\") pod \"router-default-88497859b-8nkdj\" (UID: \"5551e120-e4db-4d93-8d2a-9e3626db076e\") " pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:17.473103 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:17.472739    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/590ddb7c-a30e-4ad5-9803-7ae4b2267d66-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-z6gr5\" (UID: \"590ddb7c-a30e-4ad5-9803-7ae4b2267d66\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-z6gr5"
Sep 08 15:18:17.473103 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:17.472833    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 15:18:17.473103 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:17.472835    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 15:18:17.473103 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:17.472897    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5551e120-e4db-4d93-8d2a-9e3626db076e-metrics-certs podName:5551e120-e4db-4d93-8d2a-9e3626db076e nodeName:}" failed. No retries permitted until 2026-09-08 15:18:33.472883026 +0000 UTC m=+66.184825495 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5551e120-e4db-4d93-8d2a-9e3626db076e-metrics-certs") pod "router-default-88497859b-8nkdj" (UID: "5551e120-e4db-4d93-8d2a-9e3626db076e") : secret "router-metrics-certs-default" not found
Sep 08 15:18:17.473103 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:17.472966    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/590ddb7c-a30e-4ad5-9803-7ae4b2267d66-samples-operator-tls podName:590ddb7c-a30e-4ad5-9803-7ae4b2267d66 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:33.472947918 +0000 UTC m=+66.184890387 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/590ddb7c-a30e-4ad5-9803-7ae4b2267d66-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-z6gr5" (UID: "590ddb7c-a30e-4ad5-9803-7ae4b2267d66") : secret "samples-operator-tls" not found
Sep 08 15:18:17.473103 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:17.473035    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c64984d1-f8b2-414b-8f55-0a4d68e867cf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-8dlwl\" (UID: \"c64984d1-f8b2-414b-8f55-0a4d68e867cf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8dlwl"
Sep 08 15:18:17.473103 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:17.473057    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5551e120-e4db-4d93-8d2a-9e3626db076e-service-ca-bundle\") pod \"router-default-88497859b-8nkdj\" (UID: \"5551e120-e4db-4d93-8d2a-9e3626db076e\") " pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:17.473343 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:17.473115    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-tls\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:17.473343 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:17.473154    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 15:18:17.473343 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:17.473183    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c64984d1-f8b2-414b-8f55-0a4d68e867cf-cluster-monitoring-operator-tls podName:c64984d1-f8b2-414b-8f55-0a4d68e867cf nodeName:}" failed. No retries permitted until 2026-09-08 15:18:33.473174207 +0000 UTC m=+66.185116675 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c64984d1-f8b2-414b-8f55-0a4d68e867cf-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-8dlwl" (UID: "c64984d1-f8b2-414b-8f55-0a4d68e867cf") : secret "cluster-monitoring-operator-tls" not found
Sep 08 15:18:17.473343 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:17.473206    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5551e120-e4db-4d93-8d2a-9e3626db076e-service-ca-bundle podName:5551e120-e4db-4d93-8d2a-9e3626db076e nodeName:}" failed. No retries permitted until 2026-09-08 15:18:33.473196111 +0000 UTC m=+66.185138584 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5551e120-e4db-4d93-8d2a-9e3626db076e-service-ca-bundle") pod "router-default-88497859b-8nkdj" (UID: "5551e120-e4db-4d93-8d2a-9e3626db076e") : configmap references non-existent config key: service-ca.crt
Sep 08 15:18:17.473343 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:17.473238    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 15:18:17.473343 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:17.473247    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64f7cfd7bb-88t2j: secret "image-registry-tls" not found
Sep 08 15:18:17.473343 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:17.473292    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-tls podName:75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa nodeName:}" failed. No retries permitted until 2026-09-08 15:18:33.473283396 +0000 UTC m=+66.185225866 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-tls") pod "image-registry-64f7cfd7bb-88t2j" (UID: "75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa") : secret "image-registry-tls" not found
Sep 08 15:18:17.573546 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:17.573513    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2df26fe4-655b-4248-a176-8028835f7379-metrics-tls\") pod \"dns-default-ct4vs\" (UID: \"2df26fe4-655b-4248-a176-8028835f7379\") " pod="openshift-dns/dns-default-ct4vs"
Sep 08 15:18:17.573680 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:17.573660    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 15:18:17.573745 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:17.573735    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2df26fe4-655b-4248-a176-8028835f7379-metrics-tls podName:2df26fe4-655b-4248-a176-8028835f7379 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:33.573720763 +0000 UTC m=+66.285663238 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2df26fe4-655b-4248-a176-8028835f7379-metrics-tls") pod "dns-default-ct4vs" (UID: "2df26fe4-655b-4248-a176-8028835f7379") : secret "dns-default-metrics-tls" not found
Sep 08 15:18:17.628740 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:17.628707    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:19.614884 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:19.614849    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:20.540854 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:20.540821    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:20.699615 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:20.699580    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pzl9w\" (UID: \"dbacbcb2-59f9-4a4e-95a2-f42e2653eec4\") " pod="openshift-insights/insights-runtime-extractor-pzl9w"
Sep 08 15:18:20.699984 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:20.699748    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 15:18:20.699984 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:20.699800    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pzl9w\" (UID: \"dbacbcb2-59f9-4a4e-95a2-f42e2653eec4\") " pod="openshift-insights/insights-runtime-extractor-pzl9w"
Sep 08 15:18:20.699984 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:20.699835    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-insights-runtime-extractor-tls podName:dbacbcb2-59f9-4a4e-95a2-f42e2653eec4 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:28.699814874 +0000 UTC m=+61.411757348 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pzl9w" (UID: "dbacbcb2-59f9-4a4e-95a2-f42e2653eec4") : secret "insights-runtime-extractor-tls" not found
Sep 08 15:18:20.700295 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:20.700278    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pzl9w\" (UID: \"dbacbcb2-59f9-4a4e-95a2-f42e2653eec4\") " pod="openshift-insights/insights-runtime-extractor-pzl9w"
Sep 08 15:18:21.030348 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:21.030315    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:21.138187 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:21.138153    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:21.242632 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:21.242602    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:21.417994 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:21.417921    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:21.580713 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:21.580679    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:21.689137 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:21.689107    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:21.951042 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:21.950940    2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-vc5n9"
Sep 08 15:18:21.951042 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:21.951015    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-vc5n9"
Sep 08 15:18:21.951654 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:21.951477    2560 scope.go:117] "RemoveContainer" containerID="175a45ea2d12aa811f5642656db344e2743da4656cd091ce84e66db05aee5876"
Sep 08 15:18:21.951713 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:21.951681    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-vc5n9_openshift-console-operator(17e30810-dfd0-4e6f-9204-731d1a6479a3)\"" pod="openshift-console-operator/console-operator-85b454874b-vc5n9" podUID="17e30810-dfd0-4e6f-9204-731d1a6479a3"
Sep 08 15:18:22.004450 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:22.004418    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:22.809333 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:22.809297    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:23.207004 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:23.206953    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:23.410574 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:23.410537    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:23.612535 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:23.612466    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:25.280869 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:25.280811    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:25.380582 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:25.380522    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:25.482275 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:25.482241    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:25.622171 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:25.622098    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:25.822542 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:25.822512    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:26.808806 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:26.808767    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:26.929981 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:26.929955    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rgrvb"
Sep 08 15:18:28.036195 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:28.036157    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:28.134168 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:28.134134    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:28.654928 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:28.654895    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:28.773378 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:28.773346    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pzl9w\" (UID: \"dbacbcb2-59f9-4a4e-95a2-f42e2653eec4\") " pod="openshift-insights/insights-runtime-extractor-pzl9w"
Sep 08 15:18:28.775673 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:28.775651    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dbacbcb2-59f9-4a4e-95a2-f42e2653eec4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pzl9w\" (UID: \"dbacbcb2-59f9-4a4e-95a2-f42e2653eec4\") " pod="openshift-insights/insights-runtime-extractor-pzl9w"
Sep 08 15:18:28.821351 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:28.821318    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:28.862046 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:28.862021    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wln7f\""
Sep 08 15:18:28.870654 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:28.870631    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pzl9w"
Sep 08 15:18:29.144515 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:29.144446    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pzl9w"]
Sep 08 15:18:29.147166 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:18:29.147129    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbacbcb2_59f9_4a4e_95a2_f42e2653eec4.slice/crio-63b41f4c80d388b141bf6e7bee294d3512b1ca703bc57214108a857737ed20ca WatchSource:0}: Error finding container 63b41f4c80d388b141bf6e7bee294d3512b1ca703bc57214108a857737ed20ca: Status 404 returned error can't find the container with id 63b41f4c80d388b141bf6e7bee294d3512b1ca703bc57214108a857737ed20ca
Sep 08 15:18:29.208285 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:29.208262    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:29.407795 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:29.407763    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:29.610496 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:29.610415    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:29.817631 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:29.817602    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:30.077128 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:30.077096    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pzl9w" event={"ID":"dbacbcb2-59f9-4a4e-95a2-f42e2653eec4","Type":"ContainerStarted","Data":"15900db85548f166c221fa2c167078bcc7dde88c0d56dd3ccd36d31a15292f77"}
Sep 08 15:18:30.077218 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:30.077156    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pzl9w" event={"ID":"dbacbcb2-59f9-4a4e-95a2-f42e2653eec4","Type":"ContainerStarted","Data":"63b41f4c80d388b141bf6e7bee294d3512b1ca703bc57214108a857737ed20ca"}
Sep 08 15:18:30.806787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:30.806749    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:31.008973 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:31.008937    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:31.082695 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:31.082611    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pzl9w" event={"ID":"dbacbcb2-59f9-4a4e-95a2-f42e2653eec4","Type":"ContainerStarted","Data":"7b2d78610390e48e74181762d420233db9c982bfe7787de1d9d2172d4ed830d0"}
Sep 08 15:18:31.206923 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:31.206876    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:31.408999 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:31.408974    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:32.087444 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:32.087405    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pzl9w" event={"ID":"dbacbcb2-59f9-4a4e-95a2-f42e2653eec4","Type":"ContainerStarted","Data":"636e2cd6ba29fcf3e77074fa42f22f13878268773b2dbd806ca58f00ef8fd542"}
Sep 08 15:18:33.512256 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.512200    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c64984d1-f8b2-414b-8f55-0a4d68e867cf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-8dlwl\" (UID: \"c64984d1-f8b2-414b-8f55-0a4d68e867cf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8dlwl"
Sep 08 15:18:33.512256 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.512254    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5551e120-e4db-4d93-8d2a-9e3626db076e-service-ca-bundle\") pod \"router-default-88497859b-8nkdj\" (UID: \"5551e120-e4db-4d93-8d2a-9e3626db076e\") " pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:33.512810 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.512284    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-tls\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:33.512810 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.512307    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5551e120-e4db-4d93-8d2a-9e3626db076e-metrics-certs\") pod \"router-default-88497859b-8nkdj\" (UID: \"5551e120-e4db-4d93-8d2a-9e3626db076e\") " pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:33.512810 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.512330    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/590ddb7c-a30e-4ad5-9803-7ae4b2267d66-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-z6gr5\" (UID: \"590ddb7c-a30e-4ad5-9803-7ae4b2267d66\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-z6gr5"
Sep 08 15:18:33.512810 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.512371    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/443fd4ee-6833-4ac4-a15f-d4ebdf0925d5-metrics-certs\") pod \"network-metrics-daemon-dv2mt\" (UID: \"443fd4ee-6833-4ac4-a15f-d4ebdf0925d5\") " pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:18:33.513060 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.513032    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5551e120-e4db-4d93-8d2a-9e3626db076e-service-ca-bundle\") pod \"router-default-88497859b-8nkdj\" (UID: \"5551e120-e4db-4d93-8d2a-9e3626db076e\") " pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:33.514805 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.514771    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-tls\") pod \"image-registry-64f7cfd7bb-88t2j\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") " pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:33.514805 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.514794    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/590ddb7c-a30e-4ad5-9803-7ae4b2267d66-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-z6gr5\" (UID: \"590ddb7c-a30e-4ad5-9803-7ae4b2267d66\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-z6gr5"
Sep 08 15:18:33.515119 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.515100    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 08 15:18:33.515192 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.515147    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5551e120-e4db-4d93-8d2a-9e3626db076e-metrics-certs\") pod \"router-default-88497859b-8nkdj\" (UID: \"5551e120-e4db-4d93-8d2a-9e3626db076e\") " pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:33.515192 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.515160    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c64984d1-f8b2-414b-8f55-0a4d68e867cf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-8dlwl\" (UID: \"c64984d1-f8b2-414b-8f55-0a4d68e867cf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8dlwl"
Sep 08 15:18:33.524586 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.524563    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/443fd4ee-6833-4ac4-a15f-d4ebdf0925d5-metrics-certs\") pod \"network-metrics-daemon-dv2mt\" (UID: \"443fd4ee-6833-4ac4-a15f-d4ebdf0925d5\") " pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:18:33.608181 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.608155    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-76z9f\""
Sep 08 15:18:33.612954 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.612929    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk4dj\" (UniqueName: \"kubernetes.io/projected/31fb2a24-8166-40fc-b65f-4e3bbcdba36c-kube-api-access-gk4dj\") pod \"network-check-target-mb5jg\" (UID: \"31fb2a24-8166-40fc-b65f-4e3bbcdba36c\") " pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:18:33.613085 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.612968    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2df26fe4-655b-4248-a176-8028835f7379-metrics-tls\") pod \"dns-default-ct4vs\" (UID: \"2df26fe4-655b-4248-a176-8028835f7379\") " pod="openshift-dns/dns-default-ct4vs"
Sep 08 15:18:33.613085 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.612986    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b05229d-340e-424a-8533-87c77e6b97d5-original-pull-secret\") pod \"global-pull-secret-syncer-4wzj5\" (UID: \"4b05229d-340e-424a-8533-87c77e6b97d5\") " pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:18:33.615021 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.614998    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 08 15:18:33.615376 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.615357    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2df26fe4-655b-4248-a176-8028835f7379-metrics-tls\") pod \"dns-default-ct4vs\" (UID: \"2df26fe4-655b-4248-a176-8028835f7379\") " pod="openshift-dns/dns-default-ct4vs"
Sep 08 15:18:33.615489 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.615469    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gk4dj\" (UniqueName: \"kubernetes.io/projected/31fb2a24-8166-40fc-b65f-4e3bbcdba36c-kube-api-access-gk4dj\") pod \"network-check-target-mb5jg\" (UID: \"31fb2a24-8166-40fc-b65f-4e3bbcdba36c\") " pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:18:33.616186 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.616170    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dv2mt"
Sep 08 15:18:33.625022 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.625003    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b05229d-340e-424a-8533-87c77e6b97d5-original-pull-secret\") pod \"global-pull-secret-syncer-4wzj5\" (UID: \"4b05229d-340e-424a-8533-87c77e6b97d5\") " pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:18:33.704452 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.704410    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wzj5"
Sep 08 15:18:33.717064 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.717040    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-smd6z\""
Sep 08 15:18:33.724755 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.724732    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:33.725019 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.725001    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-htbkw\""
Sep 08 15:18:33.734101 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.734059    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-z6gr5"
Sep 08 15:18:33.745453 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.745426    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-s5fmp\""
Sep 08 15:18:33.753529 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.753504    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8dlwl"
Sep 08 15:18:33.779699 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.779673    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-tmffq\""
Sep 08 15:18:33.787537 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.787485    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:33.797182 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.796579    2560 scope.go:117] "RemoveContainer" containerID="175a45ea2d12aa811f5642656db344e2743da4656cd091ce84e66db05aee5876"
Sep 08 15:18:33.809198 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.809160    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-65z9v\""
Sep 08 15:18:33.817878 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.817667    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ct4vs"
Sep 08 15:18:33.890994 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.890517    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pzl9w" podStartSLOduration=19.764166166 podStartE2EDuration="21.890491669s" totalImagesPullingTime="2.126325503s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:18:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:32.121849093 +0000 UTC m=+64.833791584" watchObservedRunningTime="2026-09-08 15:18:33.890491669 +0000 UTC m=+66.602434162"
Sep 08 15:18:33.891260 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.891180    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dv2mt"]
Sep 08 15:18:33.897723 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:18:33.897682    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod443fd4ee_6833_4ac4_a15f_d4ebdf0925d5.slice/crio-f7b647fcd1358e433a640e87f671cf14dc513a3a7315dc9e9f78a4e73be94efd WatchSource:0}: Error finding container f7b647fcd1358e433a640e87f671cf14dc513a3a7315dc9e9f78a4e73be94efd: Status 404 returned error can't find the container with id f7b647fcd1358e433a640e87f671cf14dc513a3a7315dc9e9f78a4e73be94efd
Sep 08 15:18:33.914065 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.914009    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-np4gb\""
Sep 08 15:18:33.921989 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.921968    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:18:33.981099 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:18:33.981061    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b05229d_340e_424a_8533_87c77e6b97d5.slice/crio-8b5ac29d1d45bcaf06f70480cdddc3064afb1adb19d6434257d18342f1b2b09a WatchSource:0}: Error finding container 8b5ac29d1d45bcaf06f70480cdddc3064afb1adb19d6434257d18342f1b2b09a: Status 404 returned error can't find the container with id 8b5ac29d1d45bcaf06f70480cdddc3064afb1adb19d6434257d18342f1b2b09a
Sep 08 15:18:33.984595 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:33.984492    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4wzj5"]
Sep 08 15:18:34.012019 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:34.011979    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-z6gr5"]
Sep 08 15:18:34.015008 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:34.014983    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64f7cfd7bb-88t2j"]
Sep 08 15:18:34.017787 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:18:34.017735    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75c27ee1_bb0d_4b1e_bb71_c7be7e5f7baa.slice/crio-eb48de1533647d9c23d89990afcee032b06da331c5a169b69f21d9fe5232ae09 WatchSource:0}: Error finding container eb48de1533647d9c23d89990afcee032b06da331c5a169b69f21d9fe5232ae09: Status 404 returned error can't find the container with id eb48de1533647d9c23d89990afcee032b06da331c5a169b69f21d9fe5232ae09
Sep 08 15:18:34.044777 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:34.044715    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8dlwl"]
Sep 08 15:18:34.049056 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:18:34.049033    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc64984d1_f8b2_414b_8f55_0a4d68e867cf.slice/crio-a5481d4f5cdb38991d30f4615aa4072304992d32f1b5549155ad0e1ac1958119 WatchSource:0}: Error finding container a5481d4f5cdb38991d30f4615aa4072304992d32f1b5549155ad0e1ac1958119: Status 404 returned error can't find the container with id a5481d4f5cdb38991d30f4615aa4072304992d32f1b5549155ad0e1ac1958119
Sep 08 15:18:34.090887 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:34.090851    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-88497859b-8nkdj"]
Sep 08 15:18:34.093646 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:18:34.093617    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5551e120_e4db_4d93_8d2a_9e3626db076e.slice/crio-b89da32c0159e2879631331c3fb82312eaa90eb5c1fb9da827c534087ae2fadd WatchSource:0}: Error finding container b89da32c0159e2879631331c3fb82312eaa90eb5c1fb9da827c534087ae2fadd: Status 404 returned error can't find the container with id b89da32c0159e2879631331c3fb82312eaa90eb5c1fb9da827c534087ae2fadd
Sep 08 15:18:34.098823 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:34.098793    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4wzj5" event={"ID":"4b05229d-340e-424a-8533-87c77e6b97d5","Type":"ContainerStarted","Data":"8b5ac29d1d45bcaf06f70480cdddc3064afb1adb19d6434257d18342f1b2b09a"}
Sep 08 15:18:34.100378 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:34.100364    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vc5n9_17e30810-dfd0-4e6f-9204-731d1a6479a3/console-operator/2.log"
Sep 08 15:18:34.100753 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:34.100739    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vc5n9_17e30810-dfd0-4e6f-9204-731d1a6479a3/console-operator/1.log"
Sep 08 15:18:34.100812 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:34.100769    2560 generic.go:356] "Generic (PLEG): container finished" podID="17e30810-dfd0-4e6f-9204-731d1a6479a3" containerID="5dd171b161d7a73e2133b2d1513997ae1cddb0270976889a37a2527b18ef37d7" exitCode=255
Sep 08 15:18:34.100881 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:34.100854    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-vc5n9" event={"ID":"17e30810-dfd0-4e6f-9204-731d1a6479a3","Type":"ContainerDied","Data":"5dd171b161d7a73e2133b2d1513997ae1cddb0270976889a37a2527b18ef37d7"}
Sep 08 15:18:34.100943 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:34.100922    2560 scope.go:117] "RemoveContainer" containerID="175a45ea2d12aa811f5642656db344e2743da4656cd091ce84e66db05aee5876"
Sep 08 15:18:34.101317 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:34.101295    2560 scope.go:117] "RemoveContainer" containerID="5dd171b161d7a73e2133b2d1513997ae1cddb0270976889a37a2527b18ef37d7"
Sep 08 15:18:34.101501 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:34.101478    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-vc5n9_openshift-console-operator(17e30810-dfd0-4e6f-9204-731d1a6479a3)\"" pod="openshift-console-operator/console-operator-85b454874b-vc5n9" podUID="17e30810-dfd0-4e6f-9204-731d1a6479a3"
Sep 08 15:18:34.102573 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:34.102548    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j" event={"ID":"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa","Type":"ContainerStarted","Data":"0a4821b3107543a9f81382d9e74f2ec090eb2773084c58ce9051a8652b2dc467"}
Sep 08 15:18:34.102669 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:34.102587    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j" event={"ID":"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa","Type":"ContainerStarted","Data":"eb48de1533647d9c23d89990afcee032b06da331c5a169b69f21d9fe5232ae09"}
Sep 08 15:18:34.102669 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:34.102640    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:34.103747 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:34.103717    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8dlwl" event={"ID":"c64984d1-f8b2-414b-8f55-0a4d68e867cf","Type":"ContainerStarted","Data":"a5481d4f5cdb38991d30f4615aa4072304992d32f1b5549155ad0e1ac1958119"}
Sep 08 15:18:34.104886 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:34.104864    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-z6gr5" event={"ID":"590ddb7c-a30e-4ad5-9803-7ae4b2267d66","Type":"ContainerStarted","Data":"344e138e8d7486fe6bd7b0aa368451cf826bcfee53847edcddd87bb6b5b6b7ad"}
Sep 08 15:18:34.105992 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:34.105946    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dv2mt" event={"ID":"443fd4ee-6833-4ac4-a15f-d4ebdf0925d5","Type":"ContainerStarted","Data":"f7b647fcd1358e433a640e87f671cf14dc513a3a7315dc9e9f78a4e73be94efd"}
Sep 08 15:18:34.107176 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:34.107152    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-88497859b-8nkdj" event={"ID":"5551e120-e4db-4d93-8d2a-9e3626db076e","Type":"ContainerStarted","Data":"b89da32c0159e2879631331c3fb82312eaa90eb5c1fb9da827c534087ae2fadd"}
Sep 08 15:18:34.115587 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:34.115562    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ct4vs"]
Sep 08 15:18:34.124190 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:18:34.124161    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2df26fe4_655b_4248_a176_8028835f7379.slice/crio-74062a6ff0552c2a4f904415788b8660af4da7ceff0f229efbccdb2693881c1c WatchSource:0}: Error finding container 74062a6ff0552c2a4f904415788b8660af4da7ceff0f229efbccdb2693881c1c: Status 404 returned error can't find the container with id 74062a6ff0552c2a4f904415788b8660af4da7ceff0f229efbccdb2693881c1c
Sep 08 15:18:34.141919 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:34.141873    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j" podStartSLOduration=65.141857147 podStartE2EDuration="1m5.141857147s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:17:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:34.141480637 +0000 UTC m=+66.853423129" watchObservedRunningTime="2026-09-08 15:18:34.141857147 +0000 UTC m=+66.853799650"
Sep 08 15:18:34.189800 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:34.189768    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mb5jg"]
Sep 08 15:18:34.193678 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:18:34.193649    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31fb2a24_8166_40fc_b65f_4e3bbcdba36c.slice/crio-52492bda0e3fd3b8f4ace78a3dae935af9a08fdb6830f06b606c61b4efc66409 WatchSource:0}: Error finding container 52492bda0e3fd3b8f4ace78a3dae935af9a08fdb6830f06b606c61b4efc66409: Status 404 returned error can't find the container with id 52492bda0e3fd3b8f4ace78a3dae935af9a08fdb6830f06b606c61b4efc66409
Sep 08 15:18:35.115121 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:35.114628    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-88497859b-8nkdj" event={"ID":"5551e120-e4db-4d93-8d2a-9e3626db076e","Type":"ContainerStarted","Data":"c394d9487aed57b05f183d4e68a2b0ed7fae900d3cf697d1fd10471d1fc85962"}
Sep 08 15:18:35.117171 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:35.117047    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vc5n9_17e30810-dfd0-4e6f-9204-731d1a6479a3/console-operator/2.log"
Sep 08 15:18:35.120031 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:35.119922    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mb5jg" event={"ID":"31fb2a24-8166-40fc-b65f-4e3bbcdba36c","Type":"ContainerStarted","Data":"ae15282ba7903da93ca605970dc44bade078685a2c03f96e0419c369c824f845"}
Sep 08 15:18:35.120031 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:35.119983    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mb5jg" event={"ID":"31fb2a24-8166-40fc-b65f-4e3bbcdba36c","Type":"ContainerStarted","Data":"52492bda0e3fd3b8f4ace78a3dae935af9a08fdb6830f06b606c61b4efc66409"}
Sep 08 15:18:35.120031 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:35.120019    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:18:35.121301 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:35.121277    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ct4vs" event={"ID":"2df26fe4-655b-4248-a176-8028835f7379","Type":"ContainerStarted","Data":"74062a6ff0552c2a4f904415788b8660af4da7ceff0f229efbccdb2693881c1c"}
Sep 08 15:18:35.124324 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:35.124266    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dv2mt" event={"ID":"443fd4ee-6833-4ac4-a15f-d4ebdf0925d5","Type":"ContainerStarted","Data":"e0540fd20c8b0ff1f47f65f917fa74cdccd0297e0d043269e218f45fc781fac4"}
Sep 08 15:18:35.137313 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:35.137228    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-88497859b-8nkdj" podStartSLOduration=59.137210375 podStartE2EDuration="59.137210375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:17:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:35.135407363 +0000 UTC m=+67.847349854" watchObservedRunningTime="2026-09-08 15:18:35.137210375 +0000 UTC m=+67.849152867"
Sep 08 15:18:35.155796 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:35.155464    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mb5jg" podStartSLOduration=67.155445851 podStartE2EDuration="1m7.155445851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:17:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:35.155286814 +0000 UTC m=+67.867229306" watchObservedRunningTime="2026-09-08 15:18:35.155445851 +0000 UTC m=+67.867388335"
Sep 08 15:18:35.788424 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:35.788389    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:35.791416 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:35.791388    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:35.936934 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:35.936899    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97658db95-7ngrj"]
Sep 08 15:18:35.941939 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:35.941912    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97658db95-7ngrj"
Sep 08 15:18:35.944703 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:35.944675    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 08 15:18:35.945206 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:35.945017    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 08 15:18:35.945206 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:35.945129    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 08 15:18:35.945392 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:35.945277    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 08 15:18:35.945392 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:35.945304    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 08 15:18:35.945392 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:35.945329    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 08 15:18:35.945552 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:35.945431    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 08 15:18:35.953926 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:35.953899    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97658db95-7ngrj"]
Sep 08 15:18:36.036219 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.036165    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f80123cf-9974-443c-81a6-0b2d66ef186b-ca\") pod \"cluster-proxy-proxy-agent-97658db95-7ngrj\" (UID: \"f80123cf-9974-443c-81a6-0b2d66ef186b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97658db95-7ngrj"
Sep 08 15:18:36.036219 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.036222    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f80123cf-9974-443c-81a6-0b2d66ef186b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-97658db95-7ngrj\" (UID: \"f80123cf-9974-443c-81a6-0b2d66ef186b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97658db95-7ngrj"
Sep 08 15:18:36.036471 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.036273    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f80123cf-9974-443c-81a6-0b2d66ef186b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-97658db95-7ngrj\" (UID: \"f80123cf-9974-443c-81a6-0b2d66ef186b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97658db95-7ngrj"
Sep 08 15:18:36.036471 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.036333    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f80123cf-9974-443c-81a6-0b2d66ef186b-hub\") pod \"cluster-proxy-proxy-agent-97658db95-7ngrj\" (UID: \"f80123cf-9974-443c-81a6-0b2d66ef186b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97658db95-7ngrj"
Sep 08 15:18:36.036471 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.036361    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f80123cf-9974-443c-81a6-0b2d66ef186b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-97658db95-7ngrj\" (UID: \"f80123cf-9974-443c-81a6-0b2d66ef186b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97658db95-7ngrj"
Sep 08 15:18:36.036471 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.036384    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qllkh\" (UniqueName: \"kubernetes.io/projected/f80123cf-9974-443c-81a6-0b2d66ef186b-kube-api-access-qllkh\") pod \"cluster-proxy-proxy-agent-97658db95-7ngrj\" (UID: \"f80123cf-9974-443c-81a6-0b2d66ef186b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97658db95-7ngrj"
Sep 08 15:18:36.064544 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.064455    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:36.131082 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.131039    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dv2mt" event={"ID":"443fd4ee-6833-4ac4-a15f-d4ebdf0925d5","Type":"ContainerStarted","Data":"345b776df4201e38a21c87c0d6232125595879b8e0878ba1dcac074a54427fd6"}
Sep 08 15:18:36.131535 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.131518    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:36.132818 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.132795    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-88497859b-8nkdj"
Sep 08 15:18:36.137390 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.137367    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f80123cf-9974-443c-81a6-0b2d66ef186b-hub\") pod \"cluster-proxy-proxy-agent-97658db95-7ngrj\" (UID: \"f80123cf-9974-443c-81a6-0b2d66ef186b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97658db95-7ngrj"
Sep 08 15:18:36.137486 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.137405    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f80123cf-9974-443c-81a6-0b2d66ef186b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-97658db95-7ngrj\" (UID: \"f80123cf-9974-443c-81a6-0b2d66ef186b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97658db95-7ngrj"
Sep 08 15:18:36.137486 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.137424    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qllkh\" (UniqueName: \"kubernetes.io/projected/f80123cf-9974-443c-81a6-0b2d66ef186b-kube-api-access-qllkh\") pod \"cluster-proxy-proxy-agent-97658db95-7ngrj\" (UID: \"f80123cf-9974-443c-81a6-0b2d66ef186b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97658db95-7ngrj"
Sep 08 15:18:36.137486 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.137464    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f80123cf-9974-443c-81a6-0b2d66ef186b-ca\") pod \"cluster-proxy-proxy-agent-97658db95-7ngrj\" (UID: \"f80123cf-9974-443c-81a6-0b2d66ef186b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97658db95-7ngrj"
Sep 08 15:18:36.137650 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.137487    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f80123cf-9974-443c-81a6-0b2d66ef186b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-97658db95-7ngrj\" (UID: \"f80123cf-9974-443c-81a6-0b2d66ef186b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97658db95-7ngrj"
Sep 08 15:18:36.137650 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.137514    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f80123cf-9974-443c-81a6-0b2d66ef186b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-97658db95-7ngrj\" (UID: \"f80123cf-9974-443c-81a6-0b2d66ef186b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97658db95-7ngrj"
Sep 08 15:18:36.138330 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.138307    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f80123cf-9974-443c-81a6-0b2d66ef186b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-97658db95-7ngrj\" (UID: \"f80123cf-9974-443c-81a6-0b2d66ef186b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97658db95-7ngrj"
Sep 08 15:18:36.140852 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.140829    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f80123cf-9974-443c-81a6-0b2d66ef186b-ca\") pod \"cluster-proxy-proxy-agent-97658db95-7ngrj\" (UID: \"f80123cf-9974-443c-81a6-0b2d66ef186b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97658db95-7ngrj"
Sep 08 15:18:36.140934 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.140853    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f80123cf-9974-443c-81a6-0b2d66ef186b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-97658db95-7ngrj\" (UID: \"f80123cf-9974-443c-81a6-0b2d66ef186b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97658db95-7ngrj"
Sep 08 15:18:36.140934 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.140905    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f80123cf-9974-443c-81a6-0b2d66ef186b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-97658db95-7ngrj\" (UID: \"f80123cf-9974-443c-81a6-0b2d66ef186b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97658db95-7ngrj"
Sep 08 15:18:36.141351 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.141314    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f80123cf-9974-443c-81a6-0b2d66ef186b-hub\") pod \"cluster-proxy-proxy-agent-97658db95-7ngrj\" (UID: \"f80123cf-9974-443c-81a6-0b2d66ef186b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97658db95-7ngrj"
Sep 08 15:18:36.147877 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.147838    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qllkh\" (UniqueName: \"kubernetes.io/projected/f80123cf-9974-443c-81a6-0b2d66ef186b-kube-api-access-qllkh\") pod \"cluster-proxy-proxy-agent-97658db95-7ngrj\" (UID: \"f80123cf-9974-443c-81a6-0b2d66ef186b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97658db95-7ngrj"
Sep 08 15:18:36.149199 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.149153    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dv2mt" podStartSLOduration=67.076789658 podStartE2EDuration="1m8.149136286s" totalImagesPullingTime="1.072346628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:17:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:36.147471187 +0000 UTC m=+68.859413685" watchObservedRunningTime="2026-09-08 15:18:36.149136286 +0000 UTC m=+68.861078779"
Sep 08 15:18:36.161764 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.161735    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:36.243814 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.243774    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:36.260441 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.260409    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97658db95-7ngrj"
Sep 08 15:18:36.343994 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.343897    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:36.443380 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.443350    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:36.735769 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.735564    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97658db95-7ngrj"]
Sep 08 15:18:36.739263 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:18:36.739228    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf80123cf_9974_443c_81a6_0b2d66ef186b.slice/crio-eb6b6ceaa7c179605dd304f92239432af6dcff27527e375c26e6948f894b0dd2 WatchSource:0}: Error finding container eb6b6ceaa7c179605dd304f92239432af6dcff27527e375c26e6948f894b0dd2: Status 404 returned error can't find the container with id eb6b6ceaa7c179605dd304f92239432af6dcff27527e375c26e6948f894b0dd2
Sep 08 15:18:36.850234 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.850114    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:36.948914 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:36.948886    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:37.038633 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:37.038593    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:37.106810 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:37.106769    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:37.137191 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:37.137152    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97658db95-7ngrj" event={"ID":"f80123cf-9974-443c-81a6-0b2d66ef186b","Type":"ContainerStarted","Data":"eb6b6ceaa7c179605dd304f92239432af6dcff27527e375c26e6948f894b0dd2"}
Sep 08 15:18:37.139400 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:37.139306    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8dlwl" event={"ID":"c64984d1-f8b2-414b-8f55-0a4d68e867cf","Type":"ContainerStarted","Data":"154a69e84fe2d1ec7b45e85a9e30a340ed3502f61d76e5dce9efe1936e30bfde"}
Sep 08 15:18:37.141943 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:37.141919    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-z6gr5" event={"ID":"590ddb7c-a30e-4ad5-9803-7ae4b2267d66","Type":"ContainerStarted","Data":"12fa7a5fb586fe4570d2112b86dfe3747a76a0f657286a66e67a7188c1560008"}
Sep 08 15:18:37.142057 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:37.141968    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-z6gr5" event={"ID":"590ddb7c-a30e-4ad5-9803-7ae4b2267d66","Type":"ContainerStarted","Data":"52d45fd1ca38a5ab52003b5d453232212048517fa704022c1e2e0a67839231fd"}
Sep 08 15:18:37.161575 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:37.161520    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8dlwl" podStartSLOduration=58.75276315 podStartE2EDuration="1m1.161502714s" totalImagesPullingTime="2.408739564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:37.160411073 +0000 UTC m=+69.872353565" watchObservedRunningTime="2026-09-08 15:18:37.161502714 +0000 UTC m=+69.873445206"
Sep 08 15:18:37.178733 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:37.178683    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-z6gr5" podStartSLOduration=58.77372179 podStartE2EDuration="1m1.178660109s" totalImagesPullingTime="2.404938319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:37.177131161 +0000 UTC m=+69.889073652" watchObservedRunningTime="2026-09-08 15:18:37.178660109 +0000 UTC m=+69.890602600"
Sep 08 15:18:37.202766 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:37.202678    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:37.379008 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:37.378974    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:37.438336 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:37.438267    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:38.149237 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:38.149093    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ct4vs" event={"ID":"2df26fe4-655b-4248-a176-8028835f7379","Type":"ContainerStarted","Data":"0112ee3f27ce32f57548d66fe6f8d6c6b19a07d7dfc44375519e6a5e99c3a52e"}
Sep 08 15:18:38.149237 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:38.149196    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ct4vs" event={"ID":"2df26fe4-655b-4248-a176-8028835f7379","Type":"ContainerStarted","Data":"f1989eda787ef5335cf22e45d60148f3166edc44220e5ce7d03ed617f472f0d3"}
Sep 08 15:18:38.167695 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:38.167106    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-ct4vs" podStartSLOduration=33.639237708 podStartE2EDuration="37.167085927s" totalImagesPullingTime="3.527848219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:18:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:38.165685916 +0000 UTC m=+70.877628409" watchObservedRunningTime="2026-09-08 15:18:38.167085927 +0000 UTC m=+70.879028420"
Sep 08 15:18:39.154729 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:39.154676    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ct4vs"
Sep 08 15:18:41.951768 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:41.951728    2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-vc5n9"
Sep 08 15:18:41.952188 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:41.951799    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-vc5n9"
Sep 08 15:18:41.952254 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:41.952229    2560 scope.go:117] "RemoveContainer" containerID="5dd171b161d7a73e2133b2d1513997ae1cddb0270976889a37a2527b18ef37d7"
Sep 08 15:18:41.952416 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:41.952395    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-vc5n9_openshift-console-operator(17e30810-dfd0-4e6f-9204-731d1a6479a3)\"" pod="openshift-console-operator/console-operator-85b454874b-vc5n9" podUID="17e30810-dfd0-4e6f-9204-731d1a6479a3"
Sep 08 15:18:42.067341 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:42.067299    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:42.132783 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:42.132754    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:42.248542 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:42.248469    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:43.172043 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:43.171942    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97658db95-7ngrj" event={"ID":"f80123cf-9974-443c-81a6-0b2d66ef186b","Type":"ContainerStarted","Data":"0b2901c7298bb4ce61d73d7fdbb0f0aeba2cec9dc981e951fdbc8dc5e9dc1f80"}
Sep 08 15:18:43.172043 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:43.172002    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97658db95-7ngrj" event={"ID":"f80123cf-9974-443c-81a6-0b2d66ef186b","Type":"ContainerStarted","Data":"4d0e0b180277067fef8577b6633f476ac44315e3c473e633dfe6201d24abdc88"}
Sep 08 15:18:43.172043 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:43.172019    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97658db95-7ngrj" event={"ID":"f80123cf-9974-443c-81a6-0b2d66ef186b","Type":"ContainerStarted","Data":"e59792fffb52df0c6ab841ed04f03883d47751213314d0e148baebd91a149f1d"}
Sep 08 15:18:43.173496 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:43.173474    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4wzj5" event={"ID":"4b05229d-340e-424a-8533-87c77e6b97d5","Type":"ContainerStarted","Data":"ce9be2bff763fe4d669aa3e3eae32defdbe1a3713d3fa5e889fff263a592ff07"}
Sep 08 15:18:43.191622 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:43.191495    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97658db95-7ngrj" podStartSLOduration=2.321064178 podStartE2EDuration="8.19148048s" totalImagesPullingTime="5.870416302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:18:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:43.191372427 +0000 UTC m=+75.903314922" watchObservedRunningTime="2026-09-08 15:18:43.19148048 +0000 UTC m=+75.903422971"
Sep 08 15:18:43.201501 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:43.201471    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:43.206456 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:43.206419    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4wzj5" podStartSLOduration=65.591776781 podStartE2EDuration="1m14.206407259s" totalImagesPullingTime="8.614630478s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:17:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:43.204871591 +0000 UTC m=+75.916814082" watchObservedRunningTime="2026-09-08 15:18:43.206407259 +0000 UTC m=+75.918349749"
Sep 08 15:18:43.255771 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:43.255746    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:44.891718 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:44.891660    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:46.385242 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:46.385205    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:47.451735 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.451707    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:47.539375 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.539345    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:47.590822 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.590794    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gm4s9"]
Sep 08 15:18:47.594377 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.594361    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:47.597082 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.597048    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fp5sn\""
Sep 08 15:18:47.597189 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.597048    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 08 15:18:47.597496 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.597482    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 08 15:18:47.597720 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.597708    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 08 15:18:47.597770 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.597735    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 08 15:18:47.626288 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.626237    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:47.737330 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.737252    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e1f7427c-6a62-4e51-bac8-5241611604d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-gm4s9\" (UID: \"e1f7427c-6a62-4e51-bac8-5241611604d2\") " pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:47.737330 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.737298    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e1f7427c-6a62-4e51-bac8-5241611604d2-node-exporter-wtmp\") pod \"node-exporter-gm4s9\" (UID: \"e1f7427c-6a62-4e51-bac8-5241611604d2\") " pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:47.737330 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.737327    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1f7427c-6a62-4e51-bac8-5241611604d2-sys\") pod \"node-exporter-gm4s9\" (UID: \"e1f7427c-6a62-4e51-bac8-5241611604d2\") " pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:47.737547 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.737343    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e1f7427c-6a62-4e51-bac8-5241611604d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gm4s9\" (UID: \"e1f7427c-6a62-4e51-bac8-5241611604d2\") " pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:47.737547 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.737375    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e1f7427c-6a62-4e51-bac8-5241611604d2-node-exporter-tls\") pod \"node-exporter-gm4s9\" (UID: \"e1f7427c-6a62-4e51-bac8-5241611604d2\") " pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:47.737547 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.737393    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e1f7427c-6a62-4e51-bac8-5241611604d2-root\") pod \"node-exporter-gm4s9\" (UID: \"e1f7427c-6a62-4e51-bac8-5241611604d2\") " pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:47.737547 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.737406    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1f7427c-6a62-4e51-bac8-5241611604d2-metrics-client-ca\") pod \"node-exporter-gm4s9\" (UID: \"e1f7427c-6a62-4e51-bac8-5241611604d2\") " pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:47.737547 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.737460    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj5sk\" (UniqueName: \"kubernetes.io/projected/e1f7427c-6a62-4e51-bac8-5241611604d2-kube-api-access-bj5sk\") pod \"node-exporter-gm4s9\" (UID: \"e1f7427c-6a62-4e51-bac8-5241611604d2\") " pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:47.737547 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.737496    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e1f7427c-6a62-4e51-bac8-5241611604d2-node-exporter-textfile\") pod \"node-exporter-gm4s9\" (UID: \"e1f7427c-6a62-4e51-bac8-5241611604d2\") " pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:47.838417 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.838383    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e1f7427c-6a62-4e51-bac8-5241611604d2-node-exporter-tls\") pod \"node-exporter-gm4s9\" (UID: \"e1f7427c-6a62-4e51-bac8-5241611604d2\") " pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:47.838417 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.838423    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e1f7427c-6a62-4e51-bac8-5241611604d2-root\") pod \"node-exporter-gm4s9\" (UID: \"e1f7427c-6a62-4e51-bac8-5241611604d2\") " pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:47.838627 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.838438    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1f7427c-6a62-4e51-bac8-5241611604d2-metrics-client-ca\") pod \"node-exporter-gm4s9\" (UID: \"e1f7427c-6a62-4e51-bac8-5241611604d2\") " pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:47.838627 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.838455    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj5sk\" (UniqueName: \"kubernetes.io/projected/e1f7427c-6a62-4e51-bac8-5241611604d2-kube-api-access-bj5sk\") pod \"node-exporter-gm4s9\" (UID: \"e1f7427c-6a62-4e51-bac8-5241611604d2\") " pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:47.838627 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.838508    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e1f7427c-6a62-4e51-bac8-5241611604d2-node-exporter-textfile\") pod \"node-exporter-gm4s9\" (UID: \"e1f7427c-6a62-4e51-bac8-5241611604d2\") " pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:47.838627 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.838523    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e1f7427c-6a62-4e51-bac8-5241611604d2-root\") pod \"node-exporter-gm4s9\" (UID: \"e1f7427c-6a62-4e51-bac8-5241611604d2\") " pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:47.838627 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:47.838547    2560 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 08 15:18:47.838627 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.838533    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e1f7427c-6a62-4e51-bac8-5241611604d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-gm4s9\" (UID: \"e1f7427c-6a62-4e51-bac8-5241611604d2\") " pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:47.838898 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:18:47.838668    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1f7427c-6a62-4e51-bac8-5241611604d2-node-exporter-tls podName:e1f7427c-6a62-4e51-bac8-5241611604d2 nodeName:}" failed. No retries permitted until 2026-09-08 15:18:48.338644766 +0000 UTC m=+81.050587246 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e1f7427c-6a62-4e51-bac8-5241611604d2-node-exporter-tls") pod "node-exporter-gm4s9" (UID: "e1f7427c-6a62-4e51-bac8-5241611604d2") : secret "node-exporter-tls" not found
Sep 08 15:18:47.838898 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.838688    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e1f7427c-6a62-4e51-bac8-5241611604d2-node-exporter-wtmp\") pod \"node-exporter-gm4s9\" (UID: \"e1f7427c-6a62-4e51-bac8-5241611604d2\") " pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:47.838898 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.838738    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1f7427c-6a62-4e51-bac8-5241611604d2-sys\") pod \"node-exporter-gm4s9\" (UID: \"e1f7427c-6a62-4e51-bac8-5241611604d2\") " pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:47.838898 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.838766    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e1f7427c-6a62-4e51-bac8-5241611604d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gm4s9\" (UID: \"e1f7427c-6a62-4e51-bac8-5241611604d2\") " pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:47.839060 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.838919    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e1f7427c-6a62-4e51-bac8-5241611604d2-node-exporter-wtmp\") pod \"node-exporter-gm4s9\" (UID: \"e1f7427c-6a62-4e51-bac8-5241611604d2\") " pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:47.839060 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.838971    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1f7427c-6a62-4e51-bac8-5241611604d2-sys\") pod \"node-exporter-gm4s9\" (UID: \"e1f7427c-6a62-4e51-bac8-5241611604d2\") " pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:47.839060 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.838986    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e1f7427c-6a62-4e51-bac8-5241611604d2-node-exporter-textfile\") pod \"node-exporter-gm4s9\" (UID: \"e1f7427c-6a62-4e51-bac8-5241611604d2\") " pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:47.839188 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.839145    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1f7427c-6a62-4e51-bac8-5241611604d2-metrics-client-ca\") pod \"node-exporter-gm4s9\" (UID: \"e1f7427c-6a62-4e51-bac8-5241611604d2\") " pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:47.839188 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.839167    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e1f7427c-6a62-4e51-bac8-5241611604d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-gm4s9\" (UID: \"e1f7427c-6a62-4e51-bac8-5241611604d2\") " pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:47.841043 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.841025    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e1f7427c-6a62-4e51-bac8-5241611604d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gm4s9\" (UID: \"e1f7427c-6a62-4e51-bac8-5241611604d2\") " pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:47.856497 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:47.856471    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj5sk\" (UniqueName: \"kubernetes.io/projected/e1f7427c-6a62-4e51-bac8-5241611604d2-kube-api-access-bj5sk\") pod \"node-exporter-gm4s9\" (UID: \"e1f7427c-6a62-4e51-bac8-5241611604d2\") " pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:48.343178 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:48.343135    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e1f7427c-6a62-4e51-bac8-5241611604d2-node-exporter-tls\") pod \"node-exporter-gm4s9\" (UID: \"e1f7427c-6a62-4e51-bac8-5241611604d2\") " pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:48.345467 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:48.345438    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e1f7427c-6a62-4e51-bac8-5241611604d2-node-exporter-tls\") pod \"node-exporter-gm4s9\" (UID: \"e1f7427c-6a62-4e51-bac8-5241611604d2\") " pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:48.503705 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:48.503669    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gm4s9"
Sep 08 15:18:48.514002 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:18:48.513964    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1f7427c_6a62_4e51_bac8_5241611604d2.slice/crio-f37dc64d6b492fec261d452fc4d481c20c8fc4bc6e454869913b069e5ef09ebc WatchSource:0}: Error finding container f37dc64d6b492fec261d452fc4d481c20c8fc4bc6e454869913b069e5ef09ebc: Status 404 returned error can't find the container with id f37dc64d6b492fec261d452fc4d481c20c8fc4bc6e454869913b069e5ef09ebc
Sep 08 15:18:48.975729 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:48.975693    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:49.146766 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:49.146733    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:49.160768 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:49.160740    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ct4vs"
Sep 08 15:18:49.200220 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:49.200182    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gm4s9" event={"ID":"e1f7427c-6a62-4e51-bac8-5241611604d2","Type":"ContainerStarted","Data":"f37dc64d6b492fec261d452fc4d481c20c8fc4bc6e454869913b069e5ef09ebc"}
Sep 08 15:18:49.951660 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:49.951576    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:50.072380 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:50.072351    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:50.205505 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:50.205368    2560 generic.go:356] "Generic (PLEG): container finished" podID="e1f7427c-6a62-4e51-bac8-5241611604d2" containerID="514a06519b618bacdbfcca1d3da08ee1571090f8111cf00d78194ba1eb7b6f49" exitCode=0
Sep 08 15:18:50.205505 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:50.205443    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gm4s9" event={"ID":"e1f7427c-6a62-4e51-bac8-5241611604d2","Type":"ContainerDied","Data":"514a06519b618bacdbfcca1d3da08ee1571090f8111cf00d78194ba1eb7b6f49"}
Sep 08 15:18:50.542666 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:50.542584    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:50.598042 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:50.598013    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:50.749704 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:50.749672    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:50.850286 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:50.850210    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:50.950803 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:50.950771    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:51.174587 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:51.174559    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:51.213858 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:51.213820    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gm4s9" event={"ID":"e1f7427c-6a62-4e51-bac8-5241611604d2","Type":"ContainerStarted","Data":"18436c6325046e21bcd458c59597394a1278db0cd7b5c5c465379f59574b35f3"}
Sep 08 15:18:51.214027 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:51.213912    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gm4s9" event={"ID":"e1f7427c-6a62-4e51-bac8-5241611604d2","Type":"ContainerStarted","Data":"23fdb9cfb933f3cf837a222854475bc24f58aead00e0fd6eacb3d13bdd331533"}
Sep 08 15:18:51.232826 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:51.232776    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gm4s9" podStartSLOduration=2.989889169 podStartE2EDuration="4.232759664s" totalImagesPullingTime="1.17879515s" totalInitContainerRuntime="64.075345ms" isStatefulPod=true podCreationTimestamp="2026-09-08 15:18:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:51.230948623 +0000 UTC m=+83.942891114" watchObservedRunningTime="2026-09-08 15:18:51.232759664 +0000 UTC m=+83.944702154"
Sep 08 15:18:51.236357 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:51.236330    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:52.314701 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:52.314667    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:52.567103 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:52.567013    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:53.729566 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:53.729530    2560 patch_prober.go:28] interesting pod/image-registry-64f7cfd7bb-88t2j container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 08 15:18:53.729988 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:53.729598    2560 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j" podUID="75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 15:18:53.827766 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:53.827706    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:55.129546 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:55.129517    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:18:55.333122 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:55.333087    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:55.794524 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:55.794492    2560 scope.go:117] "RemoveContainer" containerID="5dd171b161d7a73e2133b2d1513997ae1cddb0270976889a37a2527b18ef37d7"
Sep 08 15:18:56.137685 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:56.137615    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:56.239534 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:56.239505    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vc5n9_17e30810-dfd0-4e6f-9204-731d1a6479a3/console-operator/2.log"
Sep 08 15:18:56.239889 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:56.239602    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-vc5n9" event={"ID":"17e30810-dfd0-4e6f-9204-731d1a6479a3","Type":"ContainerStarted","Data":"7274f19e039d2349eb294d809d9f913d555fe9a53f0930f253c0ac1793ee2065"}
Sep 08 15:18:56.239966 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:56.239887    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-vc5n9"
Sep 08 15:18:56.261682 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:56.261632    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-vc5n9" podStartSLOduration=69.064161929 podStartE2EDuration="1m18.261614703s" totalImagesPullingTime="9.197452774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:17:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:56.260481116 +0000 UTC m=+88.972423608" watchObservedRunningTime="2026-09-08 15:18:56.261614703 +0000 UTC m=+88.973557194"
Sep 08 15:18:56.303414 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:56.303369    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-vc5n9"
Sep 08 15:18:57.308935 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:57.308900    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:57.417952 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:57.417917    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:57.485910 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:57.485879    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:57.584253 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:57.584173    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:57.656984 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:57.656949    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:57.974118 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:57.974087    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:58.072085 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:58.072046    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:58.202210 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:58.202175    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:58.414119 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:58.413982    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:58.490602 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:58.490571    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:58.658575 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:58.658542    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:58.786528 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:58.786495    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64f7cfd7bb-88t2j"]
Sep 08 15:18:58.923306 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:58.923272    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:59.028475 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:59.028442    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:59.406330 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:59.406299    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:59.474119 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:59.474063    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:59.962608 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:18:59.962574    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:00.061766 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:00.061734    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:06.133962 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:06.133931    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mb5jg"
Sep 08 15:19:06.206330 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:06.206240    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:06.338187 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:06.338136    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:07.533715 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:07.533675    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:07.598138 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:07.598105    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:09.186847 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:09.186809    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:09.274706 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:09.274669    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:10.667105 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:10.667055    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:10.758000 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:10.757970    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:12.977200 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:12.977162    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:13.060188 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:13.060156    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:13.624642 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:13.624609    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:13.744648 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:13.744610    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:20.478324 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:20.478276    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:20.585708 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:20.585678    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:20.818137 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:20.818016    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:20.918605 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:20.918555    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:21.253398 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:21.253367    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:22.237580 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:22.237551    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:22.891430 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:22.891399    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:23.339135 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:23.339102    2560 generic.go:356] "Generic (PLEG): container finished" podID="149a58aa-2b85-4704-b93b-1056c13054df" containerID="d029b29b48918eafb91765b8d00fe68e33bf7e27c012964076b531fa649fba16" exitCode=0
Sep 08 15:19:23.339585 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:23.339176    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pvz7x" event={"ID":"149a58aa-2b85-4704-b93b-1056c13054df","Type":"ContainerDied","Data":"d029b29b48918eafb91765b8d00fe68e33bf7e27c012964076b531fa649fba16"}
Sep 08 15:19:23.339585 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:23.339540    2560 scope.go:117] "RemoveContainer" containerID="d029b29b48918eafb91765b8d00fe68e33bf7e27c012964076b531fa649fba16"
Sep 08 15:19:23.343671 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:23.343646    2560 generic.go:356] "Generic (PLEG): container finished" podID="21604064-e43d-4ec1-b6e9-9af32f6208ed" containerID="d35b466dc72d1837328ef6bf7e76a750a2e63deeaa0f6511c98cc769dd769452" exitCode=0
Sep 08 15:19:23.343745 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:23.343693    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-857nk" event={"ID":"21604064-e43d-4ec1-b6e9-9af32f6208ed","Type":"ContainerDied","Data":"d35b466dc72d1837328ef6bf7e76a750a2e63deeaa0f6511c98cc769dd769452"}
Sep 08 15:19:23.344037 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:23.344020    2560 scope.go:117] "RemoveContainer" containerID="d35b466dc72d1837328ef6bf7e76a750a2e63deeaa0f6511c98cc769dd769452"
Sep 08 15:19:23.808616 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:23.808577    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j" podUID="75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa" containerName="registry" containerID="cri-o://0a4821b3107543a9f81382d9e74f2ec090eb2773084c58ce9051a8652b2dc467" gracePeriod=30
Sep 08 15:19:24.047607 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.047583    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:19:24.116508 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.116414    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-bound-sa-token\") pod \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") "
Sep 08 15:19:24.116508 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.116494    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-image-registry-private-configuration\") pod \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") "
Sep 08 15:19:24.116737 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.116521    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-trusted-ca\") pod \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") "
Sep 08 15:19:24.116737 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.116561    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qfmp\" (UniqueName: \"kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-kube-api-access-7qfmp\") pod \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") "
Sep 08 15:19:24.116737 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.116694    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-ca-trust-extracted\") pod \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") "
Sep 08 15:19:24.116887 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.116752    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-certificates\") pod \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") "
Sep 08 15:19:24.116887 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.116791    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-tls\") pod \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") "
Sep 08 15:19:24.116887 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.116831    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-installation-pull-secrets\") pod \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\" (UID: \"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa\") "
Sep 08 15:19:24.117049 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.117011    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa" (UID: "75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:19:24.117140 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.117121    2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-trusted-ca\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:19:24.117198 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.117134    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa" (UID: "75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:19:24.119130 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.119100    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa" (UID: "75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:19:24.119251 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.119180    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa" (UID: "75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:19:24.119301 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.119258    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa" (UID: "75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:19:24.119393 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.119375    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa" (UID: "75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:19:24.119429 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.119375    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-kube-api-access-7qfmp" (OuterVolumeSpecName: "kube-api-access-7qfmp") pod "75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa" (UID: "75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa"). InnerVolumeSpecName "kube-api-access-7qfmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:19:24.124986 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.124960    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa" (UID: "75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:19:24.217738 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.217702    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qfmp\" (UniqueName: \"kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-kube-api-access-7qfmp\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:19:24.217738 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.217729    2560 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-ca-trust-extracted\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:19:24.217738 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.217739    2560 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-certificates\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:19:24.217738 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.217749    2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-registry-tls\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:19:24.217977 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.217758    2560 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-installation-pull-secrets\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:19:24.217977 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.217767    2560 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-bound-sa-token\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:19:24.217977 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.217775    2560 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa-image-registry-private-configuration\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:19:24.349274 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.349237    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-pvz7x" event={"ID":"149a58aa-2b85-4704-b93b-1056c13054df","Type":"ContainerStarted","Data":"66801ee5477b0beb3de16333140999e39ed84c31a90d521debc306fde3d61867"}
Sep 08 15:19:24.350987 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.350957    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-857nk" event={"ID":"21604064-e43d-4ec1-b6e9-9af32f6208ed","Type":"ContainerStarted","Data":"5b355527b004caa3f221af60bc801f8ad8ecb93083039b8489be952ccd0e16a9"}
Sep 08 15:19:24.352104 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.352064    2560 generic.go:356] "Generic (PLEG): container finished" podID="75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa" containerID="0a4821b3107543a9f81382d9e74f2ec090eb2773084c58ce9051a8652b2dc467" exitCode=0
Sep 08 15:19:24.352104 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.352092    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j" event={"ID":"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa","Type":"ContainerDied","Data":"0a4821b3107543a9f81382d9e74f2ec090eb2773084c58ce9051a8652b2dc467"}
Sep 08 15:19:24.352244 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.352119    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j"
Sep 08 15:19:24.352244 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.352142    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64f7cfd7bb-88t2j" event={"ID":"75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa","Type":"ContainerDied","Data":"eb48de1533647d9c23d89990afcee032b06da331c5a169b69f21d9fe5232ae09"}
Sep 08 15:19:24.352244 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.352167    2560 scope.go:117] "RemoveContainer" containerID="0a4821b3107543a9f81382d9e74f2ec090eb2773084c58ce9051a8652b2dc467"
Sep 08 15:19:24.360341 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.360321    2560 scope.go:117] "RemoveContainer" containerID="0a4821b3107543a9f81382d9e74f2ec090eb2773084c58ce9051a8652b2dc467"
Sep 08 15:19:24.360624 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:19:24.360597    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a4821b3107543a9f81382d9e74f2ec090eb2773084c58ce9051a8652b2dc467\": container with ID starting with 0a4821b3107543a9f81382d9e74f2ec090eb2773084c58ce9051a8652b2dc467 not found: ID does not exist" containerID="0a4821b3107543a9f81382d9e74f2ec090eb2773084c58ce9051a8652b2dc467"
Sep 08 15:19:24.360677 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.360650    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a4821b3107543a9f81382d9e74f2ec090eb2773084c58ce9051a8652b2dc467"} err="failed to get container status \"0a4821b3107543a9f81382d9e74f2ec090eb2773084c58ce9051a8652b2dc467\": rpc error: code = NotFound desc = could not find container \"0a4821b3107543a9f81382d9e74f2ec090eb2773084c58ce9051a8652b2dc467\": container with ID starting with 0a4821b3107543a9f81382d9e74f2ec090eb2773084c58ce9051a8652b2dc467 not found: ID does not exist"
Sep 08 15:19:24.386099 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.386020    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64f7cfd7bb-88t2j"]
Sep 08 15:19:24.391274 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:24.391250    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-64f7cfd7bb-88t2j"]
Sep 08 15:19:25.798734 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:25.798701    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa" path="/var/lib/kubelet/pods/75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa/volumes"
Sep 08 15:19:28.056696 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:28.056656    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:28.157727 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:28.157699    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:31.710021 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:31.709992    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:31.817046 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:31.817012    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:32.383408 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:32.383374    2560 generic.go:356] "Generic (PLEG): container finished" podID="d4a10483-c29e-4a1c-9eb8-79d3197c1e35" containerID="82117695d7f282d676404e994bb47a03206ccb3a001a02fbd1b2dba9533b3c75" exitCode=0
Sep 08 15:19:32.383559 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:32.383450    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-dkwzw" event={"ID":"d4a10483-c29e-4a1c-9eb8-79d3197c1e35","Type":"ContainerDied","Data":"82117695d7f282d676404e994bb47a03206ccb3a001a02fbd1b2dba9533b3c75"}
Sep 08 15:19:32.383818 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:32.383804    2560 scope.go:117] "RemoveContainer" containerID="82117695d7f282d676404e994bb47a03206ccb3a001a02fbd1b2dba9533b3c75"
Sep 08 15:19:33.388846 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:33.388810    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-dkwzw" event={"ID":"d4a10483-c29e-4a1c-9eb8-79d3197c1e35","Type":"ContainerStarted","Data":"9819e75269ee2c79ed772df50eabc075f2095dd14d014316ad06297ff1947bb4"}
Sep 08 15:19:38.348100 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:19:38.348050    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:03.717457 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:20:03.717425    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:03.775589 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:20:03.775553    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:15.692125 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:20:15.692089    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:15.749506 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:20:15.749480    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:20.600126 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:20:20.600045    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:27.997391 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:20:27.997354    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:28.084006 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:20:28.083971    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:28.898706 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:20:28.898667    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:31.498058 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:20:31.498028    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:31.549721 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:20:31.549672    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:31.616240 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:20:31.616211    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:37.795703 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:20:37.795669    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:21:03.080920 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:21:03.080882    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:21:40.584756 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:21:40.584718    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:21:40.681338 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:21:40.681304    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:07.273256 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:07.273218    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:08.286716 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:08.286679    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:10.891265 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:10.891232    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:10.990958 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:10.990920    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:13.590999 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:13.590952    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:14.725323 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:14.725291    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:14.823242 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:14.823213    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:15.637446 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:15.637416    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:16.088640 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:16.088603    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:23.623821 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:23.623787    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:27.159030 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:27.158993    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:27.660368 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:27.660343    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vc5n9_17e30810-dfd0-4e6f-9204-731d1a6479a3/console-operator/2.log"
Sep 08 15:22:27.661900 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:27.661876    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vc5n9_17e30810-dfd0-4e6f-9204-731d1a6479a3/console-operator/2.log"
Sep 08 15:22:27.671279 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:27.671261    2560 kubelet.go:1648] "Image garbage collection succeeded"
Sep 08 15:22:27.920191 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:27.919993    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-55vzg"]
Sep 08 15:22:27.924373 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:27.920335    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa" containerName="registry"
Sep 08 15:22:27.924373 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:27.920354    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa" containerName="registry"
Sep 08 15:22:27.924373 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:27.920443    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="75c27ee1-bb0d-4b1e-bb71-c7be7e5f7baa" containerName="registry"
Sep 08 15:22:27.924506 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:27.924378    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-55vzg"
Sep 08 15:22:27.927022 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:27.927000    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 08 15:22:27.927022 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:27.927002    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 08 15:22:27.927259 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:27.927102    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-2gpr5\""
Sep 08 15:22:27.941233 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:27.941213    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-55vzg"]
Sep 08 15:22:28.117803 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:28.117759    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7xs4\" (UniqueName: \"kubernetes.io/projected/9ddacf7a-3a11-4e36-a847-76f693bc9960-kube-api-access-f7xs4\") pod \"openshift-lws-operator-fd8ccff4c-55vzg\" (UID: \"9ddacf7a-3a11-4e36-a847-76f693bc9960\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-55vzg"
Sep 08 15:22:28.117967 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:28.117814    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ddacf7a-3a11-4e36-a847-76f693bc9960-tmp\") pod \"openshift-lws-operator-fd8ccff4c-55vzg\" (UID: \"9ddacf7a-3a11-4e36-a847-76f693bc9960\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-55vzg"
Sep 08 15:22:28.218467 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:28.218433    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7xs4\" (UniqueName: \"kubernetes.io/projected/9ddacf7a-3a11-4e36-a847-76f693bc9960-kube-api-access-f7xs4\") pod \"openshift-lws-operator-fd8ccff4c-55vzg\" (UID: \"9ddacf7a-3a11-4e36-a847-76f693bc9960\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-55vzg"
Sep 08 15:22:28.218821 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:28.218491    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ddacf7a-3a11-4e36-a847-76f693bc9960-tmp\") pod \"openshift-lws-operator-fd8ccff4c-55vzg\" (UID: \"9ddacf7a-3a11-4e36-a847-76f693bc9960\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-55vzg"
Sep 08 15:22:28.218890 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:28.218869    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ddacf7a-3a11-4e36-a847-76f693bc9960-tmp\") pod \"openshift-lws-operator-fd8ccff4c-55vzg\" (UID: \"9ddacf7a-3a11-4e36-a847-76f693bc9960\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-55vzg"
Sep 08 15:22:28.227158 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:28.227132    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7xs4\" (UniqueName: \"kubernetes.io/projected/9ddacf7a-3a11-4e36-a847-76f693bc9960-kube-api-access-f7xs4\") pod \"openshift-lws-operator-fd8ccff4c-55vzg\" (UID: \"9ddacf7a-3a11-4e36-a847-76f693bc9960\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-55vzg"
Sep 08 15:22:28.232869 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:28.232850    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-55vzg"
Sep 08 15:22:28.521447 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:28.521372    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-55vzg"]
Sep 08 15:22:28.524782 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:22:28.524755    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ddacf7a_3a11_4e36_a847_76f693bc9960.slice/crio-ab8569f9aa9fb86b60ca8e1719ac6c136328ba7c05f5451f1dfeaab3eec8cdc8 WatchSource:0}: Error finding container ab8569f9aa9fb86b60ca8e1719ac6c136328ba7c05f5451f1dfeaab3eec8cdc8: Status 404 returned error can't find the container with id ab8569f9aa9fb86b60ca8e1719ac6c136328ba7c05f5451f1dfeaab3eec8cdc8
Sep 08 15:22:29.009542 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:29.009498    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-55vzg" event={"ID":"9ddacf7a-3a11-4e36-a847-76f693bc9960","Type":"ContainerStarted","Data":"ab8569f9aa9fb86b60ca8e1719ac6c136328ba7c05f5451f1dfeaab3eec8cdc8"}
Sep 08 15:22:31.020567 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:31.020531    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-55vzg" event={"ID":"9ddacf7a-3a11-4e36-a847-76f693bc9960","Type":"ContainerStarted","Data":"f39e970cb3751d074c08dfb3672b38fe239f892f9463ae9d19cb8c747962ecce"}
Sep 08 15:22:31.046347 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:31.046292    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-55vzg" podStartSLOduration=1.719338897 podStartE2EDuration="4.046271329s" totalImagesPullingTime="2.326932432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:22:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:22:31.043772035 +0000 UTC m=+303.755714529" watchObservedRunningTime="2026-09-08 15:22:31.046271329 +0000 UTC m=+303.758213824"
Sep 08 15:22:31.806240 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:31.806195    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:31.927481 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:31.927446    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:33.932400 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:33.932368    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:43.231369 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:43.231332    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:43.330572 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:43.330538    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:49.560044 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:49.560010    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:49.662910 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:49.662878    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:50.159953 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.159885    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:50.316328 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.316298    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:50.417481 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.417395    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:50.603008 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.602973    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-847d47f6c8-c7hnl"]
Sep 08 15:22:50.606547 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.606530    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-c7hnl"
Sep 08 15:22:50.608588 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.608557    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 08 15:22:50.608707 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.608664    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 08 15:22:50.608707 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.608665    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4lc5s\""
Sep 08 15:22:50.609117 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.609103    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 08 15:22:50.610311 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.610295    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 08 15:22:50.637084 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.637041    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-847d47f6c8-c7hnl"]
Sep 08 15:22:50.701499 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.701467    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a72080d0-7eab-402c-a477-9d487435aab9-manifests\") pod \"opendatahub-operator-controller-manager-847d47f6c8-c7hnl\" (UID: \"a72080d0-7eab-402c-a477-9d487435aab9\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-c7hnl"
Sep 08 15:22:50.701648 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.701503    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a72080d0-7eab-402c-a477-9d487435aab9-tmp\") pod \"opendatahub-operator-controller-manager-847d47f6c8-c7hnl\" (UID: \"a72080d0-7eab-402c-a477-9d487435aab9\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-c7hnl"
Sep 08 15:22:50.701648 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.701528    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a72080d0-7eab-402c-a477-9d487435aab9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-847d47f6c8-c7hnl\" (UID: \"a72080d0-7eab-402c-a477-9d487435aab9\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-c7hnl"
Sep 08 15:22:50.701648 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.701593    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a72080d0-7eab-402c-a477-9d487435aab9-webhook-cert\") pod \"opendatahub-operator-controller-manager-847d47f6c8-c7hnl\" (UID: \"a72080d0-7eab-402c-a477-9d487435aab9\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-c7hnl"
Sep 08 15:22:50.701758 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.701657    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qgg2\" (UniqueName: \"kubernetes.io/projected/a72080d0-7eab-402c-a477-9d487435aab9-kube-api-access-2qgg2\") pod \"opendatahub-operator-controller-manager-847d47f6c8-c7hnl\" (UID: \"a72080d0-7eab-402c-a477-9d487435aab9\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-c7hnl"
Sep 08 15:22:50.803003 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.802955    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a72080d0-7eab-402c-a477-9d487435aab9-manifests\") pod \"opendatahub-operator-controller-manager-847d47f6c8-c7hnl\" (UID: \"a72080d0-7eab-402c-a477-9d487435aab9\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-c7hnl"
Sep 08 15:22:50.803003 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.803001    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a72080d0-7eab-402c-a477-9d487435aab9-tmp\") pod \"opendatahub-operator-controller-manager-847d47f6c8-c7hnl\" (UID: \"a72080d0-7eab-402c-a477-9d487435aab9\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-c7hnl"
Sep 08 15:22:50.803003 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.803022    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a72080d0-7eab-402c-a477-9d487435aab9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-847d47f6c8-c7hnl\" (UID: \"a72080d0-7eab-402c-a477-9d487435aab9\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-c7hnl"
Sep 08 15:22:50.803315 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.803144    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a72080d0-7eab-402c-a477-9d487435aab9-webhook-cert\") pod \"opendatahub-operator-controller-manager-847d47f6c8-c7hnl\" (UID: \"a72080d0-7eab-402c-a477-9d487435aab9\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-c7hnl"
Sep 08 15:22:50.803315 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.803196    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qgg2\" (UniqueName: \"kubernetes.io/projected/a72080d0-7eab-402c-a477-9d487435aab9-kube-api-access-2qgg2\") pod \"opendatahub-operator-controller-manager-847d47f6c8-c7hnl\" (UID: \"a72080d0-7eab-402c-a477-9d487435aab9\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-c7hnl"
Sep 08 15:22:50.803573 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.803537    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a72080d0-7eab-402c-a477-9d487435aab9-manifests\") pod \"opendatahub-operator-controller-manager-847d47f6c8-c7hnl\" (UID: \"a72080d0-7eab-402c-a477-9d487435aab9\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-c7hnl"
Sep 08 15:22:50.805451 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.805426    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a72080d0-7eab-402c-a477-9d487435aab9-tmp\") pod \"opendatahub-operator-controller-manager-847d47f6c8-c7hnl\" (UID: \"a72080d0-7eab-402c-a477-9d487435aab9\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-c7hnl"
Sep 08 15:22:50.805550 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.805454    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a72080d0-7eab-402c-a477-9d487435aab9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-847d47f6c8-c7hnl\" (UID: \"a72080d0-7eab-402c-a477-9d487435aab9\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-c7hnl"
Sep 08 15:22:50.805592 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.805582    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a72080d0-7eab-402c-a477-9d487435aab9-webhook-cert\") pod \"opendatahub-operator-controller-manager-847d47f6c8-c7hnl\" (UID: \"a72080d0-7eab-402c-a477-9d487435aab9\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-c7hnl"
Sep 08 15:22:50.819058 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.819030    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qgg2\" (UniqueName: \"kubernetes.io/projected/a72080d0-7eab-402c-a477-9d487435aab9-kube-api-access-2qgg2\") pod \"opendatahub-operator-controller-manager-847d47f6c8-c7hnl\" (UID: \"a72080d0-7eab-402c-a477-9d487435aab9\") " pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-c7hnl"
Sep 08 15:22:50.893308 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.893279    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:50.916170 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.916141    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-c7hnl"
Sep 08 15:22:50.994315 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:50.994282    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:51.089032 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:51.088989    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:51.191129 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:51.191088    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:51.203800 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:51.203772    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-847d47f6c8-c7hnl"]
Sep 08 15:22:51.208969 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:22:51.208939    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda72080d0_7eab_402c_a477_9d487435aab9.slice/crio-a551eb9d73bc6c0d57fed7577bad24dd11f78fdfc4618c92d91adcf3d9ab3263 WatchSource:0}: Error finding container a551eb9d73bc6c0d57fed7577bad24dd11f78fdfc4618c92d91adcf3d9ab3263: Status 404 returned error can't find the container with id a551eb9d73bc6c0d57fed7577bad24dd11f78fdfc4618c92d91adcf3d9ab3263
Sep 08 15:22:51.210227 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:51.210209    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 15:22:51.289799 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:51.289751    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:51.399800 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:51.399763    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:51.491735 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:51.491641    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:52.104753 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:52.104672    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-c7hnl" event={"ID":"a72080d0-7eab-402c-a477-9d487435aab9","Type":"ContainerStarted","Data":"a551eb9d73bc6c0d57fed7577bad24dd11f78fdfc4618c92d91adcf3d9ab3263"}
Sep 08 15:22:55.119571 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:55.119537    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-c7hnl" event={"ID":"a72080d0-7eab-402c-a477-9d487435aab9","Type":"ContainerStarted","Data":"8003faaa625ca82dfed791306beec4f2b60443454ba0b957463471786195d59e"}
Sep 08 15:22:56.124954 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:56.124919    2560 generic.go:356] "Generic (PLEG): container finished" podID="a72080d0-7eab-402c-a477-9d487435aab9" containerID="8003faaa625ca82dfed791306beec4f2b60443454ba0b957463471786195d59e" exitCode=0
Sep 08 15:22:56.124954 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:56.124957    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-c7hnl" event={"ID":"a72080d0-7eab-402c-a477-9d487435aab9","Type":"ContainerDied","Data":"8003faaa625ca82dfed791306beec4f2b60443454ba0b957463471786195d59e"}
Sep 08 15:22:56.769570 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:56.769536    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:56.851286 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:56.851248    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:58.135335 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:58.135300    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-c7hnl" event={"ID":"a72080d0-7eab-402c-a477-9d487435aab9","Type":"ContainerStarted","Data":"3d69df2d501209d765e4306c334066baa3958b428fb847c9a03b82e9b98fa05e"}
Sep 08 15:22:58.135661 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:58.135437    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-c7hnl"
Sep 08 15:22:58.157111 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:22:58.157051    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-c7hnl" podStartSLOduration=1.965888284 podStartE2EDuration="8.157036412s" totalImagesPullingTime="5.10596914s" totalInitContainerRuntime="1.085178988s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:22:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:22:58.155889439 +0000 UTC m=+330.867831929" watchObservedRunningTime="2026-09-08 15:22:58.157036412 +0000 UTC m=+330.868978963"
Sep 08 15:23:02.773913 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:02.773877    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:02.871129 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:02.871085    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:09.141604 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:09.141573    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-847d47f6c8-c7hnl"
Sep 08 15:23:12.045308 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:12.045275    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:12.168534 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:12.168500    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:12.550496 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:12.550465    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:12.626363 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:12.626332    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7d44f46d57-r2s76"]
Sep 08 15:23:12.636455 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:12.636433    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d44f46d57-r2s76"
Sep 08 15:23:12.639213 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:12.639190    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 08 15:23:12.639640 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:12.639620    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 08 15:23:12.639728 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:12.639620    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-sfjkh\""
Sep 08 15:23:12.646684 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:12.646661    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d44f46d57-r2s76"]
Sep 08 15:23:12.772254 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:12.772222    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:12.786318 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:12.786295    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bf49137-ceba-4cbc-8f05-25b821fcab99-tls-certs\") pod \"kube-auth-proxy-7d44f46d57-r2s76\" (UID: \"5bf49137-ceba-4cbc-8f05-25b821fcab99\") " pod="openshift-ingress/kube-auth-proxy-7d44f46d57-r2s76"
Sep 08 15:23:12.786412 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:12.786332    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcr8j\" (UniqueName: \"kubernetes.io/projected/5bf49137-ceba-4cbc-8f05-25b821fcab99-kube-api-access-qcr8j\") pod \"kube-auth-proxy-7d44f46d57-r2s76\" (UID: \"5bf49137-ceba-4cbc-8f05-25b821fcab99\") " pod="openshift-ingress/kube-auth-proxy-7d44f46d57-r2s76"
Sep 08 15:23:12.786412 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:12.786352    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5bf49137-ceba-4cbc-8f05-25b821fcab99-tmp\") pod \"kube-auth-proxy-7d44f46d57-r2s76\" (UID: \"5bf49137-ceba-4cbc-8f05-25b821fcab99\") " pod="openshift-ingress/kube-auth-proxy-7d44f46d57-r2s76"
Sep 08 15:23:12.887732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:12.887655    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bf49137-ceba-4cbc-8f05-25b821fcab99-tls-certs\") pod \"kube-auth-proxy-7d44f46d57-r2s76\" (UID: \"5bf49137-ceba-4cbc-8f05-25b821fcab99\") " pod="openshift-ingress/kube-auth-proxy-7d44f46d57-r2s76"
Sep 08 15:23:12.887732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:12.887704    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcr8j\" (UniqueName: \"kubernetes.io/projected/5bf49137-ceba-4cbc-8f05-25b821fcab99-kube-api-access-qcr8j\") pod \"kube-auth-proxy-7d44f46d57-r2s76\" (UID: \"5bf49137-ceba-4cbc-8f05-25b821fcab99\") " pod="openshift-ingress/kube-auth-proxy-7d44f46d57-r2s76"
Sep 08 15:23:12.887902 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:12.887733    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5bf49137-ceba-4cbc-8f05-25b821fcab99-tmp\") pod \"kube-auth-proxy-7d44f46d57-r2s76\" (UID: \"5bf49137-ceba-4cbc-8f05-25b821fcab99\") " pod="openshift-ingress/kube-auth-proxy-7d44f46d57-r2s76"
Sep 08 15:23:12.889904 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:12.889876    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5bf49137-ceba-4cbc-8f05-25b821fcab99-tmp\") pod \"kube-auth-proxy-7d44f46d57-r2s76\" (UID: \"5bf49137-ceba-4cbc-8f05-25b821fcab99\") " pod="openshift-ingress/kube-auth-proxy-7d44f46d57-r2s76"
Sep 08 15:23:12.890232 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:12.890209    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bf49137-ceba-4cbc-8f05-25b821fcab99-tls-certs\") pod \"kube-auth-proxy-7d44f46d57-r2s76\" (UID: \"5bf49137-ceba-4cbc-8f05-25b821fcab99\") " pod="openshift-ingress/kube-auth-proxy-7d44f46d57-r2s76"
Sep 08 15:23:12.897047 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:12.897017    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcr8j\" (UniqueName: \"kubernetes.io/projected/5bf49137-ceba-4cbc-8f05-25b821fcab99-kube-api-access-qcr8j\") pod \"kube-auth-proxy-7d44f46d57-r2s76\" (UID: \"5bf49137-ceba-4cbc-8f05-25b821fcab99\") " pod="openshift-ingress/kube-auth-proxy-7d44f46d57-r2s76"
Sep 08 15:23:12.946049 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:12.946019    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d44f46d57-r2s76"
Sep 08 15:23:13.224086 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:13.224009    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d44f46d57-r2s76"]
Sep 08 15:23:13.226707 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:23:13.226681    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bf49137_ceba_4cbc_8f05_25b821fcab99.slice/crio-fb27c0cf45d20027a73eaff01bd408cabf491b05215d8b3ae01789e5955f94a5 WatchSource:0}: Error finding container fb27c0cf45d20027a73eaff01bd408cabf491b05215d8b3ae01789e5955f94a5: Status 404 returned error can't find the container with id fb27c0cf45d20027a73eaff01bd408cabf491b05215d8b3ae01789e5955f94a5
Sep 08 15:23:14.051226 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.051190    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:14.060407 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.060383    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"]
Sep 08 15:23:14.064206 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.064187    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"
Sep 08 15:23:14.065929 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.065905    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 08 15:23:14.066212 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.066035    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 08 15:23:14.066212 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.066087    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 08 15:23:14.066212 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.066182    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-bcc5t\""
Sep 08 15:23:14.066212 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.066201    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 08 15:23:14.081779 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.081760    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"]
Sep 08 15:23:14.123874 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.123844    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:14.199257 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.199224    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-sxqmr\" (UID: \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"
Sep 08 15:23:14.199465 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.199278    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-sxqmr\" (UID: \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"
Sep 08 15:23:14.199465 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.199384    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-sxqmr\" (UID: \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"
Sep 08 15:23:14.199465 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.199459    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmlfn\" (UniqueName: \"kubernetes.io/projected/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-kube-api-access-bmlfn\") pod \"istiod-openshift-gateway-7b7995d5cd-sxqmr\" (UID: \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"
Sep 08 15:23:14.199765 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.199492    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-sxqmr\" (UID: \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"
Sep 08 15:23:14.199765 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.199521    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-sxqmr\" (UID: \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"
Sep 08 15:23:14.199765 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.199564    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-sxqmr\" (UID: \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"
Sep 08 15:23:14.202212 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.202184    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d44f46d57-r2s76" event={"ID":"5bf49137-ceba-4cbc-8f05-25b821fcab99","Type":"ContainerStarted","Data":"fb27c0cf45d20027a73eaff01bd408cabf491b05215d8b3ae01789e5955f94a5"}
Sep 08 15:23:14.300037 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.299973    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmlfn\" (UniqueName: \"kubernetes.io/projected/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-kube-api-access-bmlfn\") pod \"istiod-openshift-gateway-7b7995d5cd-sxqmr\" (UID: \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"
Sep 08 15:23:14.300037 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.300025    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-sxqmr\" (UID: \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"
Sep 08 15:23:14.300595 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.300051    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-sxqmr\" (UID: \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"
Sep 08 15:23:14.300595 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.300100    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-sxqmr\" (UID: \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"
Sep 08 15:23:14.300595 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.300159    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-sxqmr\" (UID: \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"
Sep 08 15:23:14.300595 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.300220    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-sxqmr\" (UID: \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"
Sep 08 15:23:14.300595 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.300263    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-sxqmr\" (UID: \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"
Sep 08 15:23:14.300958 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.300895    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-sxqmr\" (UID: \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"
Sep 08 15:23:14.302980 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.302919    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-sxqmr\" (UID: \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"
Sep 08 15:23:14.303108 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.303025    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-sxqmr\" (UID: \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"
Sep 08 15:23:14.303108 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.303048    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-sxqmr\" (UID: \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"
Sep 08 15:23:14.303223 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.303139    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-sxqmr\" (UID: \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"
Sep 08 15:23:14.309180 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.309148    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-sxqmr\" (UID: \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"
Sep 08 15:23:14.309277 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.309178    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmlfn\" (UniqueName: \"kubernetes.io/projected/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-kube-api-access-bmlfn\") pod \"istiod-openshift-gateway-7b7995d5cd-sxqmr\" (UID: \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"
Sep 08 15:23:14.376228 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.376197    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"
Sep 08 15:23:14.667656 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:14.667626    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"]
Sep 08 15:23:14.669532 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:23:14.669504    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod334bf5eb_b67b_4aa1_b1f0_a2d3eb7f4b8f.slice/crio-9a8d20691eb4bbc4057c9d5ad11e9f54de588b69c3d551e4a7182dbeb71e7737 WatchSource:0}: Error finding container 9a8d20691eb4bbc4057c9d5ad11e9f54de588b69c3d551e4a7182dbeb71e7737: Status 404 returned error can't find the container with id 9a8d20691eb4bbc4057c9d5ad11e9f54de588b69c3d551e4a7182dbeb71e7737
Sep 08 15:23:15.207088 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:15.207040    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr" event={"ID":"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f","Type":"ContainerStarted","Data":"9a8d20691eb4bbc4057c9d5ad11e9f54de588b69c3d551e4a7182dbeb71e7737"}
Sep 08 15:23:15.457530 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:15.457433    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:18.378898 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:18.378853    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 08 15:23:18.379285 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:18.378940    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 08 15:23:19.229050 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:19.229010    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d44f46d57-r2s76" event={"ID":"5bf49137-ceba-4cbc-8f05-25b821fcab99","Type":"ContainerStarted","Data":"2264dd091cb51be83e67ea24e9fed0ab8f559bf63a81fb2c04f7bef41de89bf2"}
Sep 08 15:23:19.231019 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:19.230986    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr" event={"ID":"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f","Type":"ContainerStarted","Data":"80f9e80c5298dbb9569c61ff88850637fa41365ce13d89df69d0e518c5811968"}
Sep 08 15:23:19.231242 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:19.231210    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"
Sep 08 15:23:19.233064 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:19.233041    2560 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-sxqmr container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 08 15:23:19.233185 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:19.233103    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr" podUID="334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 15:23:19.246131 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:19.246098    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:19.265302 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:19.265256    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7d44f46d57-r2s76" podStartSLOduration=2.112082809 podStartE2EDuration="7.265240745s" totalImagesPullingTime="5.153157936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:23:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:23:19.263988269 +0000 UTC m=+351.975930762" watchObservedRunningTime="2026-09-08 15:23:19.265240745 +0000 UTC m=+351.977183236"
Sep 08 15:23:19.339370 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:19.339340    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:20.195433 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:20.195364    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:20.235943 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:20.235915    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"
Sep 08 15:23:20.285884 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:20.285837    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr" podStartSLOduration=2.578981321 podStartE2EDuration="6.285820952s" totalImagesPullingTime="3.706839631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:23:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:23:19.296555561 +0000 UTC m=+352.008498054" watchObservedRunningTime="2026-09-08 15:23:20.285820952 +0000 UTC m=+352.997763442"
Sep 08 15:23:20.412955 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:20.412922    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:20.620042 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:20.619962    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:23.228846 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:23.228806    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kn4vl"]
Sep 08 15:23:23.231843 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:23.231826    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-kn4vl"
Sep 08 15:23:23.233784 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:23.233759    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 08 15:23:23.233902 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:23.233820    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-xclg9\""
Sep 08 15:23:23.244249 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:23.244224    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kn4vl"]
Sep 08 15:23:23.247963 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:23.247939    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:23.344918 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:23.344886    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:23.381797 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:23.381765    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqrks\" (UniqueName: \"kubernetes.io/projected/5f1e5b0c-dddb-4859-8041-57996d5ec812-kube-api-access-qqrks\") pod \"odh-model-controller-7998c67dc4-kn4vl\" (UID: \"5f1e5b0c-dddb-4859-8041-57996d5ec812\") " pod="opendatahub/odh-model-controller-7998c67dc4-kn4vl"
Sep 08 15:23:23.381908 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:23.381807    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f1e5b0c-dddb-4859-8041-57996d5ec812-cert\") pod \"odh-model-controller-7998c67dc4-kn4vl\" (UID: \"5f1e5b0c-dddb-4859-8041-57996d5ec812\") " pod="opendatahub/odh-model-controller-7998c67dc4-kn4vl"
Sep 08 15:23:23.445469 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:23.445438    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:23.483218 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:23.483144    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqrks\" (UniqueName: \"kubernetes.io/projected/5f1e5b0c-dddb-4859-8041-57996d5ec812-kube-api-access-qqrks\") pod \"odh-model-controller-7998c67dc4-kn4vl\" (UID: \"5f1e5b0c-dddb-4859-8041-57996d5ec812\") " pod="opendatahub/odh-model-controller-7998c67dc4-kn4vl"
Sep 08 15:23:23.483218 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:23.483186    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f1e5b0c-dddb-4859-8041-57996d5ec812-cert\") pod \"odh-model-controller-7998c67dc4-kn4vl\" (UID: \"5f1e5b0c-dddb-4859-8041-57996d5ec812\") " pod="opendatahub/odh-model-controller-7998c67dc4-kn4vl"
Sep 08 15:23:23.483411 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:23:23.483329    2560 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 08 15:23:23.483411 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:23:23.483390    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f1e5b0c-dddb-4859-8041-57996d5ec812-cert podName:5f1e5b0c-dddb-4859-8041-57996d5ec812 nodeName:}" failed. No retries permitted until 2026-09-08 15:23:23.983371505 +0000 UTC m=+356.695313994 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5f1e5b0c-dddb-4859-8041-57996d5ec812-cert") pod "odh-model-controller-7998c67dc4-kn4vl" (UID: "5f1e5b0c-dddb-4859-8041-57996d5ec812") : secret "odh-model-controller-webhook-cert" not found
Sep 08 15:23:23.491712 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:23.491687    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqrks\" (UniqueName: \"kubernetes.io/projected/5f1e5b0c-dddb-4859-8041-57996d5ec812-kube-api-access-qqrks\") pod \"odh-model-controller-7998c67dc4-kn4vl\" (UID: \"5f1e5b0c-dddb-4859-8041-57996d5ec812\") " pod="opendatahub/odh-model-controller-7998c67dc4-kn4vl"
Sep 08 15:23:23.545282 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:23.545252    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:23.940610 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:23.940581    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:23.988352 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:23.988320    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f1e5b0c-dddb-4859-8041-57996d5ec812-cert\") pod \"odh-model-controller-7998c67dc4-kn4vl\" (UID: \"5f1e5b0c-dddb-4859-8041-57996d5ec812\") " pod="opendatahub/odh-model-controller-7998c67dc4-kn4vl"
Sep 08 15:23:23.988518 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:23:23.988499    2560 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 08 15:23:23.988600 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:23:23.988586    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f1e5b0c-dddb-4859-8041-57996d5ec812-cert podName:5f1e5b0c-dddb-4859-8041-57996d5ec812 nodeName:}" failed. No retries permitted until 2026-09-08 15:23:24.988564298 +0000 UTC m=+357.700506784 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5f1e5b0c-dddb-4859-8041-57996d5ec812-cert") pod "odh-model-controller-7998c67dc4-kn4vl" (UID: "5f1e5b0c-dddb-4859-8041-57996d5ec812") : secret "odh-model-controller-webhook-cert" not found
Sep 08 15:23:24.336458 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:24.336371    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:24.436969 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:24.436934    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:24.996505 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:24.996464    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f1e5b0c-dddb-4859-8041-57996d5ec812-cert\") pod \"odh-model-controller-7998c67dc4-kn4vl\" (UID: \"5f1e5b0c-dddb-4859-8041-57996d5ec812\") " pod="opendatahub/odh-model-controller-7998c67dc4-kn4vl"
Sep 08 15:23:24.998730 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:24.998701    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f1e5b0c-dddb-4859-8041-57996d5ec812-cert\") pod \"odh-model-controller-7998c67dc4-kn4vl\" (UID: \"5f1e5b0c-dddb-4859-8041-57996d5ec812\") " pod="opendatahub/odh-model-controller-7998c67dc4-kn4vl"
Sep 08 15:23:25.042264 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:25.042215    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-kn4vl"
Sep 08 15:23:25.313012 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:25.312918    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kn4vl"]
Sep 08 15:23:25.316058 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:23:25.316033    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f1e5b0c_dddb_4859_8041_57996d5ec812.slice/crio-a973b654c4b61a7a2cceed472efa06f713eec763fe76b893b0b87d125c3e1e9c WatchSource:0}: Error finding container a973b654c4b61a7a2cceed472efa06f713eec763fe76b893b0b87d125c3e1e9c: Status 404 returned error can't find the container with id a973b654c4b61a7a2cceed472efa06f713eec763fe76b893b0b87d125c3e1e9c
Sep 08 15:23:26.262916 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:26.262869    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-kn4vl" event={"ID":"5f1e5b0c-dddb-4859-8041-57996d5ec812","Type":"ContainerStarted","Data":"a973b654c4b61a7a2cceed472efa06f713eec763fe76b893b0b87d125c3e1e9c"}
Sep 08 15:23:29.278836 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:29.278803    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-kn4vl" event={"ID":"5f1e5b0c-dddb-4859-8041-57996d5ec812","Type":"ContainerStarted","Data":"ab298d8a044c3ef8c5912c2f7981154c42a29b96166a120b82b566b84963a978"}
Sep 08 15:23:29.279248 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:29.278925    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-kn4vl"
Sep 08 15:23:29.306863 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:29.306811    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-kn4vl" podStartSLOduration=3.094972732 podStartE2EDuration="6.306793397s" totalImagesPullingTime="3.211820665s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:23:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:23:29.3054135 +0000 UTC m=+362.017356013" watchObservedRunningTime="2026-09-08 15:23:29.306793397 +0000 UTC m=+362.018735889"
Sep 08 15:23:30.887321 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:30.887283    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd"]
Sep 08 15:23:30.890387 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:30.890363    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd"
Sep 08 15:23:30.914882 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:30.914848    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd"]
Sep 08 15:23:30.949849 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:30.949817    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a1a62f91-49cb-4c70-a3e0-92989a5ceb43-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-kt4pd\" (UID: \"a1a62f91-49cb-4c70-a3e0-92989a5ceb43\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd"
Sep 08 15:23:30.949849 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:30.949855    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a1a62f91-49cb-4c70-a3e0-92989a5ceb43-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-kt4pd\" (UID: \"a1a62f91-49cb-4c70-a3e0-92989a5ceb43\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd"
Sep 08 15:23:30.950096 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:30.949916    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1a62f91-49cb-4c70-a3e0-92989a5ceb43-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-kt4pd\" (UID: \"a1a62f91-49cb-4c70-a3e0-92989a5ceb43\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd"
Sep 08 15:23:30.950096 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:30.949940    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbwdr\" (UniqueName: \"kubernetes.io/projected/a1a62f91-49cb-4c70-a3e0-92989a5ceb43-kube-api-access-jbwdr\") pod \"istiod-openshift-gateway-5677ccb49b-kt4pd\" (UID: \"a1a62f91-49cb-4c70-a3e0-92989a5ceb43\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd"
Sep 08 15:23:30.950096 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:30.949964    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a1a62f91-49cb-4c70-a3e0-92989a5ceb43-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-kt4pd\" (UID: \"a1a62f91-49cb-4c70-a3e0-92989a5ceb43\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd"
Sep 08 15:23:30.950096 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:30.950038    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1a62f91-49cb-4c70-a3e0-92989a5ceb43-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-kt4pd\" (UID: \"a1a62f91-49cb-4c70-a3e0-92989a5ceb43\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd"
Sep 08 15:23:30.950096 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:30.950095    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a1a62f91-49cb-4c70-a3e0-92989a5ceb43-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-kt4pd\" (UID: \"a1a62f91-49cb-4c70-a3e0-92989a5ceb43\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd"
Sep 08 15:23:31.050576 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:31.050540    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a1a62f91-49cb-4c70-a3e0-92989a5ceb43-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-kt4pd\" (UID: \"a1a62f91-49cb-4c70-a3e0-92989a5ceb43\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd"
Sep 08 15:23:31.050576 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:31.050578    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a1a62f91-49cb-4c70-a3e0-92989a5ceb43-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-kt4pd\" (UID: \"a1a62f91-49cb-4c70-a3e0-92989a5ceb43\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd"
Sep 08 15:23:31.050811 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:31.050615    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1a62f91-49cb-4c70-a3e0-92989a5ceb43-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-kt4pd\" (UID: \"a1a62f91-49cb-4c70-a3e0-92989a5ceb43\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd"
Sep 08 15:23:31.050811 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:31.050651    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbwdr\" (UniqueName: \"kubernetes.io/projected/a1a62f91-49cb-4c70-a3e0-92989a5ceb43-kube-api-access-jbwdr\") pod \"istiod-openshift-gateway-5677ccb49b-kt4pd\" (UID: \"a1a62f91-49cb-4c70-a3e0-92989a5ceb43\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd"
Sep 08 15:23:31.050811 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:31.050684    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a1a62f91-49cb-4c70-a3e0-92989a5ceb43-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-kt4pd\" (UID: \"a1a62f91-49cb-4c70-a3e0-92989a5ceb43\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd"
Sep 08 15:23:31.050811 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:31.050727    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1a62f91-49cb-4c70-a3e0-92989a5ceb43-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-kt4pd\" (UID: \"a1a62f91-49cb-4c70-a3e0-92989a5ceb43\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd"
Sep 08 15:23:31.050811 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:31.050765    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a1a62f91-49cb-4c70-a3e0-92989a5ceb43-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-kt4pd\" (UID: \"a1a62f91-49cb-4c70-a3e0-92989a5ceb43\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd"
Sep 08 15:23:31.051664 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:31.051630    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a1a62f91-49cb-4c70-a3e0-92989a5ceb43-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-kt4pd\" (UID: \"a1a62f91-49cb-4c70-a3e0-92989a5ceb43\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd"
Sep 08 15:23:31.053049 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:31.053025    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1a62f91-49cb-4c70-a3e0-92989a5ceb43-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-kt4pd\" (UID: \"a1a62f91-49cb-4c70-a3e0-92989a5ceb43\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd"
Sep 08 15:23:31.053235 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:31.053214    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a1a62f91-49cb-4c70-a3e0-92989a5ceb43-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-kt4pd\" (UID: \"a1a62f91-49cb-4c70-a3e0-92989a5ceb43\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd"
Sep 08 15:23:31.053302 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:31.053257    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a1a62f91-49cb-4c70-a3e0-92989a5ceb43-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-kt4pd\" (UID: \"a1a62f91-49cb-4c70-a3e0-92989a5ceb43\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd"
Sep 08 15:23:31.053302 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:31.053274    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a1a62f91-49cb-4c70-a3e0-92989a5ceb43-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-kt4pd\" (UID: \"a1a62f91-49cb-4c70-a3e0-92989a5ceb43\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd"
Sep 08 15:23:31.059541 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:31.059514    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbwdr\" (UniqueName: \"kubernetes.io/projected/a1a62f91-49cb-4c70-a3e0-92989a5ceb43-kube-api-access-jbwdr\") pod \"istiod-openshift-gateway-5677ccb49b-kt4pd\" (UID: \"a1a62f91-49cb-4c70-a3e0-92989a5ceb43\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd"
Sep 08 15:23:31.059624 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:31.059591    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1a62f91-49cb-4c70-a3e0-92989a5ceb43-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-kt4pd\" (UID: \"a1a62f91-49cb-4c70-a3e0-92989a5ceb43\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd"
Sep 08 15:23:31.199581 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:31.199547    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd"
Sep 08 15:23:31.489927 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:31.489831    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd"]
Sep 08 15:23:31.496000 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:23:31.495960    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1a62f91_49cb_4c70_a3e0_92989a5ceb43.slice/crio-994b43e647d3d021e7060e5913be5b7848c1948a47befebef0a623bfe3797a3f WatchSource:0}: Error finding container 994b43e647d3d021e7060e5913be5b7848c1948a47befebef0a623bfe3797a3f: Status 404 returned error can't find the container with id 994b43e647d3d021e7060e5913be5b7848c1948a47befebef0a623bfe3797a3f
Sep 08 15:23:31.498640 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:31.498599    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 08 15:23:31.498708 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:31.498676    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 08 15:23:32.293099 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:32.293049    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd" event={"ID":"a1a62f91-49cb-4c70-a3e0-92989a5ceb43","Type":"ContainerStarted","Data":"fa6a652c2c80a56929a689d8171324366ebb6c3ac134843e4162f719686eae90"}
Sep 08 15:23:32.293475 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:32.293107    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd" event={"ID":"a1a62f91-49cb-4c70-a3e0-92989a5ceb43","Type":"ContainerStarted","Data":"994b43e647d3d021e7060e5913be5b7848c1948a47befebef0a623bfe3797a3f"}
Sep 08 15:23:32.293475 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:32.293254    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd"
Sep 08 15:23:32.294938 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:32.294913    2560 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-kt4pd container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 08 15:23:32.295031 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:32.294964    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd" podUID="a1a62f91-49cb-4c70-a3e0-92989a5ceb43" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 15:23:32.320827 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:32.320776    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd" podStartSLOduration=2.320760259 podStartE2EDuration="2.320760259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:23:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:23:32.320138302 +0000 UTC m=+365.032080792" watchObservedRunningTime="2026-09-08 15:23:32.320760259 +0000 UTC m=+365.032702751"
Sep 08 15:23:33.298315 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:33.298287    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kt4pd"
Sep 08 15:23:33.406971 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:33.406940    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"]
Sep 08 15:23:33.407177 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:33.407152    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr" podUID="334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f" containerName="discovery" containerID="cri-o://80f9e80c5298dbb9569c61ff88850637fa41365ce13d89df69d0e518c5811968" gracePeriod=30
Sep 08 15:23:33.656849 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:33.656826    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"
Sep 08 15:23:33.773668 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:33.773632    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-istio-csr-dns-cert\") pod \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\" (UID: \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\") "
Sep 08 15:23:33.773668 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:33.773667    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-local-certs\") pod \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\" (UID: \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\") "
Sep 08 15:23:33.773908 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:33.773715    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-istio-token\") pod \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\" (UID: \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\") "
Sep 08 15:23:33.773908 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:33.773742    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-istio-kubeconfig\") pod \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\" (UID: \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\") "
Sep 08 15:23:33.773908 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:33.773759    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmlfn\" (UniqueName: \"kubernetes.io/projected/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-kube-api-access-bmlfn\") pod \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\" (UID: \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\") "
Sep 08 15:23:33.773908 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:33.773781    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-istio-csr-ca-configmap\") pod \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\" (UID: \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\") "
Sep 08 15:23:33.773908 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:33.773851    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-cacerts\") pod \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\" (UID: \"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f\") "
Sep 08 15:23:33.774390 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:33.774354    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f" (UID: "334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:23:33.776311 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:33.776282    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-cacerts" (OuterVolumeSpecName: "cacerts") pod "334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f" (UID: "334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:23:33.776419 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:33.776392    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-istio-token" (OuterVolumeSpecName: "istio-token") pod "334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f" (UID: "334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:23:33.776499 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:33.776455    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-local-certs" (OuterVolumeSpecName: "local-certs") pod "334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f" (UID: "334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:23:33.776499 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:33.776475    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-kube-api-access-bmlfn" (OuterVolumeSpecName: "kube-api-access-bmlfn") pod "334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f" (UID: "334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f"). InnerVolumeSpecName "kube-api-access-bmlfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:23:33.776601 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:33.776521    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f" (UID: "334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:23:33.776661 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:33.776638    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f" (UID: "334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:23:33.874551 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:33.874452    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-istio-token\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:23:33.874551 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:33.874484    2560 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-istio-kubeconfig\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:23:33.874551 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:33.874494    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmlfn\" (UniqueName: \"kubernetes.io/projected/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-kube-api-access-bmlfn\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:23:33.874551 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:33.874504    2560 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-istio-csr-ca-configmap\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:23:33.874551 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:33.874513    2560 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-cacerts\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:23:33.874551 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:33.874525    2560 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-istio-csr-dns-cert\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:23:33.874551 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:33.874538    2560 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f-local-certs\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:23:34.302531 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:34.302495    2560 generic.go:356] "Generic (PLEG): container finished" podID="334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f" containerID="80f9e80c5298dbb9569c61ff88850637fa41365ce13d89df69d0e518c5811968" exitCode=0
Sep 08 15:23:34.302920 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:34.302589    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr" event={"ID":"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f","Type":"ContainerDied","Data":"80f9e80c5298dbb9569c61ff88850637fa41365ce13d89df69d0e518c5811968"}
Sep 08 15:23:34.302920 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:34.302908    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"
Sep 08 15:23:34.303035 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:34.302943    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr" event={"ID":"334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f","Type":"ContainerDied","Data":"9a8d20691eb4bbc4057c9d5ad11e9f54de588b69c3d551e4a7182dbeb71e7737"}
Sep 08 15:23:34.303035 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:34.302984    2560 scope.go:117] "RemoveContainer" containerID="80f9e80c5298dbb9569c61ff88850637fa41365ce13d89df69d0e518c5811968"
Sep 08 15:23:34.310988 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:34.310969    2560 scope.go:117] "RemoveContainer" containerID="80f9e80c5298dbb9569c61ff88850637fa41365ce13d89df69d0e518c5811968"
Sep 08 15:23:34.311274 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:23:34.311255    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80f9e80c5298dbb9569c61ff88850637fa41365ce13d89df69d0e518c5811968\": container with ID starting with 80f9e80c5298dbb9569c61ff88850637fa41365ce13d89df69d0e518c5811968 not found: ID does not exist" containerID="80f9e80c5298dbb9569c61ff88850637fa41365ce13d89df69d0e518c5811968"
Sep 08 15:23:34.311337 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:34.311285    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80f9e80c5298dbb9569c61ff88850637fa41365ce13d89df69d0e518c5811968"} err="failed to get container status \"80f9e80c5298dbb9569c61ff88850637fa41365ce13d89df69d0e518c5811968\": rpc error: code = NotFound desc = could not find container \"80f9e80c5298dbb9569c61ff88850637fa41365ce13d89df69d0e518c5811968\": container with ID starting with 80f9e80c5298dbb9569c61ff88850637fa41365ce13d89df69d0e518c5811968 not found: ID does not exist"
Sep 08 15:23:34.345158 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:34.345091    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"]
Sep 08 15:23:34.357147 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:34.357117    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sxqmr"]
Sep 08 15:23:34.396437 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:34.396399    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:34.483839 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:34.483810    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-qlzp2"]
Sep 08 15:23:34.484157 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:34.484139    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f" containerName="discovery"
Sep 08 15:23:34.484157 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:34.484156    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f" containerName="discovery"
Sep 08 15:23:34.484289 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:34.484227    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f" containerName="discovery"
Sep 08 15:23:34.490459 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:34.487668    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:34.490459 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:34.488986    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qlzp2"
Sep 08 15:23:34.491918 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:34.491887    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 08 15:23:34.492231 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:34.492207    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-vclsq\""
Sep 08 15:23:34.501110 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:34.501083    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-qlzp2"]
Sep 08 15:23:34.580665 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:34.580588    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c3b388f-7424-47df-8af6-a1c1dc717334-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-qlzp2\" (UID: \"2c3b388f-7424-47df-8af6-a1c1dc717334\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qlzp2"
Sep 08 15:23:34.580665 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:34.580626    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7thjf\" (UniqueName: \"kubernetes.io/projected/2c3b388f-7424-47df-8af6-a1c1dc717334-kube-api-access-7thjf\") pod \"llmisvc-controller-manager-777c6bd7d5-qlzp2\" (UID: \"2c3b388f-7424-47df-8af6-a1c1dc717334\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qlzp2"
Sep 08 15:23:34.581085 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:34.581052    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:34.681427 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:34.681394    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c3b388f-7424-47df-8af6-a1c1dc717334-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-qlzp2\" (UID: \"2c3b388f-7424-47df-8af6-a1c1dc717334\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qlzp2"
Sep 08 15:23:34.681427 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:34.681432    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7thjf\" (UniqueName: \"kubernetes.io/projected/2c3b388f-7424-47df-8af6-a1c1dc717334-kube-api-access-7thjf\") pod \"llmisvc-controller-manager-777c6bd7d5-qlzp2\" (UID: \"2c3b388f-7424-47df-8af6-a1c1dc717334\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qlzp2"
Sep 08 15:23:34.684057 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:34.684023    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c3b388f-7424-47df-8af6-a1c1dc717334-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-qlzp2\" (UID: \"2c3b388f-7424-47df-8af6-a1c1dc717334\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qlzp2"
Sep 08 15:23:34.692808 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:34.692783    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7thjf\" (UniqueName: \"kubernetes.io/projected/2c3b388f-7424-47df-8af6-a1c1dc717334-kube-api-access-7thjf\") pod \"llmisvc-controller-manager-777c6bd7d5-qlzp2\" (UID: \"2c3b388f-7424-47df-8af6-a1c1dc717334\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qlzp2"
Sep 08 15:23:34.716305 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:34.716257    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:34.800953 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:34.800921    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qlzp2"
Sep 08 15:23:35.135570 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:35.135540    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-qlzp2"]
Sep 08 15:23:35.136203 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:23:35.136178    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2c3b388f_7424_47df_8af6_a1c1dc717334.slice/crio-fc107d1183e42bf46e71b1f8fde832623bce07f5bd0c9a1bd0dee4409e44008d WatchSource:0}: Error finding container fc107d1183e42bf46e71b1f8fde832623bce07f5bd0c9a1bd0dee4409e44008d: Status 404 returned error can't find the container with id fc107d1183e42bf46e71b1f8fde832623bce07f5bd0c9a1bd0dee4409e44008d
Sep 08 15:23:35.308952 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:35.308919    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qlzp2" event={"ID":"2c3b388f-7424-47df-8af6-a1c1dc717334","Type":"ContainerStarted","Data":"fc107d1183e42bf46e71b1f8fde832623bce07f5bd0c9a1bd0dee4409e44008d"}
Sep 08 15:23:35.721284 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:35.721251    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:35.800158 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:35.800114    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f" path="/var/lib/kubelet/pods/334bf5eb-b67b-4aa1-b1f0-a2d3eb7f4b8f/volumes"
Sep 08 15:23:37.816321 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:37.816286    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:37.896743 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:37.896712    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:40.285206 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:40.285113    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-kn4vl"
Sep 08 15:23:40.335336 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:40.335302    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qlzp2" event={"ID":"2c3b388f-7424-47df-8af6-a1c1dc717334","Type":"ContainerStarted","Data":"ea565ee923264f943a6983b5d73f9076003c332d48f8e8bec82b0be5e5e5252e"}
Sep 08 15:23:40.335503 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:40.335438    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qlzp2"
Sep 08 15:23:40.356267 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:23:40.356216    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qlzp2" podStartSLOduration=1.5026853230000001 podStartE2EDuration="6.356200454s" totalImagesPullingTime="4.853515131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:23:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:23:40.353315952 +0000 UTC m=+373.065258447" watchObservedRunningTime="2026-09-08 15:23:40.356200454 +0000 UTC m=+373.068142945"
Sep 08 15:24:05.848360 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:05.848330    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:05.907371 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:05.907320    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:06.408063 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:06.408019    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:06.579535 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:06.579502    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:06.633836 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:06.633789    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:11.342973 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:11.342936    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qlzp2"
Sep 08 15:24:16.682393 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:16.682356    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:16.784626 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:16.784567    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:16.926101 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:16.926054    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:24.220039 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:24.220004    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:24.323419 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:24.323381    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:26.101426 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:26.101395    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:26.188222 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:26.188189    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:32.964735 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:32.964695    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:38.120583 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:38.120553    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:38.915900 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:38.915862    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:39.711056 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:39.711020    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:42.163682 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:42.163646    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:42.386003 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:42.385972    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:42.502751 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:42.502711    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d"]
Sep 08 15:24:42.507123 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:42.507106    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d"
Sep 08 15:24:42.508992 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:42.508971    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-z9pcn\""
Sep 08 15:24:42.509143 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:42.508999    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 08 15:24:42.509215 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:42.509181    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 08 15:24:42.520334 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:42.520312    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d"]
Sep 08 15:24:42.618965 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:42.618929    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e123b3e-1f46-4289-b5d7-264efbddc6f6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-ppx5d\" (UID: \"7e123b3e-1f46-4289-b5d7-264efbddc6f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d"
Sep 08 15:24:42.619163 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:42.618975    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ljlg\" (UniqueName: \"kubernetes.io/projected/7e123b3e-1f46-4289-b5d7-264efbddc6f6-kube-api-access-8ljlg\") pod \"kuadrant-operator-controller-manager-58fc56757c-ppx5d\" (UID: \"7e123b3e-1f46-4289-b5d7-264efbddc6f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d"
Sep 08 15:24:42.719664 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:42.719628    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e123b3e-1f46-4289-b5d7-264efbddc6f6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-ppx5d\" (UID: \"7e123b3e-1f46-4289-b5d7-264efbddc6f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d"
Sep 08 15:24:42.719827 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:42.719678    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ljlg\" (UniqueName: \"kubernetes.io/projected/7e123b3e-1f46-4289-b5d7-264efbddc6f6-kube-api-access-8ljlg\") pod \"kuadrant-operator-controller-manager-58fc56757c-ppx5d\" (UID: \"7e123b3e-1f46-4289-b5d7-264efbddc6f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d"
Sep 08 15:24:42.720033 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:42.720013    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e123b3e-1f46-4289-b5d7-264efbddc6f6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-ppx5d\" (UID: \"7e123b3e-1f46-4289-b5d7-264efbddc6f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d"
Sep 08 15:24:42.733925 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:42.733897    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ljlg\" (UniqueName: \"kubernetes.io/projected/7e123b3e-1f46-4289-b5d7-264efbddc6f6-kube-api-access-8ljlg\") pod \"kuadrant-operator-controller-manager-58fc56757c-ppx5d\" (UID: \"7e123b3e-1f46-4289-b5d7-264efbddc6f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d"
Sep 08 15:24:42.816814 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:42.816723    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d"
Sep 08 15:24:43.104525 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:43.104447    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d"]
Sep 08 15:24:43.107526 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:24:43.107490    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e123b3e_1f46_4289_b5d7_264efbddc6f6.slice/crio-9616f36f2ae99cdd745f8e2e8a6eba91e37aaf05b6e4f3e5388f110d06255ff1 WatchSource:0}: Error finding container 9616f36f2ae99cdd745f8e2e8a6eba91e37aaf05b6e4f3e5388f110d06255ff1: Status 404 returned error can't find the container with id 9616f36f2ae99cdd745f8e2e8a6eba91e37aaf05b6e4f3e5388f110d06255ff1
Sep 08 15:24:43.125397 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:43.125376    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:43.601030 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:43.600997    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d" event={"ID":"7e123b3e-1f46-4289-b5d7-264efbddc6f6","Type":"ContainerStarted","Data":"9616f36f2ae99cdd745f8e2e8a6eba91e37aaf05b6e4f3e5388f110d06255ff1"}
Sep 08 15:24:44.748486 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:44.748447    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:44.857214 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:44.857178    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:45.313485 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:45.313443    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:47.572271 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:47.572229    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:47.678668 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:47.678624    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:47.796089 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:47.796042    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:48.627103 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:48.627061    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d" event={"ID":"7e123b3e-1f46-4289-b5d7-264efbddc6f6","Type":"ContainerStarted","Data":"c10eb053bcfa1de864b6c537f5e007d2dab4b369982d60584ae8021ec05fd161"}
Sep 08 15:24:48.627467 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:48.627151    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d"
Sep 08 15:24:48.651999 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:48.651938    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d" podStartSLOduration=2.167264335 podStartE2EDuration="6.651924519s" totalImagesPullingTime="4.484660184s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:24:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:24:48.649312416 +0000 UTC m=+441.361254906" watchObservedRunningTime="2026-09-08 15:24:48.651924519 +0000 UTC m=+441.363867010"
Sep 08 15:24:49.369197 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:49.369163    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:50.335308 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:50.335226    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:50.424445 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:50.424414    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:53.132881 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:53.132849    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:53.244775 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:53.244732    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:53.515168 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:53.515137    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:53.576215 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:53.576159    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:53.916016 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:53.915935    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:53.974064 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:53.974034    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:56.530406 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:56.530371    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:56.629765 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:56.629733    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:59.581946 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:59.581913    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:59.634116 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:59.634089    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d"
Sep 08 15:24:59.657900 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:24:59.657870    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:01.281929 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:01.281896    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bjjf8"]
Sep 08 15:25:01.289138 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:01.289117    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bjjf8"
Sep 08 15:25:01.299916 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:01.299891    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bjjf8"]
Sep 08 15:25:01.376965 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:01.376926    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt4dh\" (UniqueName: \"kubernetes.io/projected/1a929247-18b0-489b-8d67-faa2b403821e-kube-api-access-xt4dh\") pod \"kuadrant-operator-controller-manager-68cf589589-bjjf8\" (UID: \"1a929247-18b0-489b-8d67-faa2b403821e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bjjf8"
Sep 08 15:25:01.377160 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:01.377001    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a929247-18b0-489b-8d67-faa2b403821e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-bjjf8\" (UID: \"1a929247-18b0-489b-8d67-faa2b403821e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bjjf8"
Sep 08 15:25:01.478255 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:01.478220    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt4dh\" (UniqueName: \"kubernetes.io/projected/1a929247-18b0-489b-8d67-faa2b403821e-kube-api-access-xt4dh\") pod \"kuadrant-operator-controller-manager-68cf589589-bjjf8\" (UID: \"1a929247-18b0-489b-8d67-faa2b403821e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bjjf8"
Sep 08 15:25:01.478419 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:01.478288    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a929247-18b0-489b-8d67-faa2b403821e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-bjjf8\" (UID: \"1a929247-18b0-489b-8d67-faa2b403821e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bjjf8"
Sep 08 15:25:01.478639 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:01.478621    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a929247-18b0-489b-8d67-faa2b403821e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-bjjf8\" (UID: \"1a929247-18b0-489b-8d67-faa2b403821e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bjjf8"
Sep 08 15:25:01.505867 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:01.505828    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt4dh\" (UniqueName: \"kubernetes.io/projected/1a929247-18b0-489b-8d67-faa2b403821e-kube-api-access-xt4dh\") pod \"kuadrant-operator-controller-manager-68cf589589-bjjf8\" (UID: \"1a929247-18b0-489b-8d67-faa2b403821e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bjjf8"
Sep 08 15:25:01.599349 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:01.599261    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bjjf8"
Sep 08 15:25:01.880150 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:01.880040    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bjjf8"]
Sep 08 15:25:01.883343 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:25:01.883312    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a929247_18b0_489b_8d67_faa2b403821e.slice/crio-132b495745a015f4917eceb667a00f7b1ce3907b8a812674dd75c58a7fe5fe1e WatchSource:0}: Error finding container 132b495745a015f4917eceb667a00f7b1ce3907b8a812674dd75c58a7fe5fe1e: Status 404 returned error can't find the container with id 132b495745a015f4917eceb667a00f7b1ce3907b8a812674dd75c58a7fe5fe1e
Sep 08 15:25:01.898101 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:01.898054    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d"]
Sep 08 15:25:01.898308 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:01.898263    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d" podUID="7e123b3e-1f46-4289-b5d7-264efbddc6f6" containerName="manager" containerID="cri-o://c10eb053bcfa1de864b6c537f5e007d2dab4b369982d60584ae8021ec05fd161" gracePeriod=2
Sep 08 15:25:01.912445 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:25:01.912405    2560 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"7e123b3e-1f46-4289-b5d7-264efbddc6f6\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-58fc56757c-ppx5d\": pods \"kuadrant-operator-controller-manager-58fc56757c-ppx5d\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d"
Sep 08 15:25:01.913256 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:01.913232    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d"]
Sep 08 15:25:01.928300 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:01.928221    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bjjf8"]
Sep 08 15:25:01.929893 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:01.929865    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-sdnvs"]
Sep 08 15:25:01.930254 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:01.930238    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e123b3e-1f46-4289-b5d7-264efbddc6f6" containerName="manager"
Sep 08 15:25:01.930330 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:01.930255    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e123b3e-1f46-4289-b5d7-264efbddc6f6" containerName="manager"
Sep 08 15:25:01.930330 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:01.930310    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e123b3e-1f46-4289-b5d7-264efbddc6f6" containerName="manager"
Sep 08 15:25:01.933244 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:01.933224    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-sdnvs"
Sep 08 15:25:01.935216 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:25:01.935188    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-ppx5d\" is forbidden: User \"system:node:ip-10-0-137-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-43.ec2.internal' and this object" podUID="7e123b3e-1f46-4289-b5d7-264efbddc6f6" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d"
Sep 08 15:25:01.942798 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:01.942769    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bjjf8"]
Sep 08 15:25:01.951849 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:01.951821    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-sdnvs"]
Sep 08 15:25:01.966251 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:01.966229    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w2ngm"]
Sep 08 15:25:01.969530 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:01.969509    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w2ngm"
Sep 08 15:25:01.983158 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:25:01.983125    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-ppx5d\" is forbidden: User \"system:node:ip-10-0-137-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-43.ec2.internal' and this object" podUID="7e123b3e-1f46-4289-b5d7-264efbddc6f6" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d"
Sep 08 15:25:01.987300 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:01.987275    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w2ngm"]
Sep 08 15:25:02.082557 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.082527    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a093255-53a8-4f4f-8f1d-c4a6a0c7252c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-w2ngm\" (UID: \"7a093255-53a8-4f4f-8f1d-c4a6a0c7252c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w2ngm"
Sep 08 15:25:02.082723 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.082576    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b20a6fa8-8b3e-459f-84a4-27867cc753d9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-sdnvs\" (UID: \"b20a6fa8-8b3e-459f-84a4-27867cc753d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-sdnvs"
Sep 08 15:25:02.082723 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.082594    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfp4q\" (UniqueName: \"kubernetes.io/projected/7a093255-53a8-4f4f-8f1d-c4a6a0c7252c-kube-api-access-pfp4q\") pod \"kuadrant-operator-controller-manager-68cf589589-w2ngm\" (UID: \"7a093255-53a8-4f4f-8f1d-c4a6a0c7252c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w2ngm"
Sep 08 15:25:02.082723 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.082662    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqtqf\" (UniqueName: \"kubernetes.io/projected/b20a6fa8-8b3e-459f-84a4-27867cc753d9-kube-api-access-wqtqf\") pod \"kuadrant-operator-controller-manager-58fc56757c-sdnvs\" (UID: \"b20a6fa8-8b3e-459f-84a4-27867cc753d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-sdnvs"
Sep 08 15:25:02.125594 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.125568    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d"
Sep 08 15:25:02.127320 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:25:02.127296    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-ppx5d\" is forbidden: User \"system:node:ip-10-0-137-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-43.ec2.internal' and this object" podUID="7e123b3e-1f46-4289-b5d7-264efbddc6f6" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d"
Sep 08 15:25:02.183773 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.183745    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ljlg\" (UniqueName: \"kubernetes.io/projected/7e123b3e-1f46-4289-b5d7-264efbddc6f6-kube-api-access-8ljlg\") pod \"7e123b3e-1f46-4289-b5d7-264efbddc6f6\" (UID: \"7e123b3e-1f46-4289-b5d7-264efbddc6f6\") "
Sep 08 15:25:02.183890 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.183787    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e123b3e-1f46-4289-b5d7-264efbddc6f6-extensions-socket-volume\") pod \"7e123b3e-1f46-4289-b5d7-264efbddc6f6\" (UID: \"7e123b3e-1f46-4289-b5d7-264efbddc6f6\") "
Sep 08 15:25:02.183890 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.183868    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b20a6fa8-8b3e-459f-84a4-27867cc753d9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-sdnvs\" (UID: \"b20a6fa8-8b3e-459f-84a4-27867cc753d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-sdnvs"
Sep 08 15:25:02.184007 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.183904    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfp4q\" (UniqueName: \"kubernetes.io/projected/7a093255-53a8-4f4f-8f1d-c4a6a0c7252c-kube-api-access-pfp4q\") pod \"kuadrant-operator-controller-manager-68cf589589-w2ngm\" (UID: \"7a093255-53a8-4f4f-8f1d-c4a6a0c7252c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w2ngm"
Sep 08 15:25:02.184007 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.183946    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqtqf\" (UniqueName: \"kubernetes.io/projected/b20a6fa8-8b3e-459f-84a4-27867cc753d9-kube-api-access-wqtqf\") pod \"kuadrant-operator-controller-manager-58fc56757c-sdnvs\" (UID: \"b20a6fa8-8b3e-459f-84a4-27867cc753d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-sdnvs"
Sep 08 15:25:02.184148 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.184040    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a093255-53a8-4f4f-8f1d-c4a6a0c7252c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-w2ngm\" (UID: \"7a093255-53a8-4f4f-8f1d-c4a6a0c7252c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w2ngm"
Sep 08 15:25:02.184369 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.184344    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b20a6fa8-8b3e-459f-84a4-27867cc753d9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-sdnvs\" (UID: \"b20a6fa8-8b3e-459f-84a4-27867cc753d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-sdnvs"
Sep 08 15:25:02.184443 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.184420    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a093255-53a8-4f4f-8f1d-c4a6a0c7252c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-w2ngm\" (UID: \"7a093255-53a8-4f4f-8f1d-c4a6a0c7252c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w2ngm"
Sep 08 15:25:02.184499 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.184424    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e123b3e-1f46-4289-b5d7-264efbddc6f6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7e123b3e-1f46-4289-b5d7-264efbddc6f6" (UID: "7e123b3e-1f46-4289-b5d7-264efbddc6f6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:25:02.186047 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.186026    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e123b3e-1f46-4289-b5d7-264efbddc6f6-kube-api-access-8ljlg" (OuterVolumeSpecName: "kube-api-access-8ljlg") pod "7e123b3e-1f46-4289-b5d7-264efbddc6f6" (UID: "7e123b3e-1f46-4289-b5d7-264efbddc6f6"). InnerVolumeSpecName "kube-api-access-8ljlg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:25:02.193752 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.193731    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqtqf\" (UniqueName: \"kubernetes.io/projected/b20a6fa8-8b3e-459f-84a4-27867cc753d9-kube-api-access-wqtqf\") pod \"kuadrant-operator-controller-manager-58fc56757c-sdnvs\" (UID: \"b20a6fa8-8b3e-459f-84a4-27867cc753d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-sdnvs"
Sep 08 15:25:02.195057 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.195037    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfp4q\" (UniqueName: \"kubernetes.io/projected/7a093255-53a8-4f4f-8f1d-c4a6a0c7252c-kube-api-access-pfp4q\") pod \"kuadrant-operator-controller-manager-68cf589589-w2ngm\" (UID: \"7a093255-53a8-4f4f-8f1d-c4a6a0c7252c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w2ngm"
Sep 08 15:25:02.284536 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.284499    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ljlg\" (UniqueName: \"kubernetes.io/projected/7e123b3e-1f46-4289-b5d7-264efbddc6f6-kube-api-access-8ljlg\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:25:02.284536 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.284529    2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e123b3e-1f46-4289-b5d7-264efbddc6f6-extensions-socket-volume\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:25:02.295476 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.295440    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-sdnvs"
Sep 08 15:25:02.302733 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.302706    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w2ngm"
Sep 08 15:25:02.586859 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.586831    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-sdnvs"]
Sep 08 15:25:02.590592 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:25:02.590555    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb20a6fa8_8b3e_459f_84a4_27867cc753d9.slice/crio-b4e82eadec2b6df33da088470847af2b3dcf7b85ec1a97ec14dd659f2e48a4ab WatchSource:0}: Error finding container b4e82eadec2b6df33da088470847af2b3dcf7b85ec1a97ec14dd659f2e48a4ab: Status 404 returned error can't find the container with id b4e82eadec2b6df33da088470847af2b3dcf7b85ec1a97ec14dd659f2e48a4ab
Sep 08 15:25:02.591647 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.591622    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w2ngm"]
Sep 08 15:25:02.595170 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:25:02.595149    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a093255_53a8_4f4f_8f1d_c4a6a0c7252c.slice/crio-e334c352fecd6dc84be99bff545aa1ee9c59ca40dddee1be2051e81db574de29 WatchSource:0}: Error finding container e334c352fecd6dc84be99bff545aa1ee9c59ca40dddee1be2051e81db574de29: Status 404 returned error can't find the container with id e334c352fecd6dc84be99bff545aa1ee9c59ca40dddee1be2051e81db574de29
Sep 08 15:25:02.686266 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.686236    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-sdnvs" event={"ID":"b20a6fa8-8b3e-459f-84a4-27867cc753d9","Type":"ContainerStarted","Data":"867172e9fccbeaafa44214acc302388929a20278fb1925aeefc9367a7af1b745"}
Sep 08 15:25:02.686394 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.686295    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-sdnvs" event={"ID":"b20a6fa8-8b3e-459f-84a4-27867cc753d9","Type":"ContainerStarted","Data":"b4e82eadec2b6df33da088470847af2b3dcf7b85ec1a97ec14dd659f2e48a4ab"}
Sep 08 15:25:02.686394 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.686350    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-sdnvs"
Sep 08 15:25:02.687601 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:25:02.687576    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-ppx5d\" is forbidden: User \"system:node:ip-10-0-137-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-43.ec2.internal' and this object" podUID="7e123b3e-1f46-4289-b5d7-264efbddc6f6" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d"
Sep 08 15:25:02.687745 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.687697    2560 generic.go:356] "Generic (PLEG): container finished" podID="7e123b3e-1f46-4289-b5d7-264efbddc6f6" containerID="c10eb053bcfa1de864b6c537f5e007d2dab4b369982d60584ae8021ec05fd161" exitCode=0
Sep 08 15:25:02.687745 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.687719    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d"
Sep 08 15:25:02.687850 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.687774    2560 scope.go:117] "RemoveContainer" containerID="c10eb053bcfa1de864b6c537f5e007d2dab4b369982d60584ae8021ec05fd161"
Sep 08 15:25:02.689626 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.689603    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w2ngm" event={"ID":"7a093255-53a8-4f4f-8f1d-c4a6a0c7252c","Type":"ContainerStarted","Data":"54e37ebff45e7a2d2232c8d4da43bd37c1d98e62e3a1c5c2523f0f8709b25dc9"}
Sep 08 15:25:02.689745 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.689648    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w2ngm" event={"ID":"7a093255-53a8-4f4f-8f1d-c4a6a0c7252c","Type":"ContainerStarted","Data":"e334c352fecd6dc84be99bff545aa1ee9c59ca40dddee1be2051e81db574de29"}
Sep 08 15:25:02.690017 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.689997    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w2ngm"
Sep 08 15:25:02.691661 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.691633    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bjjf8" podUID="1a929247-18b0-489b-8d67-faa2b403821e" containerName="manager" containerID="cri-o://3573c300b429cd9e392a213f4e408629dd930e96feb64dce0203886df5e7a3b9" gracePeriod=2
Sep 08 15:25:02.697154 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.697117    2560 scope.go:117] "RemoveContainer" containerID="c10eb053bcfa1de864b6c537f5e007d2dab4b369982d60584ae8021ec05fd161"
Sep 08 15:25:02.697450 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:25:02.697428    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c10eb053bcfa1de864b6c537f5e007d2dab4b369982d60584ae8021ec05fd161\": container with ID starting with c10eb053bcfa1de864b6c537f5e007d2dab4b369982d60584ae8021ec05fd161 not found: ID does not exist" containerID="c10eb053bcfa1de864b6c537f5e007d2dab4b369982d60584ae8021ec05fd161"
Sep 08 15:25:02.697522 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.697462    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c10eb053bcfa1de864b6c537f5e007d2dab4b369982d60584ae8021ec05fd161"} err="failed to get container status \"c10eb053bcfa1de864b6c537f5e007d2dab4b369982d60584ae8021ec05fd161\": rpc error: code = NotFound desc = could not find container \"c10eb053bcfa1de864b6c537f5e007d2dab4b369982d60584ae8021ec05fd161\": container with ID starting with c10eb053bcfa1de864b6c537f5e007d2dab4b369982d60584ae8021ec05fd161 not found: ID does not exist"
Sep 08 15:25:02.711230 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:25:02.711203    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-ppx5d\" is forbidden: User \"system:node:ip-10-0-137-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-43.ec2.internal' and this object" podUID="7e123b3e-1f46-4289-b5d7-264efbddc6f6" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d"
Sep 08 15:25:02.711763 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.711724    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-sdnvs" podStartSLOduration=1.7117098830000002 podStartE2EDuration="1.711709883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:25:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:25:02.709573356 +0000 UTC m=+455.421515847" watchObservedRunningTime="2026-09-08 15:25:02.711709883 +0000 UTC m=+455.423652374"
Sep 08 15:25:02.712959 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:25:02.712934    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-68cf589589-bjjf8\" is forbidden: User \"system:node:ip-10-0-137-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-43.ec2.internal' and this object" podUID="1a929247-18b0-489b-8d67-faa2b403821e" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bjjf8"
Sep 08 15:25:02.740406 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.740342    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w2ngm" podStartSLOduration=1.7403197860000001 podStartE2EDuration="1.740319786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:25:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:25:02.738486885 +0000 UTC m=+455.450429375" watchObservedRunningTime="2026-09-08 15:25:02.740319786 +0000 UTC m=+455.452262289"
Sep 08 15:25:02.920037 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.920013    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bjjf8"
Sep 08 15:25:02.921934 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:25:02.921909    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-ppx5d\" is forbidden: User \"system:node:ip-10-0-137-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-43.ec2.internal' and this object" podUID="7e123b3e-1f46-4289-b5d7-264efbddc6f6" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d"
Sep 08 15:25:02.923811 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:25:02.923791    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-68cf589589-bjjf8\" is forbidden: User \"system:node:ip-10-0-137-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-43.ec2.internal' and this object" podUID="1a929247-18b0-489b-8d67-faa2b403821e" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bjjf8"
Sep 08 15:25:02.989586 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.989553    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xt4dh\" (UniqueName: \"kubernetes.io/projected/1a929247-18b0-489b-8d67-faa2b403821e-kube-api-access-xt4dh\") pod \"1a929247-18b0-489b-8d67-faa2b403821e\" (UID: \"1a929247-18b0-489b-8d67-faa2b403821e\") "
Sep 08 15:25:02.989768 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.989615    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a929247-18b0-489b-8d67-faa2b403821e-extensions-socket-volume\") pod \"1a929247-18b0-489b-8d67-faa2b403821e\" (UID: \"1a929247-18b0-489b-8d67-faa2b403821e\") "
Sep 08 15:25:02.989881 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.989855    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a929247-18b0-489b-8d67-faa2b403821e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1a929247-18b0-489b-8d67-faa2b403821e" (UID: "1a929247-18b0-489b-8d67-faa2b403821e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:25:02.991482 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:02.991461    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a929247-18b0-489b-8d67-faa2b403821e-kube-api-access-xt4dh" (OuterVolumeSpecName: "kube-api-access-xt4dh") pod "1a929247-18b0-489b-8d67-faa2b403821e" (UID: "1a929247-18b0-489b-8d67-faa2b403821e"). InnerVolumeSpecName "kube-api-access-xt4dh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:25:03.090658 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:03.090563    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xt4dh\" (UniqueName: \"kubernetes.io/projected/1a929247-18b0-489b-8d67-faa2b403821e-kube-api-access-xt4dh\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:25:03.090658 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:03.090604    2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a929247-18b0-489b-8d67-faa2b403821e-extensions-socket-volume\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:25:03.699469 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:03.699416    2560 generic.go:356] "Generic (PLEG): container finished" podID="1a929247-18b0-489b-8d67-faa2b403821e" containerID="3573c300b429cd9e392a213f4e408629dd930e96feb64dce0203886df5e7a3b9" exitCode=2
Sep 08 15:25:03.699469 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:03.699467    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bjjf8"
Sep 08 15:25:03.699993 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:03.699510    2560 scope.go:117] "RemoveContainer" containerID="3573c300b429cd9e392a213f4e408629dd930e96feb64dce0203886df5e7a3b9"
Sep 08 15:25:03.701255 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:25:03.701226    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-68cf589589-bjjf8\" is forbidden: User \"system:node:ip-10-0-137-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-43.ec2.internal' and this object" podUID="1a929247-18b0-489b-8d67-faa2b403821e" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bjjf8"
Sep 08 15:25:03.702710 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:25:03.702687    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-ppx5d\" is forbidden: User \"system:node:ip-10-0-137-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-43.ec2.internal' and this object" podUID="7e123b3e-1f46-4289-b5d7-264efbddc6f6" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d"
Sep 08 15:25:03.708402 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:03.708384    2560 scope.go:117] "RemoveContainer" containerID="3573c300b429cd9e392a213f4e408629dd930e96feb64dce0203886df5e7a3b9"
Sep 08 15:25:03.708681 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:25:03.708663    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3573c300b429cd9e392a213f4e408629dd930e96feb64dce0203886df5e7a3b9\": container with ID starting with 3573c300b429cd9e392a213f4e408629dd930e96feb64dce0203886df5e7a3b9 not found: ID does not exist" containerID="3573c300b429cd9e392a213f4e408629dd930e96feb64dce0203886df5e7a3b9"
Sep 08 15:25:03.708722 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:03.708688    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3573c300b429cd9e392a213f4e408629dd930e96feb64dce0203886df5e7a3b9"} err="failed to get container status \"3573c300b429cd9e392a213f4e408629dd930e96feb64dce0203886df5e7a3b9\": rpc error: code = NotFound desc = could not find container \"3573c300b429cd9e392a213f4e408629dd930e96feb64dce0203886df5e7a3b9\": container with ID starting with 3573c300b429cd9e392a213f4e408629dd930e96feb64dce0203886df5e7a3b9 not found: ID does not exist"
Sep 08 15:25:03.709635 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:25:03.709615    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-68cf589589-bjjf8\" is forbidden: User \"system:node:ip-10-0-137-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-43.ec2.internal' and this object" podUID="1a929247-18b0-489b-8d67-faa2b403821e" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-bjjf8"
Sep 08 15:25:03.710956 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:25:03.710936    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-ppx5d\" is forbidden: User \"system:node:ip-10-0-137-43.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-43.ec2.internal' and this object" podUID="7e123b3e-1f46-4289-b5d7-264efbddc6f6" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-ppx5d"
Sep 08 15:25:03.799523 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:03.799438    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a929247-18b0-489b-8d67-faa2b403821e" path="/var/lib/kubelet/pods/1a929247-18b0-489b-8d67-faa2b403821e/volumes"
Sep 08 15:25:03.799889 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:03.799868    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e123b3e-1f46-4289-b5d7-264efbddc6f6" path="/var/lib/kubelet/pods/7e123b3e-1f46-4289-b5d7-264efbddc6f6/volumes"
Sep 08 15:25:06.139007 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:06.138968    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:06.626094 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:06.626043    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:09.147584 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:09.147556    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:09.248027 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:09.247997    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:13.702097 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:13.702042    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-sdnvs"
Sep 08 15:25:14.355249 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:14.355216    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:14.707870 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:14.707843    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w2ngm"
Sep 08 15:25:14.768520 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:14.768484    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-sdnvs"]
Sep 08 15:25:14.768781 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:14.768749    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-sdnvs" podUID="b20a6fa8-8b3e-459f-84a4-27867cc753d9" containerName="manager" containerID="cri-o://867172e9fccbeaafa44214acc302388929a20278fb1925aeefc9367a7af1b745" gracePeriod=10
Sep 08 15:25:14.796490 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:14.796437    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:14.895808 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:14.895773    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:14.998802 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:14.998774    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6l8mj"]
Sep 08 15:25:14.999199 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:14.999181    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a929247-18b0-489b-8d67-faa2b403821e" containerName="manager"
Sep 08 15:25:14.999199 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:14.999201    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a929247-18b0-489b-8d67-faa2b403821e" containerName="manager"
Sep 08 15:25:14.999342 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:14.999281    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a929247-18b0-489b-8d67-faa2b403821e" containerName="manager"
Sep 08 15:25:15.000603 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.000563    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:15.001697 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.001676    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6l8mj"
Sep 08 15:25:15.007741 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.007720    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-sdnvs"
Sep 08 15:25:15.023545 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.023519    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6l8mj"]
Sep 08 15:25:15.091258 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.091225    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b20a6fa8-8b3e-459f-84a4-27867cc753d9-extensions-socket-volume\") pod \"b20a6fa8-8b3e-459f-84a4-27867cc753d9\" (UID: \"b20a6fa8-8b3e-459f-84a4-27867cc753d9\") "
Sep 08 15:25:15.091394 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.091325    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqtqf\" (UniqueName: \"kubernetes.io/projected/b20a6fa8-8b3e-459f-84a4-27867cc753d9-kube-api-access-wqtqf\") pod \"b20a6fa8-8b3e-459f-84a4-27867cc753d9\" (UID: \"b20a6fa8-8b3e-459f-84a4-27867cc753d9\") "
Sep 08 15:25:15.091476 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.091458    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hrsp\" (UniqueName: \"kubernetes.io/projected/8693ed62-2396-4039-b639-8e8063f8dc1f-kube-api-access-6hrsp\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-6l8mj\" (UID: \"8693ed62-2396-4039-b639-8e8063f8dc1f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6l8mj"
Sep 08 15:25:15.091518 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.091497    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8693ed62-2396-4039-b639-8e8063f8dc1f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-6l8mj\" (UID: \"8693ed62-2396-4039-b639-8e8063f8dc1f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6l8mj"
Sep 08 15:25:15.091601 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.091580    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b20a6fa8-8b3e-459f-84a4-27867cc753d9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b20a6fa8-8b3e-459f-84a4-27867cc753d9" (UID: "b20a6fa8-8b3e-459f-84a4-27867cc753d9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:25:15.091718 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.091698    2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b20a6fa8-8b3e-459f-84a4-27867cc753d9-extensions-socket-volume\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:25:15.093338 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.093315    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b20a6fa8-8b3e-459f-84a4-27867cc753d9-kube-api-access-wqtqf" (OuterVolumeSpecName: "kube-api-access-wqtqf") pod "b20a6fa8-8b3e-459f-84a4-27867cc753d9" (UID: "b20a6fa8-8b3e-459f-84a4-27867cc753d9"). InnerVolumeSpecName "kube-api-access-wqtqf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:25:15.192342 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.192308    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hrsp\" (UniqueName: \"kubernetes.io/projected/8693ed62-2396-4039-b639-8e8063f8dc1f-kube-api-access-6hrsp\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-6l8mj\" (UID: \"8693ed62-2396-4039-b639-8e8063f8dc1f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6l8mj"
Sep 08 15:25:15.192342 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.192345    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8693ed62-2396-4039-b639-8e8063f8dc1f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-6l8mj\" (UID: \"8693ed62-2396-4039-b639-8e8063f8dc1f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6l8mj"
Sep 08 15:25:15.192541 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.192402    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqtqf\" (UniqueName: \"kubernetes.io/projected/b20a6fa8-8b3e-459f-84a4-27867cc753d9-kube-api-access-wqtqf\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:25:15.192754 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.192736    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8693ed62-2396-4039-b639-8e8063f8dc1f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-6l8mj\" (UID: \"8693ed62-2396-4039-b639-8e8063f8dc1f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6l8mj"
Sep 08 15:25:15.200945 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.200919    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hrsp\" (UniqueName: \"kubernetes.io/projected/8693ed62-2396-4039-b639-8e8063f8dc1f-kube-api-access-6hrsp\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-6l8mj\" (UID: \"8693ed62-2396-4039-b639-8e8063f8dc1f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6l8mj"
Sep 08 15:25:15.314808 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.314718    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6l8mj"
Sep 08 15:25:15.597846 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.597772    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6l8mj"]
Sep 08 15:25:15.611743 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:25:15.611714    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8693ed62_2396_4039_b639_8e8063f8dc1f.slice/crio-45b1ea5c60f0492523b02c2c1a52e2f5048c20343cc2825ba605a0040a31abfb WatchSource:0}: Error finding container 45b1ea5c60f0492523b02c2c1a52e2f5048c20343cc2825ba605a0040a31abfb: Status 404 returned error can't find the container with id 45b1ea5c60f0492523b02c2c1a52e2f5048c20343cc2825ba605a0040a31abfb
Sep 08 15:25:15.765175 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.765138    2560 generic.go:356] "Generic (PLEG): container finished" podID="b20a6fa8-8b3e-459f-84a4-27867cc753d9" containerID="867172e9fccbeaafa44214acc302388929a20278fb1925aeefc9367a7af1b745" exitCode=0
Sep 08 15:25:15.765582 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.765182    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-sdnvs"
Sep 08 15:25:15.765582 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.765226    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-sdnvs" event={"ID":"b20a6fa8-8b3e-459f-84a4-27867cc753d9","Type":"ContainerDied","Data":"867172e9fccbeaafa44214acc302388929a20278fb1925aeefc9367a7af1b745"}
Sep 08 15:25:15.765582 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.765282    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-sdnvs" event={"ID":"b20a6fa8-8b3e-459f-84a4-27867cc753d9","Type":"ContainerDied","Data":"b4e82eadec2b6df33da088470847af2b3dcf7b85ec1a97ec14dd659f2e48a4ab"}
Sep 08 15:25:15.765582 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.765318    2560 scope.go:117] "RemoveContainer" containerID="867172e9fccbeaafa44214acc302388929a20278fb1925aeefc9367a7af1b745"
Sep 08 15:25:15.767262 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.767236    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6l8mj" event={"ID":"8693ed62-2396-4039-b639-8e8063f8dc1f","Type":"ContainerStarted","Data":"89418823ee960d460e0f0c4787e19d2690f0bdd967ffa5f5e97140a0a2dd1551"}
Sep 08 15:25:15.767382 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.767283    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6l8mj" event={"ID":"8693ed62-2396-4039-b639-8e8063f8dc1f","Type":"ContainerStarted","Data":"45b1ea5c60f0492523b02c2c1a52e2f5048c20343cc2825ba605a0040a31abfb"}
Sep 08 15:25:15.767382 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.767322    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6l8mj"
Sep 08 15:25:15.774528 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.774373    2560 scope.go:117] "RemoveContainer" containerID="867172e9fccbeaafa44214acc302388929a20278fb1925aeefc9367a7af1b745"
Sep 08 15:25:15.774664 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:25:15.774646    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"867172e9fccbeaafa44214acc302388929a20278fb1925aeefc9367a7af1b745\": container with ID starting with 867172e9fccbeaafa44214acc302388929a20278fb1925aeefc9367a7af1b745 not found: ID does not exist" containerID="867172e9fccbeaafa44214acc302388929a20278fb1925aeefc9367a7af1b745"
Sep 08 15:25:15.774707 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.774673    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"867172e9fccbeaafa44214acc302388929a20278fb1925aeefc9367a7af1b745"} err="failed to get container status \"867172e9fccbeaafa44214acc302388929a20278fb1925aeefc9367a7af1b745\": rpc error: code = NotFound desc = could not find container \"867172e9fccbeaafa44214acc302388929a20278fb1925aeefc9367a7af1b745\": container with ID starting with 867172e9fccbeaafa44214acc302388929a20278fb1925aeefc9367a7af1b745 not found: ID does not exist"
Sep 08 15:25:15.792814 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.792759    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6l8mj" podStartSLOduration=1.792743671 podStartE2EDuration="1.792743671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:25:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:25:15.79084126 +0000 UTC m=+468.502783751" watchObservedRunningTime="2026-09-08 15:25:15.792743671 +0000 UTC m=+468.504686173"
Sep 08 15:25:15.814926 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.812664    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-sdnvs"]
Sep 08 15:25:15.820079 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:15.820036    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-sdnvs"]
Sep 08 15:25:17.799310 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:17.799275    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b20a6fa8-8b3e-459f-84a4-27867cc753d9" path="/var/lib/kubelet/pods/b20a6fa8-8b3e-459f-84a4-27867cc753d9/volumes"
Sep 08 15:25:21.403062 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:21.403028    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:21.504174 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:21.504142    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:26.776189 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:26.776155    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6l8mj"
Sep 08 15:25:26.865046 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:26.865010    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w2ngm"]
Sep 08 15:25:26.865289 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:26.865264    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w2ngm" podUID="7a093255-53a8-4f4f-8f1d-c4a6a0c7252c" containerName="manager" containerID="cri-o://54e37ebff45e7a2d2232c8d4da43bd37c1d98e62e3a1c5c2523f0f8709b25dc9" gracePeriod=10
Sep 08 15:25:27.104534 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:27.104510    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w2ngm"
Sep 08 15:25:27.197421 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:27.197387    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a093255-53a8-4f4f-8f1d-c4a6a0c7252c-extensions-socket-volume\") pod \"7a093255-53a8-4f4f-8f1d-c4a6a0c7252c\" (UID: \"7a093255-53a8-4f4f-8f1d-c4a6a0c7252c\") "
Sep 08 15:25:27.197604 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:27.197488    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfp4q\" (UniqueName: \"kubernetes.io/projected/7a093255-53a8-4f4f-8f1d-c4a6a0c7252c-kube-api-access-pfp4q\") pod \"7a093255-53a8-4f4f-8f1d-c4a6a0c7252c\" (UID: \"7a093255-53a8-4f4f-8f1d-c4a6a0c7252c\") "
Sep 08 15:25:27.197809 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:27.197787    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a093255-53a8-4f4f-8f1d-c4a6a0c7252c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7a093255-53a8-4f4f-8f1d-c4a6a0c7252c" (UID: "7a093255-53a8-4f4f-8f1d-c4a6a0c7252c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:25:27.199630 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:27.199604    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a093255-53a8-4f4f-8f1d-c4a6a0c7252c-kube-api-access-pfp4q" (OuterVolumeSpecName: "kube-api-access-pfp4q") pod "7a093255-53a8-4f4f-8f1d-c4a6a0c7252c" (UID: "7a093255-53a8-4f4f-8f1d-c4a6a0c7252c"). InnerVolumeSpecName "kube-api-access-pfp4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:25:27.298909 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:27.298823    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfp4q\" (UniqueName: \"kubernetes.io/projected/7a093255-53a8-4f4f-8f1d-c4a6a0c7252c-kube-api-access-pfp4q\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:25:27.298909 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:27.298858    2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a093255-53a8-4f4f-8f1d-c4a6a0c7252c-extensions-socket-volume\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:25:27.822305 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:27.822271    2560 generic.go:356] "Generic (PLEG): container finished" podID="7a093255-53a8-4f4f-8f1d-c4a6a0c7252c" containerID="54e37ebff45e7a2d2232c8d4da43bd37c1d98e62e3a1c5c2523f0f8709b25dc9" exitCode=0
Sep 08 15:25:27.822660 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:27.822328    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w2ngm"
Sep 08 15:25:27.822660 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:27.822348    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w2ngm" event={"ID":"7a093255-53a8-4f4f-8f1d-c4a6a0c7252c","Type":"ContainerDied","Data":"54e37ebff45e7a2d2232c8d4da43bd37c1d98e62e3a1c5c2523f0f8709b25dc9"}
Sep 08 15:25:27.822660 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:27.822423    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w2ngm" event={"ID":"7a093255-53a8-4f4f-8f1d-c4a6a0c7252c","Type":"ContainerDied","Data":"e334c352fecd6dc84be99bff545aa1ee9c59ca40dddee1be2051e81db574de29"}
Sep 08 15:25:27.822660 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:27.822455    2560 scope.go:117] "RemoveContainer" containerID="54e37ebff45e7a2d2232c8d4da43bd37c1d98e62e3a1c5c2523f0f8709b25dc9"
Sep 08 15:25:27.830850 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:27.830832    2560 scope.go:117] "RemoveContainer" containerID="54e37ebff45e7a2d2232c8d4da43bd37c1d98e62e3a1c5c2523f0f8709b25dc9"
Sep 08 15:25:27.831135 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:25:27.831116    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54e37ebff45e7a2d2232c8d4da43bd37c1d98e62e3a1c5c2523f0f8709b25dc9\": container with ID starting with 54e37ebff45e7a2d2232c8d4da43bd37c1d98e62e3a1c5c2523f0f8709b25dc9 not found: ID does not exist" containerID="54e37ebff45e7a2d2232c8d4da43bd37c1d98e62e3a1c5c2523f0f8709b25dc9"
Sep 08 15:25:27.831191 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:27.831143    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54e37ebff45e7a2d2232c8d4da43bd37c1d98e62e3a1c5c2523f0f8709b25dc9"} err="failed to get container status \"54e37ebff45e7a2d2232c8d4da43bd37c1d98e62e3a1c5c2523f0f8709b25dc9\": rpc error: code = NotFound desc = could not find container \"54e37ebff45e7a2d2232c8d4da43bd37c1d98e62e3a1c5c2523f0f8709b25dc9\": container with ID starting with 54e37ebff45e7a2d2232c8d4da43bd37c1d98e62e3a1c5c2523f0f8709b25dc9 not found: ID does not exist"
Sep 08 15:25:27.851110 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:27.851088    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w2ngm"]
Sep 08 15:25:27.856165 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:27.856146    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-w2ngm"]
Sep 08 15:25:28.179300 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:28.179273    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:28.279363 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:28.279329    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:29.799887 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:29.799854    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a093255-53a8-4f4f-8f1d-c4a6a0c7252c" path="/var/lib/kubelet/pods/7a093255-53a8-4f4f-8f1d-c4a6a0c7252c/volumes"
Sep 08 15:25:31.372607 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:31.372576    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:31.474948 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:31.474912    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:31.572888 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:31.572857    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:43.717537 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:43.717498    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:43.725489 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:43.725457    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-tpbf6"]
Sep 08 15:25:43.725834 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:43.725817    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a093255-53a8-4f4f-8f1d-c4a6a0c7252c" containerName="manager"
Sep 08 15:25:43.725911 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:43.725836    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a093255-53a8-4f4f-8f1d-c4a6a0c7252c" containerName="manager"
Sep 08 15:25:43.725911 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:43.725850    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b20a6fa8-8b3e-459f-84a4-27867cc753d9" containerName="manager"
Sep 08 15:25:43.725911 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:43.725859    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b20a6fa8-8b3e-459f-84a4-27867cc753d9" containerName="manager"
Sep 08 15:25:43.726059 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:43.725938    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b20a6fa8-8b3e-459f-84a4-27867cc753d9" containerName="manager"
Sep 08 15:25:43.726059 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:43.725950    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a093255-53a8-4f4f-8f1d-c4a6a0c7252c" containerName="manager"
Sep 08 15:25:43.728902 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:43.728884    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-tpbf6"
Sep 08 15:25:43.730888 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:43.730863    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2cn8b\""
Sep 08 15:25:43.730993 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:43.730867    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 08 15:25:43.738328 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:43.738308    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-tpbf6"]
Sep 08 15:25:43.819020 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:43.818984    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-tpbf6"]
Sep 08 15:25:43.820937 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:43.820908    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/81054fb4-a005-4cc2-b579-0bd2814d8b85-config-file\") pod \"limitador-limitador-859c4d96d6-tpbf6\" (UID: \"81054fb4-a005-4cc2-b579-0bd2814d8b85\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-tpbf6"
Sep 08 15:25:43.821101 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:43.820956    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g26cf\" (UniqueName: \"kubernetes.io/projected/81054fb4-a005-4cc2-b579-0bd2814d8b85-kube-api-access-g26cf\") pod \"limitador-limitador-859c4d96d6-tpbf6\" (UID: \"81054fb4-a005-4cc2-b579-0bd2814d8b85\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-tpbf6"
Sep 08 15:25:43.821645 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:43.821624    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:43.922392 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:43.922355    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/81054fb4-a005-4cc2-b579-0bd2814d8b85-config-file\") pod \"limitador-limitador-859c4d96d6-tpbf6\" (UID: \"81054fb4-a005-4cc2-b579-0bd2814d8b85\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-tpbf6"
Sep 08 15:25:43.922572 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:43.922405    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g26cf\" (UniqueName: \"kubernetes.io/projected/81054fb4-a005-4cc2-b579-0bd2814d8b85-kube-api-access-g26cf\") pod \"limitador-limitador-859c4d96d6-tpbf6\" (UID: \"81054fb4-a005-4cc2-b579-0bd2814d8b85\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-tpbf6"
Sep 08 15:25:43.922998 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:43.922975    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/81054fb4-a005-4cc2-b579-0bd2814d8b85-config-file\") pod \"limitador-limitador-859c4d96d6-tpbf6\" (UID: \"81054fb4-a005-4cc2-b579-0bd2814d8b85\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-tpbf6"
Sep 08 15:25:43.926628 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:43.926593    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:43.931274 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:43.931254    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g26cf\" (UniqueName: \"kubernetes.io/projected/81054fb4-a005-4cc2-b579-0bd2814d8b85-kube-api-access-g26cf\") pod \"limitador-limitador-859c4d96d6-tpbf6\" (UID: \"81054fb4-a005-4cc2-b579-0bd2814d8b85\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-tpbf6"
Sep 08 15:25:44.038710 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:44.038613    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-tpbf6"
Sep 08 15:25:44.309653 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:44.309626    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-tpbf6"]
Sep 08 15:25:44.312549 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:25:44.312519    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81054fb4_a005_4cc2_b579_0bd2814d8b85.slice/crio-dbfd15dfcc9a6beb9a7a90162bf0ab92613a9ebc44c2db6996361bae6b07a4e7 WatchSource:0}: Error finding container dbfd15dfcc9a6beb9a7a90162bf0ab92613a9ebc44c2db6996361bae6b07a4e7: Status 404 returned error can't find the container with id dbfd15dfcc9a6beb9a7a90162bf0ab92613a9ebc44c2db6996361bae6b07a4e7
Sep 08 15:25:44.594509 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:44.594423    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:44.696760 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:44.696729    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:44.896017 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:44.895933    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-tpbf6" event={"ID":"81054fb4-a005-4cc2-b579-0bd2814d8b85","Type":"ContainerStarted","Data":"dbfd15dfcc9a6beb9a7a90162bf0ab92613a9ebc44c2db6996361bae6b07a4e7"}
Sep 08 15:25:46.906720 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:46.906690    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-tpbf6" event={"ID":"81054fb4-a005-4cc2-b579-0bd2814d8b85","Type":"ContainerStarted","Data":"0ab64710a191a783290b0d37253606b9a98bb16e93187925c61b6ccb689d1d0c"}
Sep 08 15:25:46.907193 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:46.906793    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-tpbf6"
Sep 08 15:25:46.932628 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:46.932583    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-tpbf6" podStartSLOduration=2.11355934 podStartE2EDuration="3.932563069s" totalImagesPullingTime="1.819003729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:25:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:25:46.932175986 +0000 UTC m=+499.644118478" watchObservedRunningTime="2026-09-08 15:25:46.932563069 +0000 UTC m=+499.644505562"
Sep 08 15:25:52.119578 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:52.119531    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:52.295872 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:52.295811    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:55.086206 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:55.086171    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:57.513185 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:57.513149    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-tpbf6"]
Sep 08 15:25:57.513628 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:57.513340    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-tpbf6" podUID="81054fb4-a005-4cc2-b579-0bd2814d8b85" containerName="limitador" containerID="cri-o://0ab64710a191a783290b0d37253606b9a98bb16e93187925c61b6ccb689d1d0c" gracePeriod=30
Sep 08 15:25:57.514019 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:57.513949    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-tpbf6"
Sep 08 15:25:57.956226 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:57.956196    2560 generic.go:356] "Generic (PLEG): container finished" podID="81054fb4-a005-4cc2-b579-0bd2814d8b85" containerID="0ab64710a191a783290b0d37253606b9a98bb16e93187925c61b6ccb689d1d0c" exitCode=0
Sep 08 15:25:57.956399 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:57.956274    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-tpbf6" event={"ID":"81054fb4-a005-4cc2-b579-0bd2814d8b85","Type":"ContainerDied","Data":"0ab64710a191a783290b0d37253606b9a98bb16e93187925c61b6ccb689d1d0c"}
Sep 08 15:25:58.450734 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:58.450711    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-tpbf6"
Sep 08 15:25:58.553211 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:58.553179    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g26cf\" (UniqueName: \"kubernetes.io/projected/81054fb4-a005-4cc2-b579-0bd2814d8b85-kube-api-access-g26cf\") pod \"81054fb4-a005-4cc2-b579-0bd2814d8b85\" (UID: \"81054fb4-a005-4cc2-b579-0bd2814d8b85\") "
Sep 08 15:25:58.553554 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:58.553222    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/81054fb4-a005-4cc2-b579-0bd2814d8b85-config-file\") pod \"81054fb4-a005-4cc2-b579-0bd2814d8b85\" (UID: \"81054fb4-a005-4cc2-b579-0bd2814d8b85\") "
Sep 08 15:25:58.553598 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:58.553546    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81054fb4-a005-4cc2-b579-0bd2814d8b85-config-file" (OuterVolumeSpecName: "config-file") pod "81054fb4-a005-4cc2-b579-0bd2814d8b85" (UID: "81054fb4-a005-4cc2-b579-0bd2814d8b85"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:25:58.555276 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:58.555253    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81054fb4-a005-4cc2-b579-0bd2814d8b85-kube-api-access-g26cf" (OuterVolumeSpecName: "kube-api-access-g26cf") pod "81054fb4-a005-4cc2-b579-0bd2814d8b85" (UID: "81054fb4-a005-4cc2-b579-0bd2814d8b85"). InnerVolumeSpecName "kube-api-access-g26cf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:25:58.654300 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:58.654216    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g26cf\" (UniqueName: \"kubernetes.io/projected/81054fb4-a005-4cc2-b579-0bd2814d8b85-kube-api-access-g26cf\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:25:58.654300 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:58.654245    2560 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/81054fb4-a005-4cc2-b579-0bd2814d8b85-config-file\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:25:58.961673 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:58.961640    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-tpbf6" event={"ID":"81054fb4-a005-4cc2-b579-0bd2814d8b85","Type":"ContainerDied","Data":"dbfd15dfcc9a6beb9a7a90162bf0ab92613a9ebc44c2db6996361bae6b07a4e7"}
Sep 08 15:25:58.961673 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:58.961673    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-tpbf6"
Sep 08 15:25:58.961909 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:58.961712    2560 scope.go:117] "RemoveContainer" containerID="0ab64710a191a783290b0d37253606b9a98bb16e93187925c61b6ccb689d1d0c"
Sep 08 15:25:58.984720 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:58.984698    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-tpbf6"]
Sep 08 15:25:58.988733 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:58.988712    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-tpbf6"]
Sep 08 15:25:58.999812 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:25:58.999791    2560 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81054fb4_a005_4cc2_b579_0bd2814d8b85.slice\": RecentStats: unable to find data in memory cache]"
Sep 08 15:25:59.799822 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:25:59.799782    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81054fb4-a005-4cc2-b579-0bd2814d8b85" path="/var/lib/kubelet/pods/81054fb4-a005-4cc2-b579-0bd2814d8b85/volumes"
Sep 08 15:26:03.242303 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:03.242270    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-97xj8"]
Sep 08 15:26:03.242648 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:03.242579    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81054fb4-a005-4cc2-b579-0bd2814d8b85" containerName="limitador"
Sep 08 15:26:03.242648 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:03.242591    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="81054fb4-a005-4cc2-b579-0bd2814d8b85" containerName="limitador"
Sep 08 15:26:03.242648 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:03.242649    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="81054fb4-a005-4cc2-b579-0bd2814d8b85" containerName="limitador"
Sep 08 15:26:03.245224 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:03.245205    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-97xj8"
Sep 08 15:26:03.247953 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:03.247925    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 08 15:26:03.248116 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:03.248098    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-spd5v\""
Sep 08 15:26:03.248539 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:03.248524    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 08 15:26:03.248752 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:03.248738    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 15:26:03.265422 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:03.265390    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-97xj8"]
Sep 08 15:26:03.295548 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:03.295515    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/9d29db75-3ab3-496e-8d91-ee03ec995deb-data\") pod \"postgres-69fb4594cf-97xj8\" (UID: \"9d29db75-3ab3-496e-8d91-ee03ec995deb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-97xj8"
Sep 08 15:26:03.295683 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:03.295559    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6lsq\" (UniqueName: \"kubernetes.io/projected/9d29db75-3ab3-496e-8d91-ee03ec995deb-kube-api-access-w6lsq\") pod \"postgres-69fb4594cf-97xj8\" (UID: \"9d29db75-3ab3-496e-8d91-ee03ec995deb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-97xj8"
Sep 08 15:26:03.396237 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:03.396201    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/9d29db75-3ab3-496e-8d91-ee03ec995deb-data\") pod \"postgres-69fb4594cf-97xj8\" (UID: \"9d29db75-3ab3-496e-8d91-ee03ec995deb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-97xj8"
Sep 08 15:26:03.396237 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:03.396240    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6lsq\" (UniqueName: \"kubernetes.io/projected/9d29db75-3ab3-496e-8d91-ee03ec995deb-kube-api-access-w6lsq\") pod \"postgres-69fb4594cf-97xj8\" (UID: \"9d29db75-3ab3-496e-8d91-ee03ec995deb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-97xj8"
Sep 08 15:26:03.396651 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:03.396631    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/9d29db75-3ab3-496e-8d91-ee03ec995deb-data\") pod \"postgres-69fb4594cf-97xj8\" (UID: \"9d29db75-3ab3-496e-8d91-ee03ec995deb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-97xj8"
Sep 08 15:26:03.404592 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:03.404571    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6lsq\" (UniqueName: \"kubernetes.io/projected/9d29db75-3ab3-496e-8d91-ee03ec995deb-kube-api-access-w6lsq\") pod \"postgres-69fb4594cf-97xj8\" (UID: \"9d29db75-3ab3-496e-8d91-ee03ec995deb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-97xj8"
Sep 08 15:26:03.555425 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:03.555346    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-97xj8"
Sep 08 15:26:03.721232 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:03.721192    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:03.824859 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:03.824778    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:03.832354 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:03.832327    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-97xj8"]
Sep 08 15:26:03.835958 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:26:03.835928    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d29db75_3ab3_496e_8d91_ee03ec995deb.slice/crio-09eaa627ed77913688d4678f65bbfdc9dfda124ab2b03841863b9d0a1b6cdbc4 WatchSource:0}: Error finding container 09eaa627ed77913688d4678f65bbfdc9dfda124ab2b03841863b9d0a1b6cdbc4: Status 404 returned error can't find the container with id 09eaa627ed77913688d4678f65bbfdc9dfda124ab2b03841863b9d0a1b6cdbc4
Sep 08 15:26:03.984661 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:03.984625    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-97xj8" event={"ID":"9d29db75-3ab3-496e-8d91-ee03ec995deb","Type":"ContainerStarted","Data":"09eaa627ed77913688d4678f65bbfdc9dfda124ab2b03841863b9d0a1b6cdbc4"}
Sep 08 15:26:10.016353 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:10.016317    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-97xj8" event={"ID":"9d29db75-3ab3-496e-8d91-ee03ec995deb","Type":"ContainerStarted","Data":"450c6407df784e7170f12de24e0b1b7e3d3be664369d0911e9d645c835868181"}
Sep 08 15:26:10.016739 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:10.016443    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-97xj8"
Sep 08 15:26:10.035264 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:10.035213    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-97xj8" podStartSLOduration=1.612346013 podStartE2EDuration="7.035197872s" totalImagesPullingTime="5.422851859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:26:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:26:10.033093326 +0000 UTC m=+522.745035810" watchObservedRunningTime="2026-09-08 15:26:10.035197872 +0000 UTC m=+522.747140359"
Sep 08 15:26:16.049263 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:16.049233    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-97xj8"
Sep 08 15:26:16.375151 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:16.375060    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:16.645667 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:16.645580    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:23.861096 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:23.860996    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-757f9dc676-lcklb"]
Sep 08 15:26:23.864118 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:23.864097    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-757f9dc676-lcklb"
Sep 08 15:26:23.868551 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:23.868528    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 08 15:26:23.868674 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:23.868618    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 08 15:26:23.869342 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:23.869320    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wpgkm\""
Sep 08 15:26:23.869460 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:23.869420    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 08 15:26:23.883216 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:23.883189    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-757f9dc676-lcklb"]
Sep 08 15:26:23.980604 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:23.980569    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b6bcc44-80fb-477e-a9f9-8acfeb987a36-metrics-certs\") pod \"maas-controller-757f9dc676-lcklb\" (UID: \"1b6bcc44-80fb-477e-a9f9-8acfeb987a36\") " pod="opendatahub/maas-controller-757f9dc676-lcklb"
Sep 08 15:26:23.980604 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:23.980605    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m979z\" (UniqueName: \"kubernetes.io/projected/1b6bcc44-80fb-477e-a9f9-8acfeb987a36-kube-api-access-m979z\") pod \"maas-controller-757f9dc676-lcklb\" (UID: \"1b6bcc44-80fb-477e-a9f9-8acfeb987a36\") " pod="opendatahub/maas-controller-757f9dc676-lcklb"
Sep 08 15:26:23.980795 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:23.980628    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b6bcc44-80fb-477e-a9f9-8acfeb987a36-webhook-cert\") pod \"maas-controller-757f9dc676-lcklb\" (UID: \"1b6bcc44-80fb-477e-a9f9-8acfeb987a36\") " pod="opendatahub/maas-controller-757f9dc676-lcklb"
Sep 08 15:26:24.053787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:24.053757    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7b994b57d8-tgtp7"]
Sep 08 15:26:24.056408 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:24.056392    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b994b57d8-tgtp7"
Sep 08 15:26:24.074170 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:24.074143    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b994b57d8-tgtp7"]
Sep 08 15:26:24.081018 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:24.080991    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b6bcc44-80fb-477e-a9f9-8acfeb987a36-metrics-certs\") pod \"maas-controller-757f9dc676-lcklb\" (UID: \"1b6bcc44-80fb-477e-a9f9-8acfeb987a36\") " pod="opendatahub/maas-controller-757f9dc676-lcklb"
Sep 08 15:26:24.081163 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:24.081027    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m979z\" (UniqueName: \"kubernetes.io/projected/1b6bcc44-80fb-477e-a9f9-8acfeb987a36-kube-api-access-m979z\") pod \"maas-controller-757f9dc676-lcklb\" (UID: \"1b6bcc44-80fb-477e-a9f9-8acfeb987a36\") " pod="opendatahub/maas-controller-757f9dc676-lcklb"
Sep 08 15:26:24.081163 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:24.081053    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b6bcc44-80fb-477e-a9f9-8acfeb987a36-webhook-cert\") pod \"maas-controller-757f9dc676-lcklb\" (UID: \"1b6bcc44-80fb-477e-a9f9-8acfeb987a36\") " pod="opendatahub/maas-controller-757f9dc676-lcklb"
Sep 08 15:26:24.083365 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:24.083344    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b6bcc44-80fb-477e-a9f9-8acfeb987a36-webhook-cert\") pod \"maas-controller-757f9dc676-lcklb\" (UID: \"1b6bcc44-80fb-477e-a9f9-8acfeb987a36\") " pod="opendatahub/maas-controller-757f9dc676-lcklb"
Sep 08 15:26:24.083460 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:24.083379    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b6bcc44-80fb-477e-a9f9-8acfeb987a36-metrics-certs\") pod \"maas-controller-757f9dc676-lcklb\" (UID: \"1b6bcc44-80fb-477e-a9f9-8acfeb987a36\") " pod="opendatahub/maas-controller-757f9dc676-lcklb"
Sep 08 15:26:24.093116 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:24.093092    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m979z\" (UniqueName: \"kubernetes.io/projected/1b6bcc44-80fb-477e-a9f9-8acfeb987a36-kube-api-access-m979z\") pod \"maas-controller-757f9dc676-lcklb\" (UID: \"1b6bcc44-80fb-477e-a9f9-8acfeb987a36\") " pod="opendatahub/maas-controller-757f9dc676-lcklb"
Sep 08 15:26:24.095401 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:24.095374    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:24.175299 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:24.175272    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-757f9dc676-lcklb"
Sep 08 15:26:24.182021 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:24.181998    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25db5961-2fd9-4e78-9397-ea9f5f2a9626-metrics-certs\") pod \"maas-controller-7b994b57d8-tgtp7\" (UID: \"25db5961-2fd9-4e78-9397-ea9f5f2a9626\") " pod="opendatahub/maas-controller-7b994b57d8-tgtp7"
Sep 08 15:26:24.182106 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:24.182050    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfrbv\" (UniqueName: \"kubernetes.io/projected/25db5961-2fd9-4e78-9397-ea9f5f2a9626-kube-api-access-nfrbv\") pod \"maas-controller-7b994b57d8-tgtp7\" (UID: \"25db5961-2fd9-4e78-9397-ea9f5f2a9626\") " pod="opendatahub/maas-controller-7b994b57d8-tgtp7"
Sep 08 15:26:24.182159 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:24.182116    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/25db5961-2fd9-4e78-9397-ea9f5f2a9626-webhook-cert\") pod \"maas-controller-7b994b57d8-tgtp7\" (UID: \"25db5961-2fd9-4e78-9397-ea9f5f2a9626\") " pod="opendatahub/maas-controller-7b994b57d8-tgtp7"
Sep 08 15:26:24.195743 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:24.195719    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:24.282733 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:24.282699    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25db5961-2fd9-4e78-9397-ea9f5f2a9626-metrics-certs\") pod \"maas-controller-7b994b57d8-tgtp7\" (UID: \"25db5961-2fd9-4e78-9397-ea9f5f2a9626\") " pod="opendatahub/maas-controller-7b994b57d8-tgtp7"
Sep 08 15:26:24.282880 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:24.282758    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfrbv\" (UniqueName: \"kubernetes.io/projected/25db5961-2fd9-4e78-9397-ea9f5f2a9626-kube-api-access-nfrbv\") pod \"maas-controller-7b994b57d8-tgtp7\" (UID: \"25db5961-2fd9-4e78-9397-ea9f5f2a9626\") " pod="opendatahub/maas-controller-7b994b57d8-tgtp7"
Sep 08 15:26:24.282880 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:24.282797    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/25db5961-2fd9-4e78-9397-ea9f5f2a9626-webhook-cert\") pod \"maas-controller-7b994b57d8-tgtp7\" (UID: \"25db5961-2fd9-4e78-9397-ea9f5f2a9626\") " pod="opendatahub/maas-controller-7b994b57d8-tgtp7"
Sep 08 15:26:24.285129 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:24.285105    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25db5961-2fd9-4e78-9397-ea9f5f2a9626-metrics-certs\") pod \"maas-controller-7b994b57d8-tgtp7\" (UID: \"25db5961-2fd9-4e78-9397-ea9f5f2a9626\") " pod="opendatahub/maas-controller-7b994b57d8-tgtp7"
Sep 08 15:26:24.285244 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:24.285144    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/25db5961-2fd9-4e78-9397-ea9f5f2a9626-webhook-cert\") pod \"maas-controller-7b994b57d8-tgtp7\" (UID: \"25db5961-2fd9-4e78-9397-ea9f5f2a9626\") " pod="opendatahub/maas-controller-7b994b57d8-tgtp7"
Sep 08 15:26:24.291503 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:24.291479    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfrbv\" (UniqueName: \"kubernetes.io/projected/25db5961-2fd9-4e78-9397-ea9f5f2a9626-kube-api-access-nfrbv\") pod \"maas-controller-7b994b57d8-tgtp7\" (UID: \"25db5961-2fd9-4e78-9397-ea9f5f2a9626\") " pod="opendatahub/maas-controller-7b994b57d8-tgtp7"
Sep 08 15:26:24.366123 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:24.366054    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b994b57d8-tgtp7"
Sep 08 15:26:24.449537 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:24.449507    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-757f9dc676-lcklb"]
Sep 08 15:26:24.452061 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:26:24.452035    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b6bcc44_80fb_477e_a9f9_8acfeb987a36.slice/crio-fd4d4bf72b3c1c15949ad23680b573071693d6d097ed67e57b9450e4693063ab WatchSource:0}: Error finding container fd4d4bf72b3c1c15949ad23680b573071693d6d097ed67e57b9450e4693063ab: Status 404 returned error can't find the container with id fd4d4bf72b3c1c15949ad23680b573071693d6d097ed67e57b9450e4693063ab
Sep 08 15:26:24.640720 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:24.640696    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b994b57d8-tgtp7"]
Sep 08 15:26:24.642999 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:26:24.642972    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25db5961_2fd9_4e78_9397_ea9f5f2a9626.slice/crio-70575cfb706a810f47cc3f98374f08e4812962e9681a58a795e67f38e4c238fe WatchSource:0}: Error finding container 70575cfb706a810f47cc3f98374f08e4812962e9681a58a795e67f38e4c238fe: Status 404 returned error can't find the container with id 70575cfb706a810f47cc3f98374f08e4812962e9681a58a795e67f38e4c238fe
Sep 08 15:26:25.087224 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:25.087190    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-757f9dc676-lcklb" event={"ID":"1b6bcc44-80fb-477e-a9f9-8acfeb987a36","Type":"ContainerStarted","Data":"fd4d4bf72b3c1c15949ad23680b573071693d6d097ed67e57b9450e4693063ab"}
Sep 08 15:26:25.088508 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:25.088479    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b994b57d8-tgtp7" event={"ID":"25db5961-2fd9-4e78-9397-ea9f5f2a9626","Type":"ContainerStarted","Data":"70575cfb706a810f47cc3f98374f08e4812962e9681a58a795e67f38e4c238fe"}
Sep 08 15:26:26.929032 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:26.928997    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:27.102625 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:27.102526    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-757f9dc676-lcklb" event={"ID":"1b6bcc44-80fb-477e-a9f9-8acfeb987a36","Type":"ContainerStarted","Data":"0012ffb606bc24644b75530c9d2c45d1250c35d59aa1c42d4799b4a994072867"}
Sep 08 15:26:27.102792 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:27.102716    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-757f9dc676-lcklb"
Sep 08 15:26:27.103459 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:27.103418    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:27.104531 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:27.104509    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b994b57d8-tgtp7" event={"ID":"25db5961-2fd9-4e78-9397-ea9f5f2a9626","Type":"ContainerStarted","Data":"26a0d89a9e044617a4741d06e1e26d63d0105d7a590fadb9235dc05ba70a89db"}
Sep 08 15:26:27.104624 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:27.104561    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7b994b57d8-tgtp7"
Sep 08 15:26:27.132093 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:27.132022    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-757f9dc676-lcklb" podStartSLOduration=1.82216555 podStartE2EDuration="4.132005257s" totalImagesPullingTime="2.309839707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:26:27.13200216 +0000 UTC m=+539.843944656" watchObservedRunningTime="2026-09-08 15:26:27.132005257 +0000 UTC m=+539.843947748"
Sep 08 15:26:27.167085 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:27.167030    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7b994b57d8-tgtp7" podStartSLOduration=1.047527673 podStartE2EDuration="3.167014437s" totalImagesPullingTime="2.119486764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:26:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:26:27.163159907 +0000 UTC m=+539.875102399" watchObservedRunningTime="2026-09-08 15:26:27.167014437 +0000 UTC m=+539.878956927"
Sep 08 15:26:38.111911 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:38.111876    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7b994b57d8-tgtp7"
Sep 08 15:26:38.112434 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:38.112054    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-757f9dc676-lcklb"
Sep 08 15:26:38.175385 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:38.175352    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-757f9dc676-lcklb"]
Sep 08 15:26:38.175573 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:38.175547    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-757f9dc676-lcklb" podUID="1b6bcc44-80fb-477e-a9f9-8acfeb987a36" containerName="manager" containerID="cri-o://0012ffb606bc24644b75530c9d2c45d1250c35d59aa1c42d4799b4a994072867" gracePeriod=30
Sep 08 15:26:38.419053 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:38.419027    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-757f9dc676-lcklb"
Sep 08 15:26:38.500293 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:38.500260    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b6bcc44-80fb-477e-a9f9-8acfeb987a36-metrics-certs\") pod \"1b6bcc44-80fb-477e-a9f9-8acfeb987a36\" (UID: \"1b6bcc44-80fb-477e-a9f9-8acfeb987a36\") "
Sep 08 15:26:38.500453 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:38.500307    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b6bcc44-80fb-477e-a9f9-8acfeb987a36-webhook-cert\") pod \"1b6bcc44-80fb-477e-a9f9-8acfeb987a36\" (UID: \"1b6bcc44-80fb-477e-a9f9-8acfeb987a36\") "
Sep 08 15:26:38.500453 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:38.500344    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m979z\" (UniqueName: \"kubernetes.io/projected/1b6bcc44-80fb-477e-a9f9-8acfeb987a36-kube-api-access-m979z\") pod \"1b6bcc44-80fb-477e-a9f9-8acfeb987a36\" (UID: \"1b6bcc44-80fb-477e-a9f9-8acfeb987a36\") "
Sep 08 15:26:38.502543 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:38.502501    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b6bcc44-80fb-477e-a9f9-8acfeb987a36-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "1b6bcc44-80fb-477e-a9f9-8acfeb987a36" (UID: "1b6bcc44-80fb-477e-a9f9-8acfeb987a36"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:26:38.502650 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:38.502582    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b6bcc44-80fb-477e-a9f9-8acfeb987a36-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1b6bcc44-80fb-477e-a9f9-8acfeb987a36" (UID: "1b6bcc44-80fb-477e-a9f9-8acfeb987a36"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:26:38.502650 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:38.502596    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b6bcc44-80fb-477e-a9f9-8acfeb987a36-kube-api-access-m979z" (OuterVolumeSpecName: "kube-api-access-m979z") pod "1b6bcc44-80fb-477e-a9f9-8acfeb987a36" (UID: "1b6bcc44-80fb-477e-a9f9-8acfeb987a36"). InnerVolumeSpecName "kube-api-access-m979z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:26:38.601746 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:38.601714    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b6bcc44-80fb-477e-a9f9-8acfeb987a36-metrics-certs\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:26:38.601746 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:38.601739    2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b6bcc44-80fb-477e-a9f9-8acfeb987a36-webhook-cert\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:26:38.601746 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:38.601750    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m979z\" (UniqueName: \"kubernetes.io/projected/1b6bcc44-80fb-477e-a9f9-8acfeb987a36-kube-api-access-m979z\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:26:39.158821 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:39.158785    2560 generic.go:356] "Generic (PLEG): container finished" podID="1b6bcc44-80fb-477e-a9f9-8acfeb987a36" containerID="0012ffb606bc24644b75530c9d2c45d1250c35d59aa1c42d4799b4a994072867" exitCode=0
Sep 08 15:26:39.158821 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:39.158824    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-757f9dc676-lcklb"
Sep 08 15:26:39.159310 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:39.158870    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-757f9dc676-lcklb" event={"ID":"1b6bcc44-80fb-477e-a9f9-8acfeb987a36","Type":"ContainerDied","Data":"0012ffb606bc24644b75530c9d2c45d1250c35d59aa1c42d4799b4a994072867"}
Sep 08 15:26:39.159310 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:39.158945    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-757f9dc676-lcklb" event={"ID":"1b6bcc44-80fb-477e-a9f9-8acfeb987a36","Type":"ContainerDied","Data":"fd4d4bf72b3c1c15949ad23680b573071693d6d097ed67e57b9450e4693063ab"}
Sep 08 15:26:39.159310 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:39.158969    2560 scope.go:117] "RemoveContainer" containerID="0012ffb606bc24644b75530c9d2c45d1250c35d59aa1c42d4799b4a994072867"
Sep 08 15:26:39.167594 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:39.167575    2560 scope.go:117] "RemoveContainer" containerID="0012ffb606bc24644b75530c9d2c45d1250c35d59aa1c42d4799b4a994072867"
Sep 08 15:26:39.167855 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:26:39.167830    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0012ffb606bc24644b75530c9d2c45d1250c35d59aa1c42d4799b4a994072867\": container with ID starting with 0012ffb606bc24644b75530c9d2c45d1250c35d59aa1c42d4799b4a994072867 not found: ID does not exist" containerID="0012ffb606bc24644b75530c9d2c45d1250c35d59aa1c42d4799b4a994072867"
Sep 08 15:26:39.167919 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:39.167862    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0012ffb606bc24644b75530c9d2c45d1250c35d59aa1c42d4799b4a994072867"} err="failed to get container status \"0012ffb606bc24644b75530c9d2c45d1250c35d59aa1c42d4799b4a994072867\": rpc error: code = NotFound desc = could not find container \"0012ffb606bc24644b75530c9d2c45d1250c35d59aa1c42d4799b4a994072867\": container with ID starting with 0012ffb606bc24644b75530c9d2c45d1250c35d59aa1c42d4799b4a994072867 not found: ID does not exist"
Sep 08 15:26:39.178731 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:39.178706    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-757f9dc676-lcklb"]
Sep 08 15:26:39.184922 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:39.184901    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-757f9dc676-lcklb"]
Sep 08 15:26:39.801931 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:39.801899    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b6bcc44-80fb-477e-a9f9-8acfeb987a36" path="/var/lib/kubelet/pods/1b6bcc44-80fb-477e-a9f9-8acfeb987a36/volumes"
Sep 08 15:26:57.738862 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:57.738824    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:57.840932 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:57.840894    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:59.237454 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.237420    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:59.337648 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.337615    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:59.376541 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.376513    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn"]
Sep 08 15:26:59.376904 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.376887    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b6bcc44-80fb-477e-a9f9-8acfeb987a36" containerName="manager"
Sep 08 15:26:59.376980 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.376907    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b6bcc44-80fb-477e-a9f9-8acfeb987a36" containerName="manager"
Sep 08 15:26:59.377042 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.376999    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b6bcc44-80fb-477e-a9f9-8acfeb987a36" containerName="manager"
Sep 08 15:26:59.381347 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.381314    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn"
Sep 08 15:26:59.384717 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.384691    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 08 15:26:59.384842 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.384714    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 08 15:26:59.384842 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.384771    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-2xlj4\""
Sep 08 15:26:59.390498 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.390474    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn"]
Sep 08 15:26:59.446422 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.446381    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:59.479469 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.479431    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8fce02c-f8ae-43eb-a5d6-14c107eab70e-maas-api-tls\") pod \"maas-api-7f8554757b-fgpwn\" (UID: \"a8fce02c-f8ae-43eb-a5d6-14c107eab70e\") " pod="odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn"
Sep 08 15:26:59.479611 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.479481    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcmgd\" (UniqueName: \"kubernetes.io/projected/a8fce02c-f8ae-43eb-a5d6-14c107eab70e-kube-api-access-qcmgd\") pod \"maas-api-7f8554757b-fgpwn\" (UID: \"a8fce02c-f8ae-43eb-a5d6-14c107eab70e\") " pod="odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn"
Sep 08 15:26:59.479611 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.479522    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8fce02c-f8ae-43eb-a5d6-14c107eab70e-openshift-service-ca\") pod \"maas-api-7f8554757b-fgpwn\" (UID: \"a8fce02c-f8ae-43eb-a5d6-14c107eab70e\") " pod="odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn"
Sep 08 15:26:59.479611 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.479537    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8fce02c-f8ae-43eb-a5d6-14c107eab70e-metrics-certs\") pod \"maas-api-7f8554757b-fgpwn\" (UID: \"a8fce02c-f8ae-43eb-a5d6-14c107eab70e\") " pod="odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn"
Sep 08 15:26:59.580340 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.580261    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcmgd\" (UniqueName: \"kubernetes.io/projected/a8fce02c-f8ae-43eb-a5d6-14c107eab70e-kube-api-access-qcmgd\") pod \"maas-api-7f8554757b-fgpwn\" (UID: \"a8fce02c-f8ae-43eb-a5d6-14c107eab70e\") " pod="odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn"
Sep 08 15:26:59.580340 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.580304    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8fce02c-f8ae-43eb-a5d6-14c107eab70e-openshift-service-ca\") pod \"maas-api-7f8554757b-fgpwn\" (UID: \"a8fce02c-f8ae-43eb-a5d6-14c107eab70e\") " pod="odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn"
Sep 08 15:26:59.580511 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.580348    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8fce02c-f8ae-43eb-a5d6-14c107eab70e-metrics-certs\") pod \"maas-api-7f8554757b-fgpwn\" (UID: \"a8fce02c-f8ae-43eb-a5d6-14c107eab70e\") " pod="odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn"
Sep 08 15:26:59.580511 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.580449    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8fce02c-f8ae-43eb-a5d6-14c107eab70e-maas-api-tls\") pod \"maas-api-7f8554757b-fgpwn\" (UID: \"a8fce02c-f8ae-43eb-a5d6-14c107eab70e\") " pod="odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn"
Sep 08 15:26:59.581209 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.581183    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8fce02c-f8ae-43eb-a5d6-14c107eab70e-openshift-service-ca\") pod \"maas-api-7f8554757b-fgpwn\" (UID: \"a8fce02c-f8ae-43eb-a5d6-14c107eab70e\") " pod="odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn"
Sep 08 15:26:59.582896 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.582872    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8fce02c-f8ae-43eb-a5d6-14c107eab70e-maas-api-tls\") pod \"maas-api-7f8554757b-fgpwn\" (UID: \"a8fce02c-f8ae-43eb-a5d6-14c107eab70e\") " pod="odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn"
Sep 08 15:26:59.583045 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.583026    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8fce02c-f8ae-43eb-a5d6-14c107eab70e-metrics-certs\") pod \"maas-api-7f8554757b-fgpwn\" (UID: \"a8fce02c-f8ae-43eb-a5d6-14c107eab70e\") " pod="odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn"
Sep 08 15:26:59.594284 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.594258    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcmgd\" (UniqueName: \"kubernetes.io/projected/a8fce02c-f8ae-43eb-a5d6-14c107eab70e-kube-api-access-qcmgd\") pod \"maas-api-7f8554757b-fgpwn\" (UID: \"a8fce02c-f8ae-43eb-a5d6-14c107eab70e\") " pod="odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn"
Sep 08 15:26:59.614392 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.614351    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:59.693120 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.693065    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn"
Sep 08 15:26:59.932118 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.932081    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-tscfp"]
Sep 08 15:26:59.936433 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.936411    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-tscfp"
Sep 08 15:26:59.939487 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.939465    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 08 15:26:59.939612 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.939588    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-zdsqr"]
Sep 08 15:26:59.942853 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.942835    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-zdsqr"
Sep 08 15:26:59.944879 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.944859    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-hj5pl\""
Sep 08 15:26:59.949335 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.949316    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-tscfp"]
Sep 08 15:26:59.956861 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.956833    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-zdsqr"]
Sep 08 15:26:59.983530 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:26:59.983503    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn"]
Sep 08 15:26:59.986711 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:26:59.986688    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8fce02c_f8ae_43eb_a5d6_14c107eab70e.slice/crio-24521d833d5911840bbb4c61fc44160b9ce0557ae362877bb19989cf18fb1ed9 WatchSource:0}: Error finding container 24521d833d5911840bbb4c61fc44160b9ce0557ae362877bb19989cf18fb1ed9: Status 404 returned error can't find the container with id 24521d833d5911840bbb4c61fc44160b9ce0557ae362877bb19989cf18fb1ed9
Sep 08 15:27:00.084444 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:00.084399    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77f92f27-262b-498d-a1fa-94ece7554033-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-zdsqr\" (UID: \"77f92f27-262b-498d-a1fa-94ece7554033\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-zdsqr"
Sep 08 15:27:00.084656 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:00.084451    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqx9k\" (UniqueName: \"kubernetes.io/projected/231e053b-1392-4597-bc81-2437855e9687-kube-api-access-kqx9k\") pod \"payload-pre-processing-59fbc47bf8-tscfp\" (UID: \"231e053b-1392-4597-bc81-2437855e9687\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-tscfp"
Sep 08 15:27:00.084656 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:00.084545    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/231e053b-1392-4597-bc81-2437855e9687-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-tscfp\" (UID: \"231e053b-1392-4597-bc81-2437855e9687\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-tscfp"
Sep 08 15:27:00.084656 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:00.084613    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbkbz\" (UniqueName: \"kubernetes.io/projected/77f92f27-262b-498d-a1fa-94ece7554033-kube-api-access-dbkbz\") pod \"payload-processing-5b54b7fbb7-zdsqr\" (UID: \"77f92f27-262b-498d-a1fa-94ece7554033\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-zdsqr"
Sep 08 15:27:00.185399 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:00.185317    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/231e053b-1392-4597-bc81-2437855e9687-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-tscfp\" (UID: \"231e053b-1392-4597-bc81-2437855e9687\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-tscfp"
Sep 08 15:27:00.185399 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:00.185370    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbkbz\" (UniqueName: \"kubernetes.io/projected/77f92f27-262b-498d-a1fa-94ece7554033-kube-api-access-dbkbz\") pod \"payload-processing-5b54b7fbb7-zdsqr\" (UID: \"77f92f27-262b-498d-a1fa-94ece7554033\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-zdsqr"
Sep 08 15:27:00.185591 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:00.185425    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77f92f27-262b-498d-a1fa-94ece7554033-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-zdsqr\" (UID: \"77f92f27-262b-498d-a1fa-94ece7554033\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-zdsqr"
Sep 08 15:27:00.185629 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:00.185599    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqx9k\" (UniqueName: \"kubernetes.io/projected/231e053b-1392-4597-bc81-2437855e9687-kube-api-access-kqx9k\") pod \"payload-pre-processing-59fbc47bf8-tscfp\" (UID: \"231e053b-1392-4597-bc81-2437855e9687\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-tscfp"
Sep 08 15:27:00.186028 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:00.186008    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/231e053b-1392-4597-bc81-2437855e9687-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-tscfp\" (UID: \"231e053b-1392-4597-bc81-2437855e9687\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-tscfp"
Sep 08 15:27:00.186229 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:00.186210    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77f92f27-262b-498d-a1fa-94ece7554033-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-zdsqr\" (UID: \"77f92f27-262b-498d-a1fa-94ece7554033\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-zdsqr"
Sep 08 15:27:00.193272 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:00.193238    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbkbz\" (UniqueName: \"kubernetes.io/projected/77f92f27-262b-498d-a1fa-94ece7554033-kube-api-access-dbkbz\") pod \"payload-processing-5b54b7fbb7-zdsqr\" (UID: \"77f92f27-262b-498d-a1fa-94ece7554033\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-zdsqr"
Sep 08 15:27:00.193407 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:00.193382    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqx9k\" (UniqueName: \"kubernetes.io/projected/231e053b-1392-4597-bc81-2437855e9687-kube-api-access-kqx9k\") pod \"payload-pre-processing-59fbc47bf8-tscfp\" (UID: \"231e053b-1392-4597-bc81-2437855e9687\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-tscfp"
Sep 08 15:27:00.237751 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:00.237714    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:00.247338 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:00.247310    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-tscfp"
Sep 08 15:27:00.252794 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:00.252771    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn" event={"ID":"a8fce02c-f8ae-43eb-a5d6-14c107eab70e","Type":"ContainerStarted","Data":"24521d833d5911840bbb4c61fc44160b9ce0557ae362877bb19989cf18fb1ed9"}
Sep 08 15:27:00.257977 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:00.257956    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-zdsqr"
Sep 08 15:27:00.338378 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:00.338350    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:00.529837 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:00.529757    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-tscfp"]
Sep 08 15:27:00.533012 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:27:00.532978    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod231e053b_1392_4597_bc81_2437855e9687.slice/crio-9f25d5dc2afef19b05be1cc2fb200739005e696b13ae37e760c2d364594ba0bd WatchSource:0}: Error finding container 9f25d5dc2afef19b05be1cc2fb200739005e696b13ae37e760c2d364594ba0bd: Status 404 returned error can't find the container with id 9f25d5dc2afef19b05be1cc2fb200739005e696b13ae37e760c2d364594ba0bd
Sep 08 15:27:00.545552 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:00.545537    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-zdsqr"]
Sep 08 15:27:00.547659 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:27:00.547632    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77f92f27_262b_498d_a1fa_94ece7554033.slice/crio-bc5b951d834484403adb2ef54fc695ee9ad621744c37fd9e6b4390a749a1e012 WatchSource:0}: Error finding container bc5b951d834484403adb2ef54fc695ee9ad621744c37fd9e6b4390a749a1e012: Status 404 returned error can't find the container with id bc5b951d834484403adb2ef54fc695ee9ad621744c37fd9e6b4390a749a1e012
Sep 08 15:27:01.265926 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:01.265882    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-tscfp" event={"ID":"231e053b-1392-4597-bc81-2437855e9687","Type":"ContainerStarted","Data":"9f25d5dc2afef19b05be1cc2fb200739005e696b13ae37e760c2d364594ba0bd"}
Sep 08 15:27:01.267688 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:01.267566    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-zdsqr" event={"ID":"77f92f27-262b-498d-a1fa-94ece7554033","Type":"ContainerStarted","Data":"bc5b951d834484403adb2ef54fc695ee9ad621744c37fd9e6b4390a749a1e012"}
Sep 08 15:27:04.284635 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:04.284592    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-tscfp" event={"ID":"231e053b-1392-4597-bc81-2437855e9687","Type":"ContainerStarted","Data":"0368b12b9bce04fc1192c14b8d66268c288bac86d10d107b7dc2727187577f3a"}
Sep 08 15:27:04.285143 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:04.284724    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-tscfp"
Sep 08 15:27:04.286321 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:04.286296    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-zdsqr" event={"ID":"77f92f27-262b-498d-a1fa-94ece7554033","Type":"ContainerStarted","Data":"e2168fa775dcd37602c18790ee2e1975757ecba236b176932851d12519a9ecf7"}
Sep 08 15:27:04.286434 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:04.286353    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-zdsqr"
Sep 08 15:27:04.287983 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:04.287956    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn" event={"ID":"a8fce02c-f8ae-43eb-a5d6-14c107eab70e","Type":"ContainerStarted","Data":"e86b3b530776813665f9cb133efbe6d7ca34e0dbeeb6e1022fb078e62d4c7a0e"}
Sep 08 15:27:04.288124 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:04.288111    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn"
Sep 08 15:27:04.301118 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:04.301060    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-tscfp" podStartSLOduration=2.569592111 podStartE2EDuration="5.30104676s" totalImagesPullingTime="2.731454649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:26:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:27:04.299436594 +0000 UTC m=+577.011379098" watchObservedRunningTime="2026-09-08 15:27:04.30104676 +0000 UTC m=+577.012989250"
Sep 08 15:27:04.315879 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:04.315835    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn" podStartSLOduration=2.101431413 podStartE2EDuration="5.315820573s" totalImagesPullingTime="3.21438916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:26:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:27:04.314855231 +0000 UTC m=+577.026797735" watchObservedRunningTime="2026-09-08 15:27:04.315820573 +0000 UTC m=+577.027763072"
Sep 08 15:27:04.334036 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:04.333977    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-zdsqr" podStartSLOduration=2.623454222 podStartE2EDuration="5.333956485s" totalImagesPullingTime="2.710502263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:26:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:27:04.331785741 +0000 UTC m=+577.043728243" watchObservedRunningTime="2026-09-08 15:27:04.333956485 +0000 UTC m=+577.045898980"
Sep 08 15:27:10.298651 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:10.298622    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn"
Sep 08 15:27:11.260240 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:11.260202    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:11.359483 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:11.359450    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:15.296152 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:15.296122    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-tscfp"
Sep 08 15:27:15.296665 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:15.296198    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-zdsqr"
Sep 08 15:27:17.441832 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:17.441791    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:24.805313 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:24.805266    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:27.695222 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:27.695187    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vc5n9_17e30810-dfd0-4e6f-9204-731d1a6479a3/console-operator/2.log"
Sep 08 15:27:27.695222 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:27.695203    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vc5n9_17e30810-dfd0-4e6f-9204-731d1a6479a3/console-operator/2.log"
Sep 08 15:27:34.392709 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:34.392670    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4"]
Sep 08 15:27:34.396530 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:34.396506    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4"
Sep 08 15:27:34.398892 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:34.398867    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 08 15:27:34.398892 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:34.398886    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 08 15:27:34.399101 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:34.398886    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-tmrxc\""
Sep 08 15:27:34.399284 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:34.399266    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 08 15:27:34.409054 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:34.409032    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4"]
Sep 08 15:27:34.424481 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:34.424449    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:34.478755 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:34.478716    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea31698f-b405-4f28-9378-2affc68d27be-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4\" (UID: \"ea31698f-b405-4f28-9378-2affc68d27be\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4"
Sep 08 15:27:34.478919 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:34.478759    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea31698f-b405-4f28-9378-2affc68d27be-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4\" (UID: \"ea31698f-b405-4f28-9378-2affc68d27be\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4"
Sep 08 15:27:34.478919 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:34.478791    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea31698f-b405-4f28-9378-2affc68d27be-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4\" (UID: \"ea31698f-b405-4f28-9378-2affc68d27be\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4"
Sep 08 15:27:34.478919 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:34.478854    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj9c2\" (UniqueName: \"kubernetes.io/projected/ea31698f-b405-4f28-9378-2affc68d27be-kube-api-access-qj9c2\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4\" (UID: \"ea31698f-b405-4f28-9378-2affc68d27be\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4"
Sep 08 15:27:34.478919 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:34.478914    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea31698f-b405-4f28-9378-2affc68d27be-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4\" (UID: \"ea31698f-b405-4f28-9378-2affc68d27be\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4"
Sep 08 15:27:34.479060 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:34.478958    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea31698f-b405-4f28-9378-2affc68d27be-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4\" (UID: \"ea31698f-b405-4f28-9378-2affc68d27be\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4"
Sep 08 15:27:34.579568 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:34.579533    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea31698f-b405-4f28-9378-2affc68d27be-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4\" (UID: \"ea31698f-b405-4f28-9378-2affc68d27be\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4"
Sep 08 15:27:34.579756 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:34.579577    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea31698f-b405-4f28-9378-2affc68d27be-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4\" (UID: \"ea31698f-b405-4f28-9378-2affc68d27be\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4"
Sep 08 15:27:34.579756 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:34.579598    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea31698f-b405-4f28-9378-2affc68d27be-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4\" (UID: \"ea31698f-b405-4f28-9378-2affc68d27be\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4"
Sep 08 15:27:34.579756 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:34.579725    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea31698f-b405-4f28-9378-2affc68d27be-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4\" (UID: \"ea31698f-b405-4f28-9378-2affc68d27be\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4"
Sep 08 15:27:34.579930 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:34.579787    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj9c2\" (UniqueName: \"kubernetes.io/projected/ea31698f-b405-4f28-9378-2affc68d27be-kube-api-access-qj9c2\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4\" (UID: \"ea31698f-b405-4f28-9378-2affc68d27be\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4"
Sep 08 15:27:34.579930 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:34.579871    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea31698f-b405-4f28-9378-2affc68d27be-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4\" (UID: \"ea31698f-b405-4f28-9378-2affc68d27be\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4"
Sep 08 15:27:34.580027 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:34.579997    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea31698f-b405-4f28-9378-2affc68d27be-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4\" (UID: \"ea31698f-b405-4f28-9378-2affc68d27be\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4"
Sep 08 15:27:34.580105 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:34.580042    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea31698f-b405-4f28-9378-2affc68d27be-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4\" (UID: \"ea31698f-b405-4f28-9378-2affc68d27be\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4"
Sep 08 15:27:34.580242 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:34.580222    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea31698f-b405-4f28-9378-2affc68d27be-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4\" (UID: \"ea31698f-b405-4f28-9378-2affc68d27be\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4"
Sep 08 15:27:34.581974 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:34.581951    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea31698f-b405-4f28-9378-2affc68d27be-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4\" (UID: \"ea31698f-b405-4f28-9378-2affc68d27be\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4"
Sep 08 15:27:34.582274 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:34.582253    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea31698f-b405-4f28-9378-2affc68d27be-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4\" (UID: \"ea31698f-b405-4f28-9378-2affc68d27be\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4"
Sep 08 15:27:34.588606 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:34.588583    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj9c2\" (UniqueName: \"kubernetes.io/projected/ea31698f-b405-4f28-9378-2affc68d27be-kube-api-access-qj9c2\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4\" (UID: \"ea31698f-b405-4f28-9378-2affc68d27be\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4"
Sep 08 15:27:34.709813 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:34.709782    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4"
Sep 08 15:27:34.984895 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:34.984868    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4"]
Sep 08 15:27:34.987445 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:27:34.987420    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea31698f_b405_4f28_9378_2affc68d27be.slice/crio-0911b8ded23c07f8d3db1ddb87184eab51ac265bc2c0d1f1f226758110174b6f WatchSource:0}: Error finding container 0911b8ded23c07f8d3db1ddb87184eab51ac265bc2c0d1f1f226758110174b6f: Status 404 returned error can't find the container with id 0911b8ded23c07f8d3db1ddb87184eab51ac265bc2c0d1f1f226758110174b6f
Sep 08 15:27:35.446440 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:35.446404    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4" event={"ID":"ea31698f-b405-4f28-9378-2affc68d27be","Type":"ContainerStarted","Data":"0911b8ded23c07f8d3db1ddb87184eab51ac265bc2c0d1f1f226758110174b6f"}
Sep 08 15:27:36.452876 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:36.452834    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4" event={"ID":"ea31698f-b405-4f28-9378-2affc68d27be","Type":"ContainerStarted","Data":"6db0c030cfd7c92e49cec17589e723abddedbcaec3d149b3f7036e0b9a2971c2"}
Sep 08 15:27:36.473933 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:36.473885    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4" podStartSLOduration=1.457242377 podStartE2EDuration="2.473868668s" totalImagesPullingTime="1.016626291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:27:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:27:36.472307291 +0000 UTC m=+609.184249782" watchObservedRunningTime="2026-09-08 15:27:36.473868668 +0000 UTC m=+609.185811158"
Sep 08 15:27:44.709947 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:44.709893    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4"
Sep 08 15:27:44.710343 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:44.710008    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4"
Sep 08 15:27:44.722213 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:44.722188    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4"
Sep 08 15:27:45.510383 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:45.510352    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4"
Sep 08 15:27:49.302280 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:49.302247    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8"]
Sep 08 15:27:49.307198 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:49.307174    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8"
Sep 08 15:27:49.309040 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:49.309015    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 08 15:27:49.315519 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:49.315498    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8"]
Sep 08 15:27:49.326213 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:49.326184    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:49.408267 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:49.408229    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9ff10e1-57d3-4eba-bac5-2802834deacc-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ltcb8\" (UID: \"e9ff10e1-57d3-4eba-bac5-2802834deacc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8"
Sep 08 15:27:49.408454 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:49.408275    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9ff10e1-57d3-4eba-bac5-2802834deacc-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ltcb8\" (UID: \"e9ff10e1-57d3-4eba-bac5-2802834deacc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8"
Sep 08 15:27:49.408454 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:49.408295    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9ff10e1-57d3-4eba-bac5-2802834deacc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ltcb8\" (UID: \"e9ff10e1-57d3-4eba-bac5-2802834deacc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8"
Sep 08 15:27:49.408454 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:49.408415    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ttxp\" (UniqueName: \"kubernetes.io/projected/e9ff10e1-57d3-4eba-bac5-2802834deacc-kube-api-access-9ttxp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ltcb8\" (UID: \"e9ff10e1-57d3-4eba-bac5-2802834deacc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8"
Sep 08 15:27:49.408581 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:49.408453    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9ff10e1-57d3-4eba-bac5-2802834deacc-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ltcb8\" (UID: \"e9ff10e1-57d3-4eba-bac5-2802834deacc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8"
Sep 08 15:27:49.408581 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:49.408518    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9ff10e1-57d3-4eba-bac5-2802834deacc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ltcb8\" (UID: \"e9ff10e1-57d3-4eba-bac5-2802834deacc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8"
Sep 08 15:27:49.509259 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:49.509210    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ttxp\" (UniqueName: \"kubernetes.io/projected/e9ff10e1-57d3-4eba-bac5-2802834deacc-kube-api-access-9ttxp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ltcb8\" (UID: \"e9ff10e1-57d3-4eba-bac5-2802834deacc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8"
Sep 08 15:27:49.509259 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:49.509255    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9ff10e1-57d3-4eba-bac5-2802834deacc-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ltcb8\" (UID: \"e9ff10e1-57d3-4eba-bac5-2802834deacc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8"
Sep 08 15:27:49.509517 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:49.509294    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9ff10e1-57d3-4eba-bac5-2802834deacc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ltcb8\" (UID: \"e9ff10e1-57d3-4eba-bac5-2802834deacc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8"
Sep 08 15:27:49.509517 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:49.509353    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9ff10e1-57d3-4eba-bac5-2802834deacc-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ltcb8\" (UID: \"e9ff10e1-57d3-4eba-bac5-2802834deacc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8"
Sep 08 15:27:49.509517 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:49.509385    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9ff10e1-57d3-4eba-bac5-2802834deacc-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ltcb8\" (UID: \"e9ff10e1-57d3-4eba-bac5-2802834deacc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8"
Sep 08 15:27:49.509517 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:49.509412    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9ff10e1-57d3-4eba-bac5-2802834deacc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ltcb8\" (UID: \"e9ff10e1-57d3-4eba-bac5-2802834deacc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8"
Sep 08 15:27:49.509790 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:49.509768    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9ff10e1-57d3-4eba-bac5-2802834deacc-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ltcb8\" (UID: \"e9ff10e1-57d3-4eba-bac5-2802834deacc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8"
Sep 08 15:27:49.509867 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:49.509843    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9ff10e1-57d3-4eba-bac5-2802834deacc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ltcb8\" (UID: \"e9ff10e1-57d3-4eba-bac5-2802834deacc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8"
Sep 08 15:27:49.509923 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:49.509882    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9ff10e1-57d3-4eba-bac5-2802834deacc-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ltcb8\" (UID: \"e9ff10e1-57d3-4eba-bac5-2802834deacc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8"
Sep 08 15:27:49.511687 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:49.511658    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9ff10e1-57d3-4eba-bac5-2802834deacc-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ltcb8\" (UID: \"e9ff10e1-57d3-4eba-bac5-2802834deacc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8"
Sep 08 15:27:49.512356 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:49.512310    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9ff10e1-57d3-4eba-bac5-2802834deacc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ltcb8\" (UID: \"e9ff10e1-57d3-4eba-bac5-2802834deacc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8"
Sep 08 15:27:49.521209 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:49.521185    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ttxp\" (UniqueName: \"kubernetes.io/projected/e9ff10e1-57d3-4eba-bac5-2802834deacc-kube-api-access-9ttxp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ltcb8\" (UID: \"e9ff10e1-57d3-4eba-bac5-2802834deacc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8"
Sep 08 15:27:49.617675 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:49.617573    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8"
Sep 08 15:27:49.891459 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:49.891433    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8"]
Sep 08 15:27:49.894214 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:27:49.894189    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9ff10e1_57d3_4eba_bac5_2802834deacc.slice/crio-8bfb5ae8b00e95f867269e109dcdea9997aee17b03829fd326e369efd510f3df WatchSource:0}: Error finding container 8bfb5ae8b00e95f867269e109dcdea9997aee17b03829fd326e369efd510f3df: Status 404 returned error can't find the container with id 8bfb5ae8b00e95f867269e109dcdea9997aee17b03829fd326e369efd510f3df
Sep 08 15:27:50.528458 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:50.528417    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8" event={"ID":"e9ff10e1-57d3-4eba-bac5-2802834deacc","Type":"ContainerStarted","Data":"d036c63011c5a50bf2ae74d315f3502b00a5f8a7b19da3d95222451903b7ac44"}
Sep 08 15:27:50.528876 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:50.528467    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8" event={"ID":"e9ff10e1-57d3-4eba-bac5-2802834deacc","Type":"ContainerStarted","Data":"8bfb5ae8b00e95f867269e109dcdea9997aee17b03829fd326e369efd510f3df"}
Sep 08 15:27:50.548829 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:50.548752    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8" podStartSLOduration=1.346992907 podStartE2EDuration="1.548735355s" totalImagesPullingTime="201.742448ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:27:50.548240019 +0000 UTC m=+623.260182509" watchObservedRunningTime="2026-09-08 15:27:50.548735355 +0000 UTC m=+623.260677852"
Sep 08 15:27:55.821256 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:55.821222    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:59.618341 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:59.618303    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8"
Sep 08 15:27:59.618716 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:59.618586    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8"
Sep 08 15:27:59.631172 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:27:59.631146    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8"
Sep 08 15:28:00.588083 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:28:00.588039    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ltcb8"
Sep 08 15:28:04.950230 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:28:04.950193    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:12.605536 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:28:12.605496    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:12.675399 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:28:12.675364    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:17.687169 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:28:17.687131    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:25.230467 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:29:25.230384    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:35.207199 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:29:35.207162    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:42.228756 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:29:42.228716    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:42.317735 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:29:42.317691    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:43.483873 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:29:43.483838    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:43.689245 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:29:43.689213    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:00.461658 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:00.461579    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:00.633644 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:00.633607    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:01.230649 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:01.230609    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:01.451227 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:01.451169    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:02.220836 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:02.220799    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:02.727871 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:02.727837    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:03.082102 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:03.082006    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:05.004423 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:05.004387    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:05.169210 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:05.169126    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:06.370331 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:06.370299    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v"]
Sep 08 15:30:06.372661 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:06.372638    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v"
Sep 08 15:30:06.390135 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:06.390115    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-34fed8-serving-cert\""
Sep 08 15:30:06.396570 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:06.396546    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v"]
Sep 08 15:30:06.465489 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:06.465442    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:06.497135 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:06.497106    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6796befc-a590-4901-979e-070a4678ffc8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-34fed8-df7446444-tlk9v\" (UID: \"6796befc-a590-4901-979e-070a4678ffc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v"
Sep 08 15:30:06.497305 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:06.497147    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6796befc-a590-4901-979e-070a4678ffc8-metrics-certs\") pod \"maas-api-e2e-inf-a-34fed8-df7446444-tlk9v\" (UID: \"6796befc-a590-4901-979e-070a4678ffc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v"
Sep 08 15:30:06.497305 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:06.497175    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6796befc-a590-4901-979e-070a4678ffc8-maas-api-tls\") pod \"maas-api-e2e-inf-a-34fed8-df7446444-tlk9v\" (UID: \"6796befc-a590-4901-979e-070a4678ffc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v"
Sep 08 15:30:06.497305 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:06.497194    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm8cp\" (UniqueName: \"kubernetes.io/projected/6796befc-a590-4901-979e-070a4678ffc8-kube-api-access-gm8cp\") pod \"maas-api-e2e-inf-a-34fed8-df7446444-tlk9v\" (UID: \"6796befc-a590-4901-979e-070a4678ffc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v"
Sep 08 15:30:06.598309 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:06.598278    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6796befc-a590-4901-979e-070a4678ffc8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-34fed8-df7446444-tlk9v\" (UID: \"6796befc-a590-4901-979e-070a4678ffc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v"
Sep 08 15:30:06.598482 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:06.598313    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6796befc-a590-4901-979e-070a4678ffc8-metrics-certs\") pod \"maas-api-e2e-inf-a-34fed8-df7446444-tlk9v\" (UID: \"6796befc-a590-4901-979e-070a4678ffc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v"
Sep 08 15:30:06.598482 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:06.598338    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6796befc-a590-4901-979e-070a4678ffc8-maas-api-tls\") pod \"maas-api-e2e-inf-a-34fed8-df7446444-tlk9v\" (UID: \"6796befc-a590-4901-979e-070a4678ffc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v"
Sep 08 15:30:06.598482 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:06.598356    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm8cp\" (UniqueName: \"kubernetes.io/projected/6796befc-a590-4901-979e-070a4678ffc8-kube-api-access-gm8cp\") pod \"maas-api-e2e-inf-a-34fed8-df7446444-tlk9v\" (UID: \"6796befc-a590-4901-979e-070a4678ffc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v"
Sep 08 15:30:06.598482 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:30:06.598456    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-serving-cert: secret "maas-api-e2e-inf-a-34fed8-serving-cert" not found
Sep 08 15:30:06.598691 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:30:06.598523    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6796befc-a590-4901-979e-070a4678ffc8-maas-api-tls podName:6796befc-a590-4901-979e-070a4678ffc8 nodeName:}" failed. No retries permitted until 2026-09-08 15:30:07.098507182 +0000 UTC m=+759.810449656 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6796befc-a590-4901-979e-070a4678ffc8-maas-api-tls") pod "maas-api-e2e-inf-a-34fed8-df7446444-tlk9v" (UID: "6796befc-a590-4901-979e-070a4678ffc8") : secret "maas-api-e2e-inf-a-34fed8-serving-cert" not found
Sep 08 15:30:06.598966 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:06.598946    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6796befc-a590-4901-979e-070a4678ffc8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-34fed8-df7446444-tlk9v\" (UID: \"6796befc-a590-4901-979e-070a4678ffc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v"
Sep 08 15:30:06.600760 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:06.600737    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6796befc-a590-4901-979e-070a4678ffc8-metrics-certs\") pod \"maas-api-e2e-inf-a-34fed8-df7446444-tlk9v\" (UID: \"6796befc-a590-4901-979e-070a4678ffc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v"
Sep 08 15:30:06.619728 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:06.619696    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:06.626669 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:06.626602    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm8cp\" (UniqueName: \"kubernetes.io/projected/6796befc-a590-4901-979e-070a4678ffc8-kube-api-access-gm8cp\") pod \"maas-api-e2e-inf-a-34fed8-df7446444-tlk9v\" (UID: \"6796befc-a590-4901-979e-070a4678ffc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v"
Sep 08 15:30:07.104030 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:07.103993    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6796befc-a590-4901-979e-070a4678ffc8-maas-api-tls\") pod \"maas-api-e2e-inf-a-34fed8-df7446444-tlk9v\" (UID: \"6796befc-a590-4901-979e-070a4678ffc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v"
Sep 08 15:30:07.106577 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:07.106551    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6796befc-a590-4901-979e-070a4678ffc8-maas-api-tls\") pod \"maas-api-e2e-inf-a-34fed8-df7446444-tlk9v\" (UID: \"6796befc-a590-4901-979e-070a4678ffc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v"
Sep 08 15:30:07.126559 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:07.126530    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:07.283433 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:07.283400    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v"
Sep 08 15:30:07.912696 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:07.912670    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v"]
Sep 08 15:30:07.915443 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:30:07.915415    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6796befc_a590_4901_979e_070a4678ffc8.slice/crio-495227613ee23abafd26f916ed461152e0883891a189f14db956382562181938 WatchSource:0}: Error finding container 495227613ee23abafd26f916ed461152e0883891a189f14db956382562181938: Status 404 returned error can't find the container with id 495227613ee23abafd26f916ed461152e0883891a189f14db956382562181938
Sep 08 15:30:07.916693 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:07.916673    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 15:30:08.173671 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:08.173637    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v" event={"ID":"6796befc-a590-4901-979e-070a4678ffc8","Type":"ContainerStarted","Data":"495227613ee23abafd26f916ed461152e0883891a189f14db956382562181938"}
Sep 08 15:30:09.180324 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:09.180287    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v" event={"ID":"6796befc-a590-4901-979e-070a4678ffc8","Type":"ContainerStarted","Data":"8802737cd69128549fe0e5b5abe2bb0f74a2863fcfb9a964b70b1ab5b75dca35"}
Sep 08 15:30:09.180718 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:09.180399    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v"
Sep 08 15:30:09.191644 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:09.191614    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:09.290148 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:09.290086    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v" podStartSLOduration=2.762619388 podStartE2EDuration="3.290036202s" totalImagesPullingTime="527.416814ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:30:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:30:09.281341099 +0000 UTC m=+761.993283589" watchObservedRunningTime="2026-09-08 15:30:09.290036202 +0000 UTC m=+762.001978693"
Sep 08 15:30:09.445423 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:09.445337    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:09.789841 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:09.789758    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb"]
Sep 08 15:30:09.792122 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:09.792090    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl"]
Sep 08 15:30:09.792232 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:09.792161    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb"
Sep 08 15:30:09.794647 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:09.794626    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl"
Sep 08 15:30:09.803690 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:09.803668    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-34fed8-dockercfg-w7w7c\""
Sep 08 15:30:09.811356 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:09.811336    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-34fed8\""
Sep 08 15:30:09.816718 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:09.816693    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl"]
Sep 08 15:30:09.826361 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:09.826336    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42f1de90-30c3-4706-b5fc-6d85f6ca6186-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb\" (UID: \"42f1de90-30c3-4706-b5fc-6d85f6ca6186\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb"
Sep 08 15:30:09.826476 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:09.826402    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc792\" (UniqueName: \"kubernetes.io/projected/42f1de90-30c3-4706-b5fc-6d85f6ca6186-kube-api-access-tc792\") pod \"payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb\" (UID: \"42f1de90-30c3-4706-b5fc-6d85f6ca6186\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb"
Sep 08 15:30:09.826476 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:09.826428    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7mnq\" (UniqueName: \"kubernetes.io/projected/1c7d4493-b471-4433-9644-0ef10ae92544-kube-api-access-r7mnq\") pod \"payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl\" (UID: \"1c7d4493-b471-4433-9644-0ef10ae92544\") " pod="openshift-ingress/payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl"
Sep 08 15:30:09.826476 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:09.826466    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c7d4493-b471-4433-9644-0ef10ae92544-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl\" (UID: \"1c7d4493-b471-4433-9644-0ef10ae92544\") " pod="openshift-ingress/payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl"
Sep 08 15:30:09.850432 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:09.850407    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb"]
Sep 08 15:30:09.926937 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:09.926899    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7mnq\" (UniqueName: \"kubernetes.io/projected/1c7d4493-b471-4433-9644-0ef10ae92544-kube-api-access-r7mnq\") pod \"payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl\" (UID: \"1c7d4493-b471-4433-9644-0ef10ae92544\") " pod="openshift-ingress/payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl"
Sep 08 15:30:09.927145 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:09.926945    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c7d4493-b471-4433-9644-0ef10ae92544-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl\" (UID: \"1c7d4493-b471-4433-9644-0ef10ae92544\") " pod="openshift-ingress/payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl"
Sep 08 15:30:09.927145 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:09.927022    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42f1de90-30c3-4706-b5fc-6d85f6ca6186-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb\" (UID: \"42f1de90-30c3-4706-b5fc-6d85f6ca6186\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb"
Sep 08 15:30:09.927145 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:09.927106    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tc792\" (UniqueName: \"kubernetes.io/projected/42f1de90-30c3-4706-b5fc-6d85f6ca6186-kube-api-access-tc792\") pod \"payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb\" (UID: \"42f1de90-30c3-4706-b5fc-6d85f6ca6186\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb"
Sep 08 15:30:09.927926 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:09.927898    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c7d4493-b471-4433-9644-0ef10ae92544-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl\" (UID: \"1c7d4493-b471-4433-9644-0ef10ae92544\") " pod="openshift-ingress/payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl"
Sep 08 15:30:09.927989 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:09.927931    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42f1de90-30c3-4706-b5fc-6d85f6ca6186-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb\" (UID: \"42f1de90-30c3-4706-b5fc-6d85f6ca6186\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb"
Sep 08 15:30:09.973821 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:09.973787    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc792\" (UniqueName: \"kubernetes.io/projected/42f1de90-30c3-4706-b5fc-6d85f6ca6186-kube-api-access-tc792\") pod \"payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb\" (UID: \"42f1de90-30c3-4706-b5fc-6d85f6ca6186\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb"
Sep 08 15:30:09.973929 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:09.973849    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7mnq\" (UniqueName: \"kubernetes.io/projected/1c7d4493-b471-4433-9644-0ef10ae92544-kube-api-access-r7mnq\") pod \"payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl\" (UID: \"1c7d4493-b471-4433-9644-0ef10ae92544\") " pod="openshift-ingress/payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl"
Sep 08 15:30:10.107314 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:10.107223    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb"
Sep 08 15:30:10.113861 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:10.113833    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl"
Sep 08 15:30:10.420909 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:10.420821    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl"]
Sep 08 15:30:10.424175 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:30:10.424145    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c7d4493_b471_4433_9644_0ef10ae92544.slice/crio-311e73d9d90978f5a4cd01f3686275398be3d4d188f323cf4bcc69238dd35e11 WatchSource:0}: Error finding container 311e73d9d90978f5a4cd01f3686275398be3d4d188f323cf4bcc69238dd35e11: Status 404 returned error can't find the container with id 311e73d9d90978f5a4cd01f3686275398be3d4d188f323cf4bcc69238dd35e11
Sep 08 15:30:10.431851 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:30:10.431825    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42f1de90_30c3_4706_b5fc_6d85f6ca6186.slice/crio-21d260f9547f2b9a68b1c5ddf89106bac20cd95d567ff254157b57a0c7d81a7b WatchSource:0}: Error finding container 21d260f9547f2b9a68b1c5ddf89106bac20cd95d567ff254157b57a0c7d81a7b: Status 404 returned error can't find the container with id 21d260f9547f2b9a68b1c5ddf89106bac20cd95d567ff254157b57a0c7d81a7b
Sep 08 15:30:10.438473 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:10.438447    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb"]
Sep 08 15:30:11.193508 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:11.193472    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl" event={"ID":"1c7d4493-b471-4433-9644-0ef10ae92544","Type":"ContainerStarted","Data":"78eddefc3612c9ba23b337e1a7e80f624eb01758103c576e589e9257915e763f"}
Sep 08 15:30:11.193724 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:11.193534    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl" event={"ID":"1c7d4493-b471-4433-9644-0ef10ae92544","Type":"ContainerStarted","Data":"311e73d9d90978f5a4cd01f3686275398be3d4d188f323cf4bcc69238dd35e11"}
Sep 08 15:30:11.193724 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:11.193565    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl"
Sep 08 15:30:11.195108 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:11.195055    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb" event={"ID":"42f1de90-30c3-4706-b5fc-6d85f6ca6186","Type":"ContainerStarted","Data":"fe143af4c95281ecc7b2217a02696b35576c42477043eb2f7779f6ddbb458811"}
Sep 08 15:30:11.195240 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:11.195125    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb" event={"ID":"42f1de90-30c3-4706-b5fc-6d85f6ca6186","Type":"ContainerStarted","Data":"21d260f9547f2b9a68b1c5ddf89106bac20cd95d567ff254157b57a0c7d81a7b"}
Sep 08 15:30:11.195240 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:11.195194    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb"
Sep 08 15:30:11.243204 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:11.243147    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl" podStartSLOduration=2.24312811 podStartE2EDuration="2.24312811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:30:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:30:11.240332821 +0000 UTC m=+763.952275314" watchObservedRunningTime="2026-09-08 15:30:11.24312811 +0000 UTC m=+763.955070604"
Sep 08 15:30:11.276972 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:11.276909    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb" podStartSLOduration=2.276888183 podStartE2EDuration="2.276888183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:30:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:30:11.276437781 +0000 UTC m=+763.988380275" watchObservedRunningTime="2026-09-08 15:30:11.276888183 +0000 UTC m=+763.988830675"
Sep 08 15:30:12.352653 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:12.352615    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:12.745862 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:12.745828    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:13.093937 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:13.093841    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:15.191887 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:15.191858    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v"
Sep 08 15:30:15.692275 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:15.692239    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:15.799825 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:15.799791    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:16.609821 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:16.609785    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:16.712412 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:16.712364    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:16.822133 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:16.822095    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:17.554773 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:17.554741    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc"]
Sep 08 15:30:17.557163 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:17.557145    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc"
Sep 08 15:30:17.559091 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:17.559055    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-8debe9-serving-cert\""
Sep 08 15:30:17.594609 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:17.594584    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc"]
Sep 08 15:30:17.647454 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:17.647422    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:17.700792 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:17.700761    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4fds\" (UniqueName: \"kubernetes.io/projected/d0aaa612-20ec-492d-a3d9-9e8bf485cb87-kube-api-access-f4fds\") pod \"maas-api-e2e-autores-8debe9-d9564576c-nznjc\" (UID: \"d0aaa612-20ec-492d-a3d9-9e8bf485cb87\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc"
Sep 08 15:30:17.700955 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:17.700823    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0aaa612-20ec-492d-a3d9-9e8bf485cb87-maas-api-tls\") pod \"maas-api-e2e-autores-8debe9-d9564576c-nznjc\" (UID: \"d0aaa612-20ec-492d-a3d9-9e8bf485cb87\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc"
Sep 08 15:30:17.700955 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:17.700852    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0aaa612-20ec-492d-a3d9-9e8bf485cb87-metrics-certs\") pod \"maas-api-e2e-autores-8debe9-d9564576c-nznjc\" (UID: \"d0aaa612-20ec-492d-a3d9-9e8bf485cb87\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc"
Sep 08 15:30:17.700955 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:17.700891    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0aaa612-20ec-492d-a3d9-9e8bf485cb87-openshift-service-ca\") pod \"maas-api-e2e-autores-8debe9-d9564576c-nznjc\" (UID: \"d0aaa612-20ec-492d-a3d9-9e8bf485cb87\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc"
Sep 08 15:30:17.793095 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:17.793045    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:17.802057 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:17.802029    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4fds\" (UniqueName: \"kubernetes.io/projected/d0aaa612-20ec-492d-a3d9-9e8bf485cb87-kube-api-access-f4fds\") pod \"maas-api-e2e-autores-8debe9-d9564576c-nznjc\" (UID: \"d0aaa612-20ec-492d-a3d9-9e8bf485cb87\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc"
Sep 08 15:30:17.802236 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:17.802118    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0aaa612-20ec-492d-a3d9-9e8bf485cb87-maas-api-tls\") pod \"maas-api-e2e-autores-8debe9-d9564576c-nznjc\" (UID: \"d0aaa612-20ec-492d-a3d9-9e8bf485cb87\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc"
Sep 08 15:30:17.802236 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:17.802143    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0aaa612-20ec-492d-a3d9-9e8bf485cb87-metrics-certs\") pod \"maas-api-e2e-autores-8debe9-d9564576c-nznjc\" (UID: \"d0aaa612-20ec-492d-a3d9-9e8bf485cb87\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc"
Sep 08 15:30:17.802236 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:17.802180    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0aaa612-20ec-492d-a3d9-9e8bf485cb87-openshift-service-ca\") pod \"maas-api-e2e-autores-8debe9-d9564576c-nznjc\" (UID: \"d0aaa612-20ec-492d-a3d9-9e8bf485cb87\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc"
Sep 08 15:30:17.802403 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:30:17.802246    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-serving-cert: secret "maas-api-e2e-autores-8debe9-serving-cert" not found
Sep 08 15:30:17.802403 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:30:17.802302    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0aaa612-20ec-492d-a3d9-9e8bf485cb87-maas-api-tls podName:d0aaa612-20ec-492d-a3d9-9e8bf485cb87 nodeName:}" failed. No retries permitted until 2026-09-08 15:30:18.302287014 +0000 UTC m=+771.014229483 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d0aaa612-20ec-492d-a3d9-9e8bf485cb87-maas-api-tls") pod "maas-api-e2e-autores-8debe9-d9564576c-nznjc" (UID: "d0aaa612-20ec-492d-a3d9-9e8bf485cb87") : secret "maas-api-e2e-autores-8debe9-serving-cert" not found
Sep 08 15:30:17.802890 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:17.802862    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0aaa612-20ec-492d-a3d9-9e8bf485cb87-openshift-service-ca\") pod \"maas-api-e2e-autores-8debe9-d9564576c-nznjc\" (UID: \"d0aaa612-20ec-492d-a3d9-9e8bf485cb87\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc"
Sep 08 15:30:17.804845 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:17.804799    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0aaa612-20ec-492d-a3d9-9e8bf485cb87-metrics-certs\") pod \"maas-api-e2e-autores-8debe9-d9564576c-nznjc\" (UID: \"d0aaa612-20ec-492d-a3d9-9e8bf485cb87\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc"
Sep 08 15:30:17.838754 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:17.838725    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4fds\" (UniqueName: \"kubernetes.io/projected/d0aaa612-20ec-492d-a3d9-9e8bf485cb87-kube-api-access-f4fds\") pod \"maas-api-e2e-autores-8debe9-d9564576c-nznjc\" (UID: \"d0aaa612-20ec-492d-a3d9-9e8bf485cb87\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc"
Sep 08 15:30:18.307121 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:18.307058    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0aaa612-20ec-492d-a3d9-9e8bf485cb87-maas-api-tls\") pod \"maas-api-e2e-autores-8debe9-d9564576c-nznjc\" (UID: \"d0aaa612-20ec-492d-a3d9-9e8bf485cb87\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc"
Sep 08 15:30:18.309487 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:18.309456    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0aaa612-20ec-492d-a3d9-9e8bf485cb87-maas-api-tls\") pod \"maas-api-e2e-autores-8debe9-d9564576c-nznjc\" (UID: \"d0aaa612-20ec-492d-a3d9-9e8bf485cb87\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc"
Sep 08 15:30:18.410255 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:18.410223    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:18.467938 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:18.467893    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc"
Sep 08 15:30:18.772062 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:18.772015    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc"]
Sep 08 15:30:18.775412 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:30:18.775386    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0aaa612_20ec_492d_a3d9_9e8bf485cb87.slice/crio-453ea7efe78a55973e3efc74a19c7c323aa64963f5cbc544b40a95d7af02c754 WatchSource:0}: Error finding container 453ea7efe78a55973e3efc74a19c7c323aa64963f5cbc544b40a95d7af02c754: Status 404 returned error can't find the container with id 453ea7efe78a55973e3efc74a19c7c323aa64963f5cbc544b40a95d7af02c754
Sep 08 15:30:19.238807 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:19.238768    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc" event={"ID":"d0aaa612-20ec-492d-a3d9-9e8bf485cb87","Type":"ContainerStarted","Data":"f847d4b811fdc00400da93d486f170eef9769d54dc2e07806dccbc4296b36f50"}
Sep 08 15:30:19.239063 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:19.239041    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc" event={"ID":"d0aaa612-20ec-492d-a3d9-9e8bf485cb87","Type":"ContainerStarted","Data":"453ea7efe78a55973e3efc74a19c7c323aa64963f5cbc544b40a95d7af02c754"}
Sep 08 15:30:19.239244 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:19.239229    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc"
Sep 08 15:30:19.297804 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:19.297751    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc" podStartSLOduration=1.963122653 podStartE2EDuration="2.297735269s" totalImagesPullingTime="334.612616ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:30:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:30:19.296897286 +0000 UTC m=+772.008839804" watchObservedRunningTime="2026-09-08 15:30:19.297735269 +0000 UTC m=+772.009677808"
Sep 08 15:30:19.567084 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:19.567040    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:19.607003 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:19.606971    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn"]
Sep 08 15:30:19.609605 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:19.609588    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn"
Sep 08 15:30:19.612166 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:19.612143    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-8debe9\""
Sep 08 15:30:19.715267 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:19.715235    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn"]
Sep 08 15:30:19.722723 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:19.722683    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0ada965-aebc-42ee-be96-bd3daec44a7b-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn\" (UID: \"d0ada965-aebc-42ee-be96-bd3daec44a7b\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn"
Sep 08 15:30:19.722723 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:19.722716    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wltk6\" (UniqueName: \"kubernetes.io/projected/d0ada965-aebc-42ee-be96-bd3daec44a7b-kube-api-access-wltk6\") pod \"payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn\" (UID: \"d0ada965-aebc-42ee-be96-bd3daec44a7b\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn"
Sep 08 15:30:19.788172 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:19.788138    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:19.824088 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:19.823987    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0ada965-aebc-42ee-be96-bd3daec44a7b-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn\" (UID: \"d0ada965-aebc-42ee-be96-bd3daec44a7b\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn"
Sep 08 15:30:19.824088 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:19.824021    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wltk6\" (UniqueName: \"kubernetes.io/projected/d0ada965-aebc-42ee-be96-bd3daec44a7b-kube-api-access-wltk6\") pod \"payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn\" (UID: \"d0ada965-aebc-42ee-be96-bd3daec44a7b\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn"
Sep 08 15:30:19.824630 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:19.824609    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0ada965-aebc-42ee-be96-bd3daec44a7b-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn\" (UID: \"d0ada965-aebc-42ee-be96-bd3daec44a7b\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn"
Sep 08 15:30:19.862333 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:19.862307    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wltk6\" (UniqueName: \"kubernetes.io/projected/d0ada965-aebc-42ee-be96-bd3daec44a7b-kube-api-access-wltk6\") pod \"payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn\" (UID: \"d0ada965-aebc-42ee-be96-bd3daec44a7b\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn"
Sep 08 15:30:19.919805 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:19.919769    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn"
Sep 08 15:30:20.083571 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:20.083474    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:20.277853 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:30:20.277823    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0ada965_aebc_42ee_be96_bd3daec44a7b.slice/crio-52e36d7fb50f1e46318f1349975ccec1466187813b1eefbdf1d79025e6e28f31 WatchSource:0}: Error finding container 52e36d7fb50f1e46318f1349975ccec1466187813b1eefbdf1d79025e6e28f31: Status 404 returned error can't find the container with id 52e36d7fb50f1e46318f1349975ccec1466187813b1eefbdf1d79025e6e28f31
Sep 08 15:30:20.295732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:20.295703    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn"]
Sep 08 15:30:20.395401 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:20.395364    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:20.618389 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:20.618296    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:20.845393 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:20.845345    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:21.253220 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:21.253182    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn" event={"ID":"d0ada965-aebc-42ee-be96-bd3daec44a7b","Type":"ContainerStarted","Data":"4966d358e008d68bf90b6ba1ccb7fe88d8715dcd59faed5165ff22f97903e7f1"}
Sep 08 15:30:21.253399 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:21.253250    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn" event={"ID":"d0ada965-aebc-42ee-be96-bd3daec44a7b","Type":"ContainerStarted","Data":"52e36d7fb50f1e46318f1349975ccec1466187813b1eefbdf1d79025e6e28f31"}
Sep 08 15:30:21.320279 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:21.320223    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn" podStartSLOduration=2.320203903 podStartE2EDuration="2.320203903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:30:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:30:21.317349212 +0000 UTC m=+774.029291713" watchObservedRunningTime="2026-09-08 15:30:21.320203903 +0000 UTC m=+774.032146390"
Sep 08 15:30:22.201986 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:22.201957    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb"
Sep 08 15:30:22.202378 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:22.202028    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl"
Sep 08 15:30:22.259008 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:22.258976    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn"
Sep 08 15:30:22.380563 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:22.380534    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:22.484129 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:22.484035    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:22.888164 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:22.888088    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:23.545934 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:23.545900    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:23.748235 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:23.748205    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:23.998171 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:23.998138    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:24.177208 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:24.177174    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp"]
Sep 08 15:30:24.179757 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:24.179731    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp"
Sep 08 15:30:24.184244 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:24.184221    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-425c6986\""
Sep 08 15:30:24.233992 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:24.233959    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp"]
Sep 08 15:30:24.271494 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:24.271421    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45cxk\" (UniqueName: \"kubernetes.io/projected/1803b630-a007-40c8-adf8-61d2e152f6b9-kube-api-access-45cxk\") pod \"payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp\" (UID: \"1803b630-a007-40c8-adf8-61d2e152f6b9\") " pod="openshift-ingress/payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp"
Sep 08 15:30:24.271635 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:24.271535    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1803b630-a007-40c8-adf8-61d2e152f6b9-plugins-config-volume\") pod \"payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp\" (UID: \"1803b630-a007-40c8-adf8-61d2e152f6b9\") " pod="openshift-ingress/payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp"
Sep 08 15:30:24.372331 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:24.372294    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1803b630-a007-40c8-adf8-61d2e152f6b9-plugins-config-volume\") pod \"payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp\" (UID: \"1803b630-a007-40c8-adf8-61d2e152f6b9\") " pod="openshift-ingress/payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp"
Sep 08 15:30:24.372331 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:24.372335    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45cxk\" (UniqueName: \"kubernetes.io/projected/1803b630-a007-40c8-adf8-61d2e152f6b9-kube-api-access-45cxk\") pod \"payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp\" (UID: \"1803b630-a007-40c8-adf8-61d2e152f6b9\") " pod="openshift-ingress/payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp"
Sep 08 15:30:24.373247 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:24.373196    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1803b630-a007-40c8-adf8-61d2e152f6b9-plugins-config-volume\") pod \"payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp\" (UID: \"1803b630-a007-40c8-adf8-61d2e152f6b9\") " pod="openshift-ingress/payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp"
Sep 08 15:30:24.399026 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:24.398991    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45cxk\" (UniqueName: \"kubernetes.io/projected/1803b630-a007-40c8-adf8-61d2e152f6b9-kube-api-access-45cxk\") pod \"payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp\" (UID: \"1803b630-a007-40c8-adf8-61d2e152f6b9\") " pod="openshift-ingress/payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp"
Sep 08 15:30:24.489684 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:24.489651    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp"
Sep 08 15:30:24.803287 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:24.803249    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp"]
Sep 08 15:30:24.806719 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:30:24.806685    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1803b630_a007_40c8_adf8_61d2e152f6b9.slice/crio-4728394b827e16b6d1276189479a35d2d9b5f379ee53c800ad795b5241c62dd9 WatchSource:0}: Error finding container 4728394b827e16b6d1276189479a35d2d9b5f379ee53c800ad795b5241c62dd9: Status 404 returned error can't find the container with id 4728394b827e16b6d1276189479a35d2d9b5f379ee53c800ad795b5241c62dd9
Sep 08 15:30:25.206478 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:25.206438    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:25.276507 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:25.276469    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp" event={"ID":"1803b630-a007-40c8-adf8-61d2e152f6b9","Type":"ContainerStarted","Data":"2bfda95022c48b56a32d3dd0c62c281563f9760f5e40943f9ac6cb7056640a8d"}
Sep 08 15:30:25.276645 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:25.276532    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp" event={"ID":"1803b630-a007-40c8-adf8-61d2e152f6b9","Type":"ContainerStarted","Data":"4728394b827e16b6d1276189479a35d2d9b5f379ee53c800ad795b5241c62dd9"}
Sep 08 15:30:25.276645 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:25.276609    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp"
Sep 08 15:30:25.521291 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:25.521210    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:26.258879 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:26.258851    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc"
Sep 08 15:30:26.321372 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:26.321318    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp" podStartSLOduration=2.321299815 podStartE2EDuration="2.321299815s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:30:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:30:25.3260125 +0000 UTC m=+778.037955002" watchObservedRunningTime="2026-09-08 15:30:26.321299815 +0000 UTC m=+779.033242305"
Sep 08 15:30:27.159333 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:27.159288    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:29.933553 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:29.933503    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:30.753649 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:30.753611    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:33.099683 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:33.099649    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:33.266530 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:33.266499    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn"
Sep 08 15:30:33.975222 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:33.975185    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:36.284139 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:36.284053    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp"
Sep 08 15:30:40.138727 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:40.138686    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:40.540317 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:40.540279    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:41.240562 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:41.240515    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:41.575470 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:41.575381    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:43.158289 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:43.158254    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:45.249318 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:45.249267    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:46.696801 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:46.696771    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:46.979351 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:46.979277    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:47.167989 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:47.167952    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:47.318404 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:47.318324    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:48.231077 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:48.231039    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:48.435522 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:48.435490    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5"]
Sep 08 15:30:48.440435 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:48.440410    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5"
Sep 08 15:30:48.443688 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:48.443661    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-e39762\""
Sep 08 15:30:48.468841 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:48.468802    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5"]
Sep 08 15:30:48.493685 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:48.493587    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c9cbb9b-2f0e-443d-803f-c92512a4bcef-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5\" (UID: \"8c9cbb9b-2f0e-443d-803f-c92512a4bcef\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5"
Sep 08 15:30:48.493852 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:48.493702    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m49rt\" (UniqueName: \"kubernetes.io/projected/8c9cbb9b-2f0e-443d-803f-c92512a4bcef-kube-api-access-m49rt\") pod \"payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5\" (UID: \"8c9cbb9b-2f0e-443d-803f-c92512a4bcef\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5"
Sep 08 15:30:48.595105 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:48.595053    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m49rt\" (UniqueName: \"kubernetes.io/projected/8c9cbb9b-2f0e-443d-803f-c92512a4bcef-kube-api-access-m49rt\") pod \"payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5\" (UID: \"8c9cbb9b-2f0e-443d-803f-c92512a4bcef\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5"
Sep 08 15:30:48.595307 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:48.595175    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c9cbb9b-2f0e-443d-803f-c92512a4bcef-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5\" (UID: \"8c9cbb9b-2f0e-443d-803f-c92512a4bcef\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5"
Sep 08 15:30:48.595876 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:48.595853    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c9cbb9b-2f0e-443d-803f-c92512a4bcef-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5\" (UID: \"8c9cbb9b-2f0e-443d-803f-c92512a4bcef\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5"
Sep 08 15:30:48.615120 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:48.615092    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m49rt\" (UniqueName: \"kubernetes.io/projected/8c9cbb9b-2f0e-443d-803f-c92512a4bcef-kube-api-access-m49rt\") pod \"payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5\" (UID: \"8c9cbb9b-2f0e-443d-803f-c92512a4bcef\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5"
Sep 08 15:30:48.751734 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:48.751650    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5"
Sep 08 15:30:49.120413 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:49.120337    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5"]
Sep 08 15:30:49.127468 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:30:49.127439    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c9cbb9b_2f0e_443d_803f_c92512a4bcef.slice/crio-96c0bc802e38088bcb51ecf3d29ef59da0da99cd8e3c68786bec51095eb8996d WatchSource:0}: Error finding container 96c0bc802e38088bcb51ecf3d29ef59da0da99cd8e3c68786bec51095eb8996d: Status 404 returned error can't find the container with id 96c0bc802e38088bcb51ecf3d29ef59da0da99cd8e3c68786bec51095eb8996d
Sep 08 15:30:49.404460 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:49.404371    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5" event={"ID":"8c9cbb9b-2f0e-443d-803f-c92512a4bcef","Type":"ContainerStarted","Data":"8343e417432b195e7385a0971c4e573b31420477e3fef20ab260b18a8a43063c"}
Sep 08 15:30:49.404460 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:49.404424    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5" event={"ID":"8c9cbb9b-2f0e-443d-803f-c92512a4bcef","Type":"ContainerStarted","Data":"96c0bc802e38088bcb51ecf3d29ef59da0da99cd8e3c68786bec51095eb8996d"}
Sep 08 15:30:49.404860 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:49.404481    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5"
Sep 08 15:30:49.489308 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:49.489276    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:49.582733 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:49.582702    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:51.113839 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:51.113744    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:51.240750 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:51.240662    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:51.475897 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:51.475867    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:52.986576 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:52.986523    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:56.881981 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:56.881931    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:58.595057 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:58.595018    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:58.658236 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:58.658205    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:58.754159 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:58.754121    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:58.934961 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:30:58.934928    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:00.412933 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:00.412902    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5"
Sep 08 15:31:00.445119 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:00.445039    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5" podStartSLOduration=12.445017131 podStartE2EDuration="12.445017131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:30:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:30:49.459397217 +0000 UTC m=+802.171339709" watchObservedRunningTime="2026-09-08 15:31:00.445017131 +0000 UTC m=+813.156959623"
Sep 08 15:31:02.814087 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:02.814036    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:02.832661 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:02.832627    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp"]
Sep 08 15:31:02.832905 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:02.832876    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp" podUID="1803b630-a007-40c8-adf8-61d2e152f6b9" containerName="payload-processing" containerID="cri-o://2bfda95022c48b56a32d3dd0c62c281563f9760f5e40943f9ac6cb7056640a8d" gracePeriod=30
Sep 08 15:31:02.892332 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:02.892302    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:03.098002 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:03.097977    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp"
Sep 08 15:31:03.168407 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:03.168360    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:03.235030 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:03.234991    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45cxk\" (UniqueName: \"kubernetes.io/projected/1803b630-a007-40c8-adf8-61d2e152f6b9-kube-api-access-45cxk\") pod \"1803b630-a007-40c8-adf8-61d2e152f6b9\" (UID: \"1803b630-a007-40c8-adf8-61d2e152f6b9\") "
Sep 08 15:31:03.235227 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:03.235045    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1803b630-a007-40c8-adf8-61d2e152f6b9-plugins-config-volume\") pod \"1803b630-a007-40c8-adf8-61d2e152f6b9\" (UID: \"1803b630-a007-40c8-adf8-61d2e152f6b9\") "
Sep 08 15:31:03.236618 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:03.236588    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1803b630-a007-40c8-adf8-61d2e152f6b9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1803b630-a007-40c8-adf8-61d2e152f6b9" (UID: "1803b630-a007-40c8-adf8-61d2e152f6b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:31:03.237355 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:03.237331    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1803b630-a007-40c8-adf8-61d2e152f6b9-kube-api-access-45cxk" (OuterVolumeSpecName: "kube-api-access-45cxk") pod "1803b630-a007-40c8-adf8-61d2e152f6b9" (UID: "1803b630-a007-40c8-adf8-61d2e152f6b9"). InnerVolumeSpecName "kube-api-access-45cxk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:31:03.336130 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:03.336044    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45cxk\" (UniqueName: \"kubernetes.io/projected/1803b630-a007-40c8-adf8-61d2e152f6b9-kube-api-access-45cxk\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:31:03.336130 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:03.336102    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1803b630-a007-40c8-adf8-61d2e152f6b9-plugins-config-volume\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:31:03.402717 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:03.402683    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:03.488094 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:03.488038    2560 generic.go:356] "Generic (PLEG): container finished" podID="1803b630-a007-40c8-adf8-61d2e152f6b9" containerID="2bfda95022c48b56a32d3dd0c62c281563f9760f5e40943f9ac6cb7056640a8d" exitCode=0
Sep 08 15:31:03.488288 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:03.488108    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp"
Sep 08 15:31:03.488288 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:03.488106    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp" event={"ID":"1803b630-a007-40c8-adf8-61d2e152f6b9","Type":"ContainerDied","Data":"2bfda95022c48b56a32d3dd0c62c281563f9760f5e40943f9ac6cb7056640a8d"}
Sep 08 15:31:03.488288 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:03.488252    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp" event={"ID":"1803b630-a007-40c8-adf8-61d2e152f6b9","Type":"ContainerDied","Data":"4728394b827e16b6d1276189479a35d2d9b5f379ee53c800ad795b5241c62dd9"}
Sep 08 15:31:03.488288 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:03.488287    2560 scope.go:117] "RemoveContainer" containerID="2bfda95022c48b56a32d3dd0c62c281563f9760f5e40943f9ac6cb7056640a8d"
Sep 08 15:31:03.497924 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:03.497905    2560 scope.go:117] "RemoveContainer" containerID="2bfda95022c48b56a32d3dd0c62c281563f9760f5e40943f9ac6cb7056640a8d"
Sep 08 15:31:03.498266 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:31:03.498242    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bfda95022c48b56a32d3dd0c62c281563f9760f5e40943f9ac6cb7056640a8d\": container with ID starting with 2bfda95022c48b56a32d3dd0c62c281563f9760f5e40943f9ac6cb7056640a8d not found: ID does not exist" containerID="2bfda95022c48b56a32d3dd0c62c281563f9760f5e40943f9ac6cb7056640a8d"
Sep 08 15:31:03.498349 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:03.498279    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bfda95022c48b56a32d3dd0c62c281563f9760f5e40943f9ac6cb7056640a8d"} err="failed to get container status \"2bfda95022c48b56a32d3dd0c62c281563f9760f5e40943f9ac6cb7056640a8d\": rpc error: code = NotFound desc = could not find container \"2bfda95022c48b56a32d3dd0c62c281563f9760f5e40943f9ac6cb7056640a8d\": container with ID starting with 2bfda95022c48b56a32d3dd0c62c281563f9760f5e40943f9ac6cb7056640a8d not found: ID does not exist"
Sep 08 15:31:03.499166 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:03.499147    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:03.531817 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:03.531780    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp"]
Sep 08 15:31:03.543546 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:03.543518    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-425c6986-7675dd4749-4gjkp"]
Sep 08 15:31:03.801121 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:03.801089    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1803b630-a007-40c8-adf8-61d2e152f6b9" path="/var/lib/kubelet/pods/1803b630-a007-40c8-adf8-61d2e152f6b9/volumes"
Sep 08 15:31:03.929307 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:03.929278    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:04.058817 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:04.058713    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:04.321992 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:04.321892    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:04.579224 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:04.579115    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:04.830510 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:04.830414    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:07.577573 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:07.577513    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:07.678608 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:07.678569    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:07.734244 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:07.734212    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx"]
Sep 08 15:31:07.734840 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:07.734823    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1803b630-a007-40c8-adf8-61d2e152f6b9" containerName="payload-processing"
Sep 08 15:31:07.734891 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:07.734845    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1803b630-a007-40c8-adf8-61d2e152f6b9" containerName="payload-processing"
Sep 08 15:31:07.734979 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:07.734967    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1803b630-a007-40c8-adf8-61d2e152f6b9" containerName="payload-processing"
Sep 08 15:31:07.740149 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:07.740125    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx"
Sep 08 15:31:07.743521 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:07.743496    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-96c53ecc-serving-cert\""
Sep 08 15:31:07.753996 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:07.753969    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx"]
Sep 08 15:31:07.783456 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:07.783418    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:07.840962 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:07.840880    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:07.878846 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:07.878810    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xhxv\" (UniqueName: \"kubernetes.io/projected/e474cbac-a9f7-4313-826a-5375b40ed02c-kube-api-access-9xhxv\") pod \"maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx\" (UID: \"e474cbac-a9f7-4313-826a-5375b40ed02c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx"
Sep 08 15:31:07.879001 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:07.878915    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e474cbac-a9f7-4313-826a-5375b40ed02c-metrics-certs\") pod \"maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx\" (UID: \"e474cbac-a9f7-4313-826a-5375b40ed02c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx"
Sep 08 15:31:07.879001 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:07.878952    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e474cbac-a9f7-4313-826a-5375b40ed02c-openshift-service-ca\") pod \"maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx\" (UID: \"e474cbac-a9f7-4313-826a-5375b40ed02c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx"
Sep 08 15:31:07.879001 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:07.878975    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e474cbac-a9f7-4313-826a-5375b40ed02c-maas-api-tls\") pod \"maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx\" (UID: \"e474cbac-a9f7-4313-826a-5375b40ed02c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx"
Sep 08 15:31:07.980115 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:07.980051    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xhxv\" (UniqueName: \"kubernetes.io/projected/e474cbac-a9f7-4313-826a-5375b40ed02c-kube-api-access-9xhxv\") pod \"maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx\" (UID: \"e474cbac-a9f7-4313-826a-5375b40ed02c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx"
Sep 08 15:31:07.980299 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:07.980149    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e474cbac-a9f7-4313-826a-5375b40ed02c-metrics-certs\") pod \"maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx\" (UID: \"e474cbac-a9f7-4313-826a-5375b40ed02c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx"
Sep 08 15:31:07.980299 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:07.980178    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e474cbac-a9f7-4313-826a-5375b40ed02c-openshift-service-ca\") pod \"maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx\" (UID: \"e474cbac-a9f7-4313-826a-5375b40ed02c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx"
Sep 08 15:31:07.980299 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:07.980195    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e474cbac-a9f7-4313-826a-5375b40ed02c-maas-api-tls\") pod \"maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx\" (UID: \"e474cbac-a9f7-4313-826a-5375b40ed02c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx"
Sep 08 15:31:07.980951 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:07.980921    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e474cbac-a9f7-4313-826a-5375b40ed02c-openshift-service-ca\") pod \"maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx\" (UID: \"e474cbac-a9f7-4313-826a-5375b40ed02c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx"
Sep 08 15:31:07.982790 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:07.982766    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e474cbac-a9f7-4313-826a-5375b40ed02c-maas-api-tls\") pod \"maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx\" (UID: \"e474cbac-a9f7-4313-826a-5375b40ed02c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx"
Sep 08 15:31:07.982872 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:07.982820    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e474cbac-a9f7-4313-826a-5375b40ed02c-metrics-certs\") pod \"maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx\" (UID: \"e474cbac-a9f7-4313-826a-5375b40ed02c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx"
Sep 08 15:31:07.994470 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:07.994447    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xhxv\" (UniqueName: \"kubernetes.io/projected/e474cbac-a9f7-4313-826a-5375b40ed02c-kube-api-access-9xhxv\") pod \"maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx\" (UID: \"e474cbac-a9f7-4313-826a-5375b40ed02c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx"
Sep 08 15:31:08.008483 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:08.008461    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:08.055535 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:08.055492    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx"
Sep 08 15:31:08.345559 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:08.345519    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx"]
Sep 08 15:31:08.348896 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:31:08.348866    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode474cbac_a9f7_4313_826a_5375b40ed02c.slice/crio-ffed0caa057d2e60c505b92ee792446f444a71ed95580df1aee6ef9b9f1aae70 WatchSource:0}: Error finding container ffed0caa057d2e60c505b92ee792446f444a71ed95580df1aee6ef9b9f1aae70: Status 404 returned error can't find the container with id ffed0caa057d2e60c505b92ee792446f444a71ed95580df1aee6ef9b9f1aae70
Sep 08 15:31:08.504428 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:08.504394    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:08.515296 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:08.515261    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx" event={"ID":"e474cbac-a9f7-4313-826a-5375b40ed02c","Type":"ContainerStarted","Data":"ffed0caa057d2e60c505b92ee792446f444a71ed95580df1aee6ef9b9f1aae70"}
Sep 08 15:31:08.612839 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:08.612807    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:08.693477 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:08.693437    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs"]
Sep 08 15:31:08.698568 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:08.698547    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs"
Sep 08 15:31:08.702114 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:08.702095    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-96c53ecc\""
Sep 08 15:31:08.702210 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:08.702168    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-96c53ecc-dockercfg-qzg8d\""
Sep 08 15:31:08.713433 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:08.713407    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs"]
Sep 08 15:31:08.734490 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:08.734424    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:08.787465 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:08.787383    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfswz\" (UniqueName: \"kubernetes.io/projected/0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84-kube-api-access-mfswz\") pod \"payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs\" (UID: \"0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs"
Sep 08 15:31:08.787465 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:08.787443    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs\" (UID: \"0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs"
Sep 08 15:31:08.888631 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:08.888605    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfswz\" (UniqueName: \"kubernetes.io/projected/0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84-kube-api-access-mfswz\") pod \"payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs\" (UID: \"0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs"
Sep 08 15:31:08.888748 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:08.888657    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs\" (UID: \"0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs"
Sep 08 15:31:08.889421 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:08.889395    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs\" (UID: \"0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs"
Sep 08 15:31:08.896617 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:08.896593    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfswz\" (UniqueName: \"kubernetes.io/projected/0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84-kube-api-access-mfswz\") pod \"payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs\" (UID: \"0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs"
Sep 08 15:31:09.009807 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:09.009763    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs"
Sep 08 15:31:09.196198 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:09.196168    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:09.289567 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:09.289542    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs"]
Sep 08 15:31:09.522711 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:09.522677    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx" event={"ID":"e474cbac-a9f7-4313-826a-5375b40ed02c","Type":"ContainerStarted","Data":"1be5aacded348c875eca2857e367c411610084a3272a89d6b2443df9e3c23b2f"}
Sep 08 15:31:09.522884 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:09.522736    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx"
Sep 08 15:31:09.525997 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:09.525959    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs" event={"ID":"0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84","Type":"ContainerStarted","Data":"dd8270c954e4e0cc6c51c345561decddd154c34475cdb8207a2bfd140a4b38aa"}
Sep 08 15:31:09.526158 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:09.526061    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs" event={"ID":"0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84","Type":"ContainerStarted","Data":"074b680274c59bb58b4263d5a7a442f21933d60163b9d47173e7c8614a506579"}
Sep 08 15:31:09.526608 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:09.526582    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs"
Sep 08 15:31:09.544621 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:09.544573    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx" podStartSLOduration=2.061477962 podStartE2EDuration="2.544558421s" totalImagesPullingTime="483.080459ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:31:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:31:09.543408119 +0000 UTC m=+822.255350610" watchObservedRunningTime="2026-09-08 15:31:09.544558421 +0000 UTC m=+822.256500913"
Sep 08 15:31:09.566569 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:09.566523    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs" podStartSLOduration=1.566508885 podStartE2EDuration="1.566508885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:31:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:31:09.564367159 +0000 UTC m=+822.276309651" watchObservedRunningTime="2026-09-08 15:31:09.566508885 +0000 UTC m=+822.278451423"
Sep 08 15:31:10.139832 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:10.139785    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:11.670627 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:11.670584    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:15.107799 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:15.107763    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:15.536193 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:15.536164    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx"
Sep 08 15:31:20.533633 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:20.533605    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs"
Sep 08 15:31:28.664485 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:28.664446    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:29.650730 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:29.650675    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:30.678155 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:30.678118    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:31.685697 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:31.685663    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:33.083766 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:33.083723    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:35.228148 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:35.228108    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:38.719739 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:38.719702    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:39.532558 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:39.532522    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx"]
Sep 08 15:31:39.532817 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:39.532790    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx" podUID="e474cbac-a9f7-4313-826a-5375b40ed02c" containerName="maas-api" containerID="cri-o://1be5aacded348c875eca2857e367c411610084a3272a89d6b2443df9e3c23b2f" gracePeriod=30
Sep 08 15:31:39.575979 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:39.575946    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:39.659158 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:39.659116    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs"]
Sep 08 15:31:39.659388 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:39.659359    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs" podUID="0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84" containerName="payload-pre-processing" containerID="cri-o://dd8270c954e4e0cc6c51c345561decddd154c34475cdb8207a2bfd140a4b38aa" gracePeriod=30
Sep 08 15:31:39.692171 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:39.692117    2560 generic.go:356] "Generic (PLEG): container finished" podID="e474cbac-a9f7-4313-826a-5375b40ed02c" containerID="1be5aacded348c875eca2857e367c411610084a3272a89d6b2443df9e3c23b2f" exitCode=0
Sep 08 15:31:39.692310 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:39.692186    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx" event={"ID":"e474cbac-a9f7-4313-826a-5375b40ed02c","Type":"ContainerDied","Data":"1be5aacded348c875eca2857e367c411610084a3272a89d6b2443df9e3c23b2f"}
Sep 08 15:31:39.706462 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:39.706435    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:39.804574 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:39.804539    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:39.808977 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:39.808955    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx"
Sep 08 15:31:39.870959 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:39.869355    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xhxv\" (UniqueName: \"kubernetes.io/projected/e474cbac-a9f7-4313-826a-5375b40ed02c-kube-api-access-9xhxv\") pod \"e474cbac-a9f7-4313-826a-5375b40ed02c\" (UID: \"e474cbac-a9f7-4313-826a-5375b40ed02c\") "
Sep 08 15:31:39.870959 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:39.869424    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e474cbac-a9f7-4313-826a-5375b40ed02c-maas-api-tls\") pod \"e474cbac-a9f7-4313-826a-5375b40ed02c\" (UID: \"e474cbac-a9f7-4313-826a-5375b40ed02c\") "
Sep 08 15:31:39.870959 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:39.869565    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e474cbac-a9f7-4313-826a-5375b40ed02c-openshift-service-ca\") pod \"e474cbac-a9f7-4313-826a-5375b40ed02c\" (UID: \"e474cbac-a9f7-4313-826a-5375b40ed02c\") "
Sep 08 15:31:39.870959 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:39.869659    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e474cbac-a9f7-4313-826a-5375b40ed02c-metrics-certs\") pod \"e474cbac-a9f7-4313-826a-5375b40ed02c\" (UID: \"e474cbac-a9f7-4313-826a-5375b40ed02c\") "
Sep 08 15:31:39.870959 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:39.870378    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e474cbac-a9f7-4313-826a-5375b40ed02c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e474cbac-a9f7-4313-826a-5375b40ed02c" (UID: "e474cbac-a9f7-4313-826a-5375b40ed02c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:31:39.875854 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:39.875822    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e474cbac-a9f7-4313-826a-5375b40ed02c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e474cbac-a9f7-4313-826a-5375b40ed02c" (UID: "e474cbac-a9f7-4313-826a-5375b40ed02c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:31:39.876225 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:39.876202    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc"]
Sep 08 15:31:39.876345 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:39.876257    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e474cbac-a9f7-4313-826a-5375b40ed02c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e474cbac-a9f7-4313-826a-5375b40ed02c" (UID: "e474cbac-a9f7-4313-826a-5375b40ed02c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:31:39.876481 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:39.876444    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc" podUID="d0aaa612-20ec-492d-a3d9-9e8bf485cb87" containerName="maas-api" containerID="cri-o://f847d4b811fdc00400da93d486f170eef9769d54dc2e07806dccbc4296b36f50" gracePeriod=30
Sep 08 15:31:39.877440 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:39.877310    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e474cbac-a9f7-4313-826a-5375b40ed02c-kube-api-access-9xhxv" (OuterVolumeSpecName: "kube-api-access-9xhxv") pod "e474cbac-a9f7-4313-826a-5375b40ed02c" (UID: "e474cbac-a9f7-4313-826a-5375b40ed02c"). InnerVolumeSpecName "kube-api-access-9xhxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:31:39.922405 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:39.922371    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:39.970794 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:39.970761    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xhxv\" (UniqueName: \"kubernetes.io/projected/e474cbac-a9f7-4313-826a-5375b40ed02c-kube-api-access-9xhxv\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:31:39.970794 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:39.970789    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e474cbac-a9f7-4313-826a-5375b40ed02c-maas-api-tls\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:31:39.970794 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:39.970798    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e474cbac-a9f7-4313-826a-5375b40ed02c-openshift-service-ca\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:31:39.971013 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:39.970807    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e474cbac-a9f7-4313-826a-5375b40ed02c-metrics-certs\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:31:39.998723 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:39.998683    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs"
Sep 08 15:31:40.052803 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.051876    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn"]
Sep 08 15:31:40.052803 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.052267    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn" podUID="d0ada965-aebc-42ee-be96-bd3daec44a7b" containerName="payload-pre-processing" containerID="cri-o://4966d358e008d68bf90b6ba1ccb7fe88d8715dcd59faed5165ff22f97903e7f1" gracePeriod=30
Sep 08 15:31:40.071786 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.071763    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfswz\" (UniqueName: \"kubernetes.io/projected/0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84-kube-api-access-mfswz\") pod \"0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84\" (UID: \"0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84\") "
Sep 08 15:31:40.071935 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.071917    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84-plugins-config-volume\") pod \"0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84\" (UID: \"0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84\") "
Sep 08 15:31:40.072364 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.072338    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84" (UID: "0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:31:40.073971 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.073949    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84-kube-api-access-mfswz" (OuterVolumeSpecName: "kube-api-access-mfswz") pod "0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84" (UID: "0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84"). InnerVolumeSpecName "kube-api-access-mfswz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:31:40.168907 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.168884    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc"
Sep 08 15:31:40.172939 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.172913    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfswz\" (UniqueName: \"kubernetes.io/projected/0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84-kube-api-access-mfswz\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:31:40.172939 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.172940    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84-plugins-config-volume\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:31:40.283325 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.281462    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0aaa612-20ec-492d-a3d9-9e8bf485cb87-metrics-certs\") pod \"d0aaa612-20ec-492d-a3d9-9e8bf485cb87\" (UID: \"d0aaa612-20ec-492d-a3d9-9e8bf485cb87\") "
Sep 08 15:31:40.283325 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.281571    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4fds\" (UniqueName: \"kubernetes.io/projected/d0aaa612-20ec-492d-a3d9-9e8bf485cb87-kube-api-access-f4fds\") pod \"d0aaa612-20ec-492d-a3d9-9e8bf485cb87\" (UID: \"d0aaa612-20ec-492d-a3d9-9e8bf485cb87\") "
Sep 08 15:31:40.283325 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.281617    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0aaa612-20ec-492d-a3d9-9e8bf485cb87-openshift-service-ca\") pod \"d0aaa612-20ec-492d-a3d9-9e8bf485cb87\" (UID: \"d0aaa612-20ec-492d-a3d9-9e8bf485cb87\") "
Sep 08 15:31:40.283325 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.281663    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0aaa612-20ec-492d-a3d9-9e8bf485cb87-maas-api-tls\") pod \"d0aaa612-20ec-492d-a3d9-9e8bf485cb87\" (UID: \"d0aaa612-20ec-492d-a3d9-9e8bf485cb87\") "
Sep 08 15:31:40.285143 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.283696    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0aaa612-20ec-492d-a3d9-9e8bf485cb87-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d0aaa612-20ec-492d-a3d9-9e8bf485cb87" (UID: "d0aaa612-20ec-492d-a3d9-9e8bf485cb87"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:31:40.285143 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.285058    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0aaa612-20ec-492d-a3d9-9e8bf485cb87-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d0aaa612-20ec-492d-a3d9-9e8bf485cb87" (UID: "d0aaa612-20ec-492d-a3d9-9e8bf485cb87"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:31:40.286842 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.286802    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0aaa612-20ec-492d-a3d9-9e8bf485cb87-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d0aaa612-20ec-492d-a3d9-9e8bf485cb87" (UID: "d0aaa612-20ec-492d-a3d9-9e8bf485cb87"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:31:40.286991 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.286918    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0aaa612-20ec-492d-a3d9-9e8bf485cb87-kube-api-access-f4fds" (OuterVolumeSpecName: "kube-api-access-f4fds") pod "d0aaa612-20ec-492d-a3d9-9e8bf485cb87" (UID: "d0aaa612-20ec-492d-a3d9-9e8bf485cb87"). InnerVolumeSpecName "kube-api-access-f4fds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:31:40.333974 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.333949    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn"
Sep 08 15:31:40.383106 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.383019    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0ada965-aebc-42ee-be96-bd3daec44a7b-plugins-config-volume\") pod \"d0ada965-aebc-42ee-be96-bd3daec44a7b\" (UID: \"d0ada965-aebc-42ee-be96-bd3daec44a7b\") "
Sep 08 15:31:40.383106 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.383104    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wltk6\" (UniqueName: \"kubernetes.io/projected/d0ada965-aebc-42ee-be96-bd3daec44a7b-kube-api-access-wltk6\") pod \"d0ada965-aebc-42ee-be96-bd3daec44a7b\" (UID: \"d0ada965-aebc-42ee-be96-bd3daec44a7b\") "
Sep 08 15:31:40.383492 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.383437    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0aaa612-20ec-492d-a3d9-9e8bf485cb87-metrics-certs\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:31:40.383492 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.383460    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0ada965-aebc-42ee-be96-bd3daec44a7b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d0ada965-aebc-42ee-be96-bd3daec44a7b" (UID: "d0ada965-aebc-42ee-be96-bd3daec44a7b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:31:40.383492 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.383477    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4fds\" (UniqueName: \"kubernetes.io/projected/d0aaa612-20ec-492d-a3d9-9e8bf485cb87-kube-api-access-f4fds\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:31:40.383492 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.383495    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0aaa612-20ec-492d-a3d9-9e8bf485cb87-openshift-service-ca\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:31:40.383788 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.383508    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0aaa612-20ec-492d-a3d9-9e8bf485cb87-maas-api-tls\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:31:40.385456 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.385432    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0ada965-aebc-42ee-be96-bd3daec44a7b-kube-api-access-wltk6" (OuterVolumeSpecName: "kube-api-access-wltk6") pod "d0ada965-aebc-42ee-be96-bd3daec44a7b" (UID: "d0ada965-aebc-42ee-be96-bd3daec44a7b"). InnerVolumeSpecName "kube-api-access-wltk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:31:40.439385 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.439348    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:40.484726 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.484690    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0ada965-aebc-42ee-be96-bd3daec44a7b-plugins-config-volume\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:31:40.484726 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.484726    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wltk6\" (UniqueName: \"kubernetes.io/projected/d0ada965-aebc-42ee-be96-bd3daec44a7b-kube-api-access-wltk6\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:31:40.561926 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.561887    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:40.660612 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.660528    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:40.698446 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.698420    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx"
Sep 08 15:31:40.698611 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.698419    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx" event={"ID":"e474cbac-a9f7-4313-826a-5375b40ed02c","Type":"ContainerDied","Data":"ffed0caa057d2e60c505b92ee792446f444a71ed95580df1aee6ef9b9f1aae70"}
Sep 08 15:31:40.698611 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.698564    2560 scope.go:117] "RemoveContainer" containerID="1be5aacded348c875eca2857e367c411610084a3272a89d6b2443df9e3c23b2f"
Sep 08 15:31:40.700126 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.699879    2560 generic.go:356] "Generic (PLEG): container finished" podID="d0aaa612-20ec-492d-a3d9-9e8bf485cb87" containerID="f847d4b811fdc00400da93d486f170eef9769d54dc2e07806dccbc4296b36f50" exitCode=0
Sep 08 15:31:40.700126 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.699938    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc"
Sep 08 15:31:40.700126 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.699958    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc" event={"ID":"d0aaa612-20ec-492d-a3d9-9e8bf485cb87","Type":"ContainerDied","Data":"f847d4b811fdc00400da93d486f170eef9769d54dc2e07806dccbc4296b36f50"}
Sep 08 15:31:40.700126 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.700003    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc" event={"ID":"d0aaa612-20ec-492d-a3d9-9e8bf485cb87","Type":"ContainerDied","Data":"453ea7efe78a55973e3efc74a19c7c323aa64963f5cbc544b40a95d7af02c754"}
Sep 08 15:31:40.707114 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.701705    2560 generic.go:356] "Generic (PLEG): container finished" podID="0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84" containerID="dd8270c954e4e0cc6c51c345561decddd154c34475cdb8207a2bfd140a4b38aa" exitCode=0
Sep 08 15:31:40.707114 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.701926    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs"
Sep 08 15:31:40.707114 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.703302    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs" event={"ID":"0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84","Type":"ContainerDied","Data":"dd8270c954e4e0cc6c51c345561decddd154c34475cdb8207a2bfd140a4b38aa"}
Sep 08 15:31:40.707114 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.703355    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs" event={"ID":"0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84","Type":"ContainerDied","Data":"074b680274c59bb58b4263d5a7a442f21933d60163b9d47173e7c8614a506579"}
Sep 08 15:31:40.707114 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.705058    2560 generic.go:356] "Generic (PLEG): container finished" podID="d0ada965-aebc-42ee-be96-bd3daec44a7b" containerID="4966d358e008d68bf90b6ba1ccb7fe88d8715dcd59faed5165ff22f97903e7f1" exitCode=0
Sep 08 15:31:40.707114 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.705123    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn" event={"ID":"d0ada965-aebc-42ee-be96-bd3daec44a7b","Type":"ContainerDied","Data":"4966d358e008d68bf90b6ba1ccb7fe88d8715dcd59faed5165ff22f97903e7f1"}
Sep 08 15:31:40.707114 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.705160    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn" event={"ID":"d0ada965-aebc-42ee-be96-bd3daec44a7b","Type":"ContainerDied","Data":"52e36d7fb50f1e46318f1349975ccec1466187813b1eefbdf1d79025e6e28f31"}
Sep 08 15:31:40.707114 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.705226    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn"
Sep 08 15:31:40.711401 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.711379    2560 scope.go:117] "RemoveContainer" containerID="f847d4b811fdc00400da93d486f170eef9769d54dc2e07806dccbc4296b36f50"
Sep 08 15:31:40.720810 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.720794    2560 scope.go:117] "RemoveContainer" containerID="f847d4b811fdc00400da93d486f170eef9769d54dc2e07806dccbc4296b36f50"
Sep 08 15:31:40.721061 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:31:40.721045    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f847d4b811fdc00400da93d486f170eef9769d54dc2e07806dccbc4296b36f50\": container with ID starting with f847d4b811fdc00400da93d486f170eef9769d54dc2e07806dccbc4296b36f50 not found: ID does not exist" containerID="f847d4b811fdc00400da93d486f170eef9769d54dc2e07806dccbc4296b36f50"
Sep 08 15:31:40.721188 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.721169    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f847d4b811fdc00400da93d486f170eef9769d54dc2e07806dccbc4296b36f50"} err="failed to get container status \"f847d4b811fdc00400da93d486f170eef9769d54dc2e07806dccbc4296b36f50\": rpc error: code = NotFound desc = could not find container \"f847d4b811fdc00400da93d486f170eef9769d54dc2e07806dccbc4296b36f50\": container with ID starting with f847d4b811fdc00400da93d486f170eef9769d54dc2e07806dccbc4296b36f50 not found: ID does not exist"
Sep 08 15:31:40.721230 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.721190    2560 scope.go:117] "RemoveContainer" containerID="dd8270c954e4e0cc6c51c345561decddd154c34475cdb8207a2bfd140a4b38aa"
Sep 08 15:31:40.729755 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.729739    2560 scope.go:117] "RemoveContainer" containerID="dd8270c954e4e0cc6c51c345561decddd154c34475cdb8207a2bfd140a4b38aa"
Sep 08 15:31:40.730029 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:31:40.730012    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd8270c954e4e0cc6c51c345561decddd154c34475cdb8207a2bfd140a4b38aa\": container with ID starting with dd8270c954e4e0cc6c51c345561decddd154c34475cdb8207a2bfd140a4b38aa not found: ID does not exist" containerID="dd8270c954e4e0cc6c51c345561decddd154c34475cdb8207a2bfd140a4b38aa"
Sep 08 15:31:40.730088 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.730037    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd8270c954e4e0cc6c51c345561decddd154c34475cdb8207a2bfd140a4b38aa"} err="failed to get container status \"dd8270c954e4e0cc6c51c345561decddd154c34475cdb8207a2bfd140a4b38aa\": rpc error: code = NotFound desc = could not find container \"dd8270c954e4e0cc6c51c345561decddd154c34475cdb8207a2bfd140a4b38aa\": container with ID starting with dd8270c954e4e0cc6c51c345561decddd154c34475cdb8207a2bfd140a4b38aa not found: ID does not exist"
Sep 08 15:31:40.730088 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.730052    2560 scope.go:117] "RemoveContainer" containerID="4966d358e008d68bf90b6ba1ccb7fe88d8715dcd59faed5165ff22f97903e7f1"
Sep 08 15:31:40.737700 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.737685    2560 scope.go:117] "RemoveContainer" containerID="4966d358e008d68bf90b6ba1ccb7fe88d8715dcd59faed5165ff22f97903e7f1"
Sep 08 15:31:40.737930 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:31:40.737909    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4966d358e008d68bf90b6ba1ccb7fe88d8715dcd59faed5165ff22f97903e7f1\": container with ID starting with 4966d358e008d68bf90b6ba1ccb7fe88d8715dcd59faed5165ff22f97903e7f1 not found: ID does not exist" containerID="4966d358e008d68bf90b6ba1ccb7fe88d8715dcd59faed5165ff22f97903e7f1"
Sep 08 15:31:40.737982 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.737936    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4966d358e008d68bf90b6ba1ccb7fe88d8715dcd59faed5165ff22f97903e7f1"} err="failed to get container status \"4966d358e008d68bf90b6ba1ccb7fe88d8715dcd59faed5165ff22f97903e7f1\": rpc error: code = NotFound desc = could not find container \"4966d358e008d68bf90b6ba1ccb7fe88d8715dcd59faed5165ff22f97903e7f1\": container with ID starting with 4966d358e008d68bf90b6ba1ccb7fe88d8715dcd59faed5165ff22f97903e7f1 not found: ID does not exist"
Sep 08 15:31:40.788193 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.788151    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn"]
Sep 08 15:31:40.790029 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.790007    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-8debe9-564758bcbb-ng7gn"]
Sep 08 15:31:40.801906 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.801880    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:40.802148 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.802131    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx"]
Sep 08 15:31:40.813096 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.813062    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-96c53ecc-597bb9c785-rkgcx"]
Sep 08 15:31:40.840633 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.840608    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs"]
Sep 08 15:31:40.852239 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.852215    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-96c53ecc-f7cbb5878-m9qgs"]
Sep 08 15:31:40.886110 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.886079    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc"]
Sep 08 15:31:40.890396 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.890374    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8debe9-d9564576c-nznjc"]
Sep 08 15:31:40.976867 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:40.976835    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:41.247438 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:41.247359    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:41.800782 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:41.800746    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84" path="/var/lib/kubelet/pods/0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84/volumes"
Sep 08 15:31:41.801170 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:41.801156    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0aaa612-20ec-492d-a3d9-9e8bf485cb87" path="/var/lib/kubelet/pods/d0aaa612-20ec-492d-a3d9-9e8bf485cb87/volumes"
Sep 08 15:31:41.801495 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:41.801482    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0ada965-aebc-42ee-be96-bd3daec44a7b" path="/var/lib/kubelet/pods/d0ada965-aebc-42ee-be96-bd3daec44a7b/volumes"
Sep 08 15:31:41.802149 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:41.802131    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e474cbac-a9f7-4313-826a-5375b40ed02c" path="/var/lib/kubelet/pods/e474cbac-a9f7-4313-826a-5375b40ed02c/volumes"
Sep 08 15:31:42.246288 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:42.246255    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:44.026830 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:44.026781    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:44.161981 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:44.161946    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:45.282003 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.281967    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl"]
Sep 08 15:31:45.282575 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.282555    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0ada965-aebc-42ee-be96-bd3daec44a7b" containerName="payload-pre-processing"
Sep 08 15:31:45.282661 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.282577    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0ada965-aebc-42ee-be96-bd3daec44a7b" containerName="payload-pre-processing"
Sep 08 15:31:45.282661 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.282592    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e474cbac-a9f7-4313-826a-5375b40ed02c" containerName="maas-api"
Sep 08 15:31:45.282661 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.282600    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e474cbac-a9f7-4313-826a-5375b40ed02c" containerName="maas-api"
Sep 08 15:31:45.282661 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.282609    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84" containerName="payload-pre-processing"
Sep 08 15:31:45.282661 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.282618    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84" containerName="payload-pre-processing"
Sep 08 15:31:45.282661 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.282656    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0aaa612-20ec-492d-a3d9-9e8bf485cb87" containerName="maas-api"
Sep 08 15:31:45.282950 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.282665    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0aaa612-20ec-492d-a3d9-9e8bf485cb87" containerName="maas-api"
Sep 08 15:31:45.282950 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.282760    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0aaa612-20ec-492d-a3d9-9e8bf485cb87" containerName="maas-api"
Sep 08 15:31:45.282950 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.282771    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f6fb581-6eb5-4b9b-8a3e-30c4ddb76f84" containerName="payload-pre-processing"
Sep 08 15:31:45.282950 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.282783    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e474cbac-a9f7-4313-826a-5375b40ed02c" containerName="maas-api"
Sep 08 15:31:45.282950 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.282796    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0ada965-aebc-42ee-be96-bd3daec44a7b" containerName="payload-pre-processing"
Sep 08 15:31:45.287477 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.287457    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl"
Sep 08 15:31:45.289516 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.289496    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f86ebb1f-serving-cert\""
Sep 08 15:31:45.305674 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.305645    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl"]
Sep 08 15:31:45.346972 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.346934    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:45.431571 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.431534    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcm49\" (UniqueName: \"kubernetes.io/projected/d362d093-efba-49cf-a0d9-b99c32299027-kube-api-access-mcm49\") pod \"maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl\" (UID: \"d362d093-efba-49cf-a0d9-b99c32299027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl"
Sep 08 15:31:45.431571 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.431573    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d362d093-efba-49cf-a0d9-b99c32299027-maas-api-tls\") pod \"maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl\" (UID: \"d362d093-efba-49cf-a0d9-b99c32299027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl"
Sep 08 15:31:45.431768 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.431649    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d362d093-efba-49cf-a0d9-b99c32299027-metrics-certs\") pod \"maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl\" (UID: \"d362d093-efba-49cf-a0d9-b99c32299027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl"
Sep 08 15:31:45.431768 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.431708    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d362d093-efba-49cf-a0d9-b99c32299027-openshift-service-ca\") pod \"maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl\" (UID: \"d362d093-efba-49cf-a0d9-b99c32299027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl"
Sep 08 15:31:45.447464 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.447431    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:45.533228 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.533141    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcm49\" (UniqueName: \"kubernetes.io/projected/d362d093-efba-49cf-a0d9-b99c32299027-kube-api-access-mcm49\") pod \"maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl\" (UID: \"d362d093-efba-49cf-a0d9-b99c32299027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl"
Sep 08 15:31:45.533228 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.533177    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d362d093-efba-49cf-a0d9-b99c32299027-maas-api-tls\") pod \"maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl\" (UID: \"d362d093-efba-49cf-a0d9-b99c32299027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl"
Sep 08 15:31:45.533228 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.533214    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d362d093-efba-49cf-a0d9-b99c32299027-metrics-certs\") pod \"maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl\" (UID: \"d362d093-efba-49cf-a0d9-b99c32299027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl"
Sep 08 15:31:45.533452 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.533247    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d362d093-efba-49cf-a0d9-b99c32299027-openshift-service-ca\") pod \"maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl\" (UID: \"d362d093-efba-49cf-a0d9-b99c32299027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl"
Sep 08 15:31:45.533921 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.533897    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d362d093-efba-49cf-a0d9-b99c32299027-openshift-service-ca\") pod \"maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl\" (UID: \"d362d093-efba-49cf-a0d9-b99c32299027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl"
Sep 08 15:31:45.535707 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.535681    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d362d093-efba-49cf-a0d9-b99c32299027-metrics-certs\") pod \"maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl\" (UID: \"d362d093-efba-49cf-a0d9-b99c32299027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl"
Sep 08 15:31:45.535790 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.535681    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d362d093-efba-49cf-a0d9-b99c32299027-maas-api-tls\") pod \"maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl\" (UID: \"d362d093-efba-49cf-a0d9-b99c32299027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl"
Sep 08 15:31:45.544458 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.544434    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcm49\" (UniqueName: \"kubernetes.io/projected/d362d093-efba-49cf-a0d9-b99c32299027-kube-api-access-mcm49\") pod \"maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl\" (UID: \"d362d093-efba-49cf-a0d9-b99c32299027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl"
Sep 08 15:31:45.553655 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.553632    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:45.599063 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.599021    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl"
Sep 08 15:31:45.885881 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:45.885854    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl"]
Sep 08 15:31:45.888901 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:31:45.888869    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd362d093_efba_49cf_a0d9_b99c32299027.slice/crio-ded9ef94c17c42290c548a304c92f6123779b3d8b4a8a4d56e3b357319f532d2 WatchSource:0}: Error finding container ded9ef94c17c42290c548a304c92f6123779b3d8b4a8a4d56e3b357319f532d2: Status 404 returned error can't find the container with id ded9ef94c17c42290c548a304c92f6123779b3d8b4a8a4d56e3b357319f532d2
Sep 08 15:31:46.055459 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.055426    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:46.199225 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.199193    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:46.308112 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.308053    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:46.447802 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.447763    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs"]
Sep 08 15:31:46.451452 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.451396    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs"
Sep 08 15:31:46.461979 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.461957    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-f86ebb1f-dockercfg-8rnbc\""
Sep 08 15:31:46.462150 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.462014    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-f86ebb1f\""
Sep 08 15:31:46.482700 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.482660    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs"]
Sep 08 15:31:46.543338 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.542795    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnzzf\" (UniqueName: \"kubernetes.io/projected/ff0175ef-1733-4f9e-bbcc-2c2990079578-kube-api-access-hnzzf\") pod \"payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs\" (UID: \"ff0175ef-1733-4f9e-bbcc-2c2990079578\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs"
Sep 08 15:31:46.543338 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.542928    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff0175ef-1733-4f9e-bbcc-2c2990079578-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs\" (UID: \"ff0175ef-1733-4f9e-bbcc-2c2990079578\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs"
Sep 08 15:31:46.567581 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.567555    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch"]
Sep 08 15:31:46.571019 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.571003    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch"
Sep 08 15:31:46.644238 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.644194    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff0175ef-1733-4f9e-bbcc-2c2990079578-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs\" (UID: \"ff0175ef-1733-4f9e-bbcc-2c2990079578\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs"
Sep 08 15:31:46.644421 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.644287    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/040d06c3-df46-4196-8ef8-d30bf517b88b-plugins-config-volume\") pod \"payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch\" (UID: \"040d06c3-df46-4196-8ef8-d30bf517b88b\") " pod="openshift-ingress/payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch"
Sep 08 15:31:46.644421 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.644326    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8tmh\" (UniqueName: \"kubernetes.io/projected/040d06c3-df46-4196-8ef8-d30bf517b88b-kube-api-access-q8tmh\") pod \"payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch\" (UID: \"040d06c3-df46-4196-8ef8-d30bf517b88b\") " pod="openshift-ingress/payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch"
Sep 08 15:31:46.644421 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.644406    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnzzf\" (UniqueName: \"kubernetes.io/projected/ff0175ef-1733-4f9e-bbcc-2c2990079578-kube-api-access-hnzzf\") pod \"payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs\" (UID: \"ff0175ef-1733-4f9e-bbcc-2c2990079578\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs"
Sep 08 15:31:46.645395 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.645367    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff0175ef-1733-4f9e-bbcc-2c2990079578-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs\" (UID: \"ff0175ef-1733-4f9e-bbcc-2c2990079578\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs"
Sep 08 15:31:46.647684 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.647660    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch"]
Sep 08 15:31:46.660739 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.660707    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnzzf\" (UniqueName: \"kubernetes.io/projected/ff0175ef-1733-4f9e-bbcc-2c2990079578-kube-api-access-hnzzf\") pod \"payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs\" (UID: \"ff0175ef-1733-4f9e-bbcc-2c2990079578\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs"
Sep 08 15:31:46.742672 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.742572    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl" event={"ID":"d362d093-efba-49cf-a0d9-b99c32299027","Type":"ContainerStarted","Data":"e1cecbdec38a1585e0d85dfe7cdf98968166588df44e7cf64b33fc3a82159d1c"}
Sep 08 15:31:46.742672 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.742631    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl" event={"ID":"d362d093-efba-49cf-a0d9-b99c32299027","Type":"ContainerStarted","Data":"ded9ef94c17c42290c548a304c92f6123779b3d8b4a8a4d56e3b357319f532d2"}
Sep 08 15:31:46.742869 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.742788    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl"
Sep 08 15:31:46.745210 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.745189    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/040d06c3-df46-4196-8ef8-d30bf517b88b-plugins-config-volume\") pod \"payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch\" (UID: \"040d06c3-df46-4196-8ef8-d30bf517b88b\") " pod="openshift-ingress/payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch"
Sep 08 15:31:46.745275 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.745221    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8tmh\" (UniqueName: \"kubernetes.io/projected/040d06c3-df46-4196-8ef8-d30bf517b88b-kube-api-access-q8tmh\") pod \"payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch\" (UID: \"040d06c3-df46-4196-8ef8-d30bf517b88b\") " pod="openshift-ingress/payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch"
Sep 08 15:31:46.745895 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.745877    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/040d06c3-df46-4196-8ef8-d30bf517b88b-plugins-config-volume\") pod \"payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch\" (UID: \"040d06c3-df46-4196-8ef8-d30bf517b88b\") " pod="openshift-ingress/payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch"
Sep 08 15:31:46.761882 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.761861    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs"
Sep 08 15:31:46.767766 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.767739    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8tmh\" (UniqueName: \"kubernetes.io/projected/040d06c3-df46-4196-8ef8-d30bf517b88b-kube-api-access-q8tmh\") pod \"payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch\" (UID: \"040d06c3-df46-4196-8ef8-d30bf517b88b\") " pod="openshift-ingress/payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch"
Sep 08 15:31:46.780221 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.780180    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl" podStartSLOduration=1.463004986 podStartE2EDuration="1.780165074s" totalImagesPullingTime="317.160088ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:31:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:31:46.772296861 +0000 UTC m=+859.484239366" watchObservedRunningTime="2026-09-08 15:31:46.780165074 +0000 UTC m=+859.492107565"
Sep 08 15:31:46.881593 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:46.881563    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch"
Sep 08 15:31:47.059094 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:47.059000    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs"]
Sep 08 15:31:47.064772 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:31:47.064747    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff0175ef_1733_4f9e_bbcc_2c2990079578.slice/crio-5864b586ccd7b8cf9e6f2888a62391d2071b9bb4547427b82475d58cb0513c1d WatchSource:0}: Error finding container 5864b586ccd7b8cf9e6f2888a62391d2071b9bb4547427b82475d58cb0513c1d: Status 404 returned error can't find the container with id 5864b586ccd7b8cf9e6f2888a62391d2071b9bb4547427b82475d58cb0513c1d
Sep 08 15:31:47.198643 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:31:47.198612    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod040d06c3_df46_4196_8ef8_d30bf517b88b.slice/crio-648aa38813da1199027bacb4b9cfae1d91467f9a516bac7f0c4a2de028f95f21 WatchSource:0}: Error finding container 648aa38813da1199027bacb4b9cfae1d91467f9a516bac7f0c4a2de028f95f21: Status 404 returned error can't find the container with id 648aa38813da1199027bacb4b9cfae1d91467f9a516bac7f0c4a2de028f95f21
Sep 08 15:31:47.199377 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:47.199316    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch"]
Sep 08 15:31:47.750492 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:47.750456    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs" event={"ID":"ff0175ef-1733-4f9e-bbcc-2c2990079578","Type":"ContainerStarted","Data":"af5ed4eeeef79e0bf7b44044ccdafd21890b1a2200bac67d42ec58bee0035f62"}
Sep 08 15:31:47.750961 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:47.750522    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs"
Sep 08 15:31:47.750961 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:47.750546    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs" event={"ID":"ff0175ef-1733-4f9e-bbcc-2c2990079578","Type":"ContainerStarted","Data":"5864b586ccd7b8cf9e6f2888a62391d2071b9bb4547427b82475d58cb0513c1d"}
Sep 08 15:31:47.752232 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:47.752172    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch" event={"ID":"040d06c3-df46-4196-8ef8-d30bf517b88b","Type":"ContainerStarted","Data":"a42a671998605db0aeb507c14a6ada1aadc17877b7dea18a03b209cb6f29a9f6"}
Sep 08 15:31:47.752364 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:47.752253    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch" event={"ID":"040d06c3-df46-4196-8ef8-d30bf517b88b","Type":"ContainerStarted","Data":"648aa38813da1199027bacb4b9cfae1d91467f9a516bac7f0c4a2de028f95f21"}
Sep 08 15:31:47.790929 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:47.790876    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs" podStartSLOduration=1.790862062 podStartE2EDuration="1.790862062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:31:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:31:47.78841383 +0000 UTC m=+860.500356346" watchObservedRunningTime="2026-09-08 15:31:47.790862062 +0000 UTC m=+860.502804553"
Sep 08 15:31:48.757278 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:48.757246    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch"
Sep 08 15:31:51.102252 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:51.102209    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:52.758360 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:52.758333    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl"
Sep 08 15:31:52.782153 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:52.782086    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch" podStartSLOduration=6.782049539 podStartE2EDuration="6.782049539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:31:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:31:47.832968549 +0000 UTC m=+860.544911042" watchObservedRunningTime="2026-09-08 15:31:52.782049539 +0000 UTC m=+865.493992032"
Sep 08 15:31:54.755293 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:54.755256    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:54.881653 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:54.881616    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:55.005841 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:55.005742    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:55.271826 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:55.271745    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:55.641590 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:55.641458    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:55.979122 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:55.979046    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:56.742270 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:56.742223    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:58.759267 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:58.759233    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs"
Sep 08 15:31:59.412103 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:59.412051    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:59.503764 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:59.503729    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:59.764156 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:31:59.764123    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch"
Sep 08 15:32:00.633005 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:00.632961    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:00.731420 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:00.731383    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:00.830441 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:00.830396    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:01.376414 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:01.376385    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:01.470780 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:01.470747    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:01.515467 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:01.515434    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6"]
Sep 08 15:32:01.519381 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:01.519358    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6"
Sep 08 15:32:01.524752 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:01.524728    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-356350-dockercfg-7wjpb\""
Sep 08 15:32:01.524889 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:01.524815    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-356350\""
Sep 08 15:32:01.542493 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:01.542454    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6"]
Sep 08 15:32:01.687016 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:01.686981    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjxg9\" (UniqueName: \"kubernetes.io/projected/d4c1069f-5271-4154-8250-83d9572b4f83-kube-api-access-rjxg9\") pod \"payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6\" (UID: \"d4c1069f-5271-4154-8250-83d9572b4f83\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6"
Sep 08 15:32:01.687185 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:01.687028    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4c1069f-5271-4154-8250-83d9572b4f83-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6\" (UID: \"d4c1069f-5271-4154-8250-83d9572b4f83\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6"
Sep 08 15:32:01.788063 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:01.788029    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjxg9\" (UniqueName: \"kubernetes.io/projected/d4c1069f-5271-4154-8250-83d9572b4f83-kube-api-access-rjxg9\") pod \"payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6\" (UID: \"d4c1069f-5271-4154-8250-83d9572b4f83\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6"
Sep 08 15:32:01.788286 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:01.788180    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4c1069f-5271-4154-8250-83d9572b4f83-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6\" (UID: \"d4c1069f-5271-4154-8250-83d9572b4f83\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6"
Sep 08 15:32:01.788985 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:01.788960    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4c1069f-5271-4154-8250-83d9572b4f83-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6\" (UID: \"d4c1069f-5271-4154-8250-83d9572b4f83\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6"
Sep 08 15:32:01.796843 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:01.796822    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjxg9\" (UniqueName: \"kubernetes.io/projected/d4c1069f-5271-4154-8250-83d9572b4f83-kube-api-access-rjxg9\") pod \"payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6\" (UID: \"d4c1069f-5271-4154-8250-83d9572b4f83\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6"
Sep 08 15:32:01.830267 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:01.830243    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6"
Sep 08 15:32:02.108478 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:02.108404    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6"]
Sep 08 15:32:02.113368 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:32:02.113331    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4c1069f_5271_4154_8250_83d9572b4f83.slice/crio-7b295ed61a8ad3b5166e2b9f525ddeab9cb64e2f1ff6e0d34150245177904427 WatchSource:0}: Error finding container 7b295ed61a8ad3b5166e2b9f525ddeab9cb64e2f1ff6e0d34150245177904427: Status 404 returned error can't find the container with id 7b295ed61a8ad3b5166e2b9f525ddeab9cb64e2f1ff6e0d34150245177904427
Sep 08 15:32:02.831883 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:02.831842    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6" event={"ID":"d4c1069f-5271-4154-8250-83d9572b4f83","Type":"ContainerStarted","Data":"47f4f2f6d9c71c8815df006588e5acbe389559eaaf417f9191a8c89ded7dec3b"}
Sep 08 15:32:02.832130 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:02.831904    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6" event={"ID":"d4c1069f-5271-4154-8250-83d9572b4f83","Type":"ContainerStarted","Data":"7b295ed61a8ad3b5166e2b9f525ddeab9cb64e2f1ff6e0d34150245177904427"}
Sep 08 15:32:02.832130 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:02.831954    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6"
Sep 08 15:32:02.880701 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:02.880638    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6" podStartSLOduration=1.88061327 podStartE2EDuration="1.88061327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:32:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:32:02.878168507 +0000 UTC m=+875.590111001" watchObservedRunningTime="2026-09-08 15:32:02.88061327 +0000 UTC m=+875.592555762"
Sep 08 15:32:07.202642 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:07.202599    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:13.839262 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:13.839231    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6"
Sep 08 15:32:16.062993 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.062960    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl"]
Sep 08 15:32:16.063512 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.063219    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl" podUID="d362d093-efba-49cf-a0d9-b99c32299027" containerName="maas-api" containerID="cri-o://e1cecbdec38a1585e0d85dfe7cdf98968166588df44e7cf64b33fc3a82159d1c" gracePeriod=30
Sep 08 15:32:16.100432 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.100391    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:16.131375 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.131341    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch"]
Sep 08 15:32:16.131572 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.131545    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch" podUID="040d06c3-df46-4196-8ef8-d30bf517b88b" containerName="payload-processing" containerID="cri-o://a42a671998605db0aeb507c14a6ada1aadc17877b7dea18a03b209cb6f29a9f6" gracePeriod=30
Sep 08 15:32:16.137719 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.137667    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs"]
Sep 08 15:32:16.137915 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.137887    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs" podUID="ff0175ef-1733-4f9e-bbcc-2c2990079578" containerName="payload-pre-processing" containerID="cri-o://af5ed4eeeef79e0bf7b44044ccdafd21890b1a2200bac67d42ec58bee0035f62" gracePeriod=30
Sep 08 15:32:16.176650 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.176620    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:16.239375 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.239338    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:16.321181 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.321003    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:16.386202 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.386097    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:16.396144 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.396115    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl"
Sep 08 15:32:16.428188 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.428151    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d362d093-efba-49cf-a0d9-b99c32299027-metrics-certs\") pod \"d362d093-efba-49cf-a0d9-b99c32299027\" (UID: \"d362d093-efba-49cf-a0d9-b99c32299027\") "
Sep 08 15:32:16.428355 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.428206    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d362d093-efba-49cf-a0d9-b99c32299027-openshift-service-ca\") pod \"d362d093-efba-49cf-a0d9-b99c32299027\" (UID: \"d362d093-efba-49cf-a0d9-b99c32299027\") "
Sep 08 15:32:16.428355 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.428266    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mcm49\" (UniqueName: \"kubernetes.io/projected/d362d093-efba-49cf-a0d9-b99c32299027-kube-api-access-mcm49\") pod \"d362d093-efba-49cf-a0d9-b99c32299027\" (UID: \"d362d093-efba-49cf-a0d9-b99c32299027\") "
Sep 08 15:32:16.428355 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.428308    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d362d093-efba-49cf-a0d9-b99c32299027-maas-api-tls\") pod \"d362d093-efba-49cf-a0d9-b99c32299027\" (UID: \"d362d093-efba-49cf-a0d9-b99c32299027\") "
Sep 08 15:32:16.428919 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.428878    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d362d093-efba-49cf-a0d9-b99c32299027-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d362d093-efba-49cf-a0d9-b99c32299027" (UID: "d362d093-efba-49cf-a0d9-b99c32299027"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:32:16.431595 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.431560    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d362d093-efba-49cf-a0d9-b99c32299027-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d362d093-efba-49cf-a0d9-b99c32299027" (UID: "d362d093-efba-49cf-a0d9-b99c32299027"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:32:16.431779 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.431741    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d362d093-efba-49cf-a0d9-b99c32299027-kube-api-access-mcm49" (OuterVolumeSpecName: "kube-api-access-mcm49") pod "d362d093-efba-49cf-a0d9-b99c32299027" (UID: "d362d093-efba-49cf-a0d9-b99c32299027"). InnerVolumeSpecName "kube-api-access-mcm49". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:32:16.431887 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.431864    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d362d093-efba-49cf-a0d9-b99c32299027-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d362d093-efba-49cf-a0d9-b99c32299027" (UID: "d362d093-efba-49cf-a0d9-b99c32299027"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:32:16.456637 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.456614    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs"
Sep 08 15:32:16.463666 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.463642    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch"
Sep 08 15:32:16.529423 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.529383    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8tmh\" (UniqueName: \"kubernetes.io/projected/040d06c3-df46-4196-8ef8-d30bf517b88b-kube-api-access-q8tmh\") pod \"040d06c3-df46-4196-8ef8-d30bf517b88b\" (UID: \"040d06c3-df46-4196-8ef8-d30bf517b88b\") "
Sep 08 15:32:16.529634 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.529438    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hnzzf\" (UniqueName: \"kubernetes.io/projected/ff0175ef-1733-4f9e-bbcc-2c2990079578-kube-api-access-hnzzf\") pod \"ff0175ef-1733-4f9e-bbcc-2c2990079578\" (UID: \"ff0175ef-1733-4f9e-bbcc-2c2990079578\") "
Sep 08 15:32:16.529634 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.529519    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/040d06c3-df46-4196-8ef8-d30bf517b88b-plugins-config-volume\") pod \"040d06c3-df46-4196-8ef8-d30bf517b88b\" (UID: \"040d06c3-df46-4196-8ef8-d30bf517b88b\") "
Sep 08 15:32:16.529634 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.529558    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff0175ef-1733-4f9e-bbcc-2c2990079578-plugins-config-volume\") pod \"ff0175ef-1733-4f9e-bbcc-2c2990079578\" (UID: \"ff0175ef-1733-4f9e-bbcc-2c2990079578\") "
Sep 08 15:32:16.529864 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.529843    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d362d093-efba-49cf-a0d9-b99c32299027-metrics-certs\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:32:16.529923 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.529869    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d362d093-efba-49cf-a0d9-b99c32299027-openshift-service-ca\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:32:16.529923 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.529885    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcm49\" (UniqueName: \"kubernetes.io/projected/d362d093-efba-49cf-a0d9-b99c32299027-kube-api-access-mcm49\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:32:16.529923 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.529898    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d362d093-efba-49cf-a0d9-b99c32299027-maas-api-tls\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:32:16.530039 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.529969    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/040d06c3-df46-4196-8ef8-d30bf517b88b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "040d06c3-df46-4196-8ef8-d30bf517b88b" (UID: "040d06c3-df46-4196-8ef8-d30bf517b88b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:32:16.530294 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.530269    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff0175ef-1733-4f9e-bbcc-2c2990079578-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ff0175ef-1733-4f9e-bbcc-2c2990079578" (UID: "ff0175ef-1733-4f9e-bbcc-2c2990079578"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:32:16.531590 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.531567    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/040d06c3-df46-4196-8ef8-d30bf517b88b-kube-api-access-q8tmh" (OuterVolumeSpecName: "kube-api-access-q8tmh") pod "040d06c3-df46-4196-8ef8-d30bf517b88b" (UID: "040d06c3-df46-4196-8ef8-d30bf517b88b"). InnerVolumeSpecName "kube-api-access-q8tmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:32:16.531844 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.531828    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff0175ef-1733-4f9e-bbcc-2c2990079578-kube-api-access-hnzzf" (OuterVolumeSpecName: "kube-api-access-hnzzf") pod "ff0175ef-1733-4f9e-bbcc-2c2990079578" (UID: "ff0175ef-1733-4f9e-bbcc-2c2990079578"). InnerVolumeSpecName "kube-api-access-hnzzf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:32:16.630919 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.630833    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff0175ef-1733-4f9e-bbcc-2c2990079578-plugins-config-volume\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:32:16.630919 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.630862    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8tmh\" (UniqueName: \"kubernetes.io/projected/040d06c3-df46-4196-8ef8-d30bf517b88b-kube-api-access-q8tmh\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:32:16.630919 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.630872    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnzzf\" (UniqueName: \"kubernetes.io/projected/ff0175ef-1733-4f9e-bbcc-2c2990079578-kube-api-access-hnzzf\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:32:16.630919 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.630881    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/040d06c3-df46-4196-8ef8-d30bf517b88b-plugins-config-volume\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:32:16.908157 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.908064    2560 generic.go:356] "Generic (PLEG): container finished" podID="ff0175ef-1733-4f9e-bbcc-2c2990079578" containerID="af5ed4eeeef79e0bf7b44044ccdafd21890b1a2200bac67d42ec58bee0035f62" exitCode=0
Sep 08 15:32:16.908157 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.908116    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs"
Sep 08 15:32:16.908157 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.908143    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs" event={"ID":"ff0175ef-1733-4f9e-bbcc-2c2990079578","Type":"ContainerDied","Data":"af5ed4eeeef79e0bf7b44044ccdafd21890b1a2200bac67d42ec58bee0035f62"}
Sep 08 15:32:16.908404 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.908196    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs" event={"ID":"ff0175ef-1733-4f9e-bbcc-2c2990079578","Type":"ContainerDied","Data":"5864b586ccd7b8cf9e6f2888a62391d2071b9bb4547427b82475d58cb0513c1d"}
Sep 08 15:32:16.908404 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.908227    2560 scope.go:117] "RemoveContainer" containerID="af5ed4eeeef79e0bf7b44044ccdafd21890b1a2200bac67d42ec58bee0035f62"
Sep 08 15:32:16.909520 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.909487    2560 generic.go:356] "Generic (PLEG): container finished" podID="040d06c3-df46-4196-8ef8-d30bf517b88b" containerID="a42a671998605db0aeb507c14a6ada1aadc17877b7dea18a03b209cb6f29a9f6" exitCode=0
Sep 08 15:32:16.909520 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.909519    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch"
Sep 08 15:32:16.909717 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.909573    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch" event={"ID":"040d06c3-df46-4196-8ef8-d30bf517b88b","Type":"ContainerDied","Data":"a42a671998605db0aeb507c14a6ada1aadc17877b7dea18a03b209cb6f29a9f6"}
Sep 08 15:32:16.909717 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.909626    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch" event={"ID":"040d06c3-df46-4196-8ef8-d30bf517b88b","Type":"ContainerDied","Data":"648aa38813da1199027bacb4b9cfae1d91467f9a516bac7f0c4a2de028f95f21"}
Sep 08 15:32:16.911139 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.911115    2560 generic.go:356] "Generic (PLEG): container finished" podID="d362d093-efba-49cf-a0d9-b99c32299027" containerID="e1cecbdec38a1585e0d85dfe7cdf98968166588df44e7cf64b33fc3a82159d1c" exitCode=0
Sep 08 15:32:16.911248 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.911196    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl"
Sep 08 15:32:16.911312 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.911149    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl" event={"ID":"d362d093-efba-49cf-a0d9-b99c32299027","Type":"ContainerDied","Data":"e1cecbdec38a1585e0d85dfe7cdf98968166588df44e7cf64b33fc3a82159d1c"}
Sep 08 15:32:16.911380 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.911330    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl" event={"ID":"d362d093-efba-49cf-a0d9-b99c32299027","Type":"ContainerDied","Data":"ded9ef94c17c42290c548a304c92f6123779b3d8b4a8a4d56e3b357319f532d2"}
Sep 08 15:32:16.919009 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.918991    2560 scope.go:117] "RemoveContainer" containerID="af5ed4eeeef79e0bf7b44044ccdafd21890b1a2200bac67d42ec58bee0035f62"
Sep 08 15:32:16.919366 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:32:16.919345    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af5ed4eeeef79e0bf7b44044ccdafd21890b1a2200bac67d42ec58bee0035f62\": container with ID starting with af5ed4eeeef79e0bf7b44044ccdafd21890b1a2200bac67d42ec58bee0035f62 not found: ID does not exist" containerID="af5ed4eeeef79e0bf7b44044ccdafd21890b1a2200bac67d42ec58bee0035f62"
Sep 08 15:32:16.919449 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.919374    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af5ed4eeeef79e0bf7b44044ccdafd21890b1a2200bac67d42ec58bee0035f62"} err="failed to get container status \"af5ed4eeeef79e0bf7b44044ccdafd21890b1a2200bac67d42ec58bee0035f62\": rpc error: code = NotFound desc = could not find container \"af5ed4eeeef79e0bf7b44044ccdafd21890b1a2200bac67d42ec58bee0035f62\": container with ID starting with af5ed4eeeef79e0bf7b44044ccdafd21890b1a2200bac67d42ec58bee0035f62 not found: ID does not exist"
Sep 08 15:32:16.919449 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.919400    2560 scope.go:117] "RemoveContainer" containerID="a42a671998605db0aeb507c14a6ada1aadc17877b7dea18a03b209cb6f29a9f6"
Sep 08 15:32:16.929606 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.929587    2560 scope.go:117] "RemoveContainer" containerID="a42a671998605db0aeb507c14a6ada1aadc17877b7dea18a03b209cb6f29a9f6"
Sep 08 15:32:16.929858 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:32:16.929841    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a42a671998605db0aeb507c14a6ada1aadc17877b7dea18a03b209cb6f29a9f6\": container with ID starting with a42a671998605db0aeb507c14a6ada1aadc17877b7dea18a03b209cb6f29a9f6 not found: ID does not exist" containerID="a42a671998605db0aeb507c14a6ada1aadc17877b7dea18a03b209cb6f29a9f6"
Sep 08 15:32:16.929913 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.929863    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a42a671998605db0aeb507c14a6ada1aadc17877b7dea18a03b209cb6f29a9f6"} err="failed to get container status \"a42a671998605db0aeb507c14a6ada1aadc17877b7dea18a03b209cb6f29a9f6\": rpc error: code = NotFound desc = could not find container \"a42a671998605db0aeb507c14a6ada1aadc17877b7dea18a03b209cb6f29a9f6\": container with ID starting with a42a671998605db0aeb507c14a6ada1aadc17877b7dea18a03b209cb6f29a9f6 not found: ID does not exist"
Sep 08 15:32:16.929913 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.929877    2560 scope.go:117] "RemoveContainer" containerID="e1cecbdec38a1585e0d85dfe7cdf98968166588df44e7cf64b33fc3a82159d1c"
Sep 08 15:32:16.940137 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.936560    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs"]
Sep 08 15:32:16.944766 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.944743    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f86ebb1f-7bf5769dd6-fcnbs"]
Sep 08 15:32:16.945996 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.945973    2560 scope.go:117] "RemoveContainer" containerID="e1cecbdec38a1585e0d85dfe7cdf98968166588df44e7cf64b33fc3a82159d1c"
Sep 08 15:32:16.946365 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:32:16.946337    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1cecbdec38a1585e0d85dfe7cdf98968166588df44e7cf64b33fc3a82159d1c\": container with ID starting with e1cecbdec38a1585e0d85dfe7cdf98968166588df44e7cf64b33fc3a82159d1c not found: ID does not exist" containerID="e1cecbdec38a1585e0d85dfe7cdf98968166588df44e7cf64b33fc3a82159d1c"
Sep 08 15:32:16.946449 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.946374    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1cecbdec38a1585e0d85dfe7cdf98968166588df44e7cf64b33fc3a82159d1c"} err="failed to get container status \"e1cecbdec38a1585e0d85dfe7cdf98968166588df44e7cf64b33fc3a82159d1c\": rpc error: code = NotFound desc = could not find container \"e1cecbdec38a1585e0d85dfe7cdf98968166588df44e7cf64b33fc3a82159d1c\": container with ID starting with e1cecbdec38a1585e0d85dfe7cdf98968166588df44e7cf64b33fc3a82159d1c not found: ID does not exist"
Sep 08 15:32:16.955871 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.955843    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch"]
Sep 08 15:32:16.960210 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.960191    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f86ebb1f-57dd4779ff-xzzch"]
Sep 08 15:32:16.970649 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.970628    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl"]
Sep 08 15:32:16.972823 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:16.972806    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f86ebb1f-5c56bff6b5-cspjl"]
Sep 08 15:32:17.534817 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:17.534779    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:17.800465 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:17.800383    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="040d06c3-df46-4196-8ef8-d30bf517b88b" path="/var/lib/kubelet/pods/040d06c3-df46-4196-8ef8-d30bf517b88b/volumes"
Sep 08 15:32:17.800777 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:17.800763    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d362d093-efba-49cf-a0d9-b99c32299027" path="/var/lib/kubelet/pods/d362d093-efba-49cf-a0d9-b99c32299027/volumes"
Sep 08 15:32:17.801092 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:17.801062    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff0175ef-1733-4f9e-bbcc-2c2990079578" path="/var/lib/kubelet/pods/ff0175ef-1733-4f9e-bbcc-2c2990079578/volumes"
Sep 08 15:32:17.975757 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:17.975722    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:18.272440 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:18.272409    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:18.419025 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:18.418992    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:20.889462 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:20.889382    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"]
Sep 08 15:32:20.889848 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:20.889830    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d362d093-efba-49cf-a0d9-b99c32299027" containerName="maas-api"
Sep 08 15:32:20.889848 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:20.889850    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d362d093-efba-49cf-a0d9-b99c32299027" containerName="maas-api"
Sep 08 15:32:20.889935 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:20.889861    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff0175ef-1733-4f9e-bbcc-2c2990079578" containerName="payload-pre-processing"
Sep 08 15:32:20.889935 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:20.889867    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff0175ef-1733-4f9e-bbcc-2c2990079578" containerName="payload-pre-processing"
Sep 08 15:32:20.889935 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:20.889876    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="040d06c3-df46-4196-8ef8-d30bf517b88b" containerName="payload-processing"
Sep 08 15:32:20.889935 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:20.889882    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="040d06c3-df46-4196-8ef8-d30bf517b88b" containerName="payload-processing"
Sep 08 15:32:20.890058 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:20.889944    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff0175ef-1733-4f9e-bbcc-2c2990079578" containerName="payload-pre-processing"
Sep 08 15:32:20.890058 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:20.889955    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d362d093-efba-49cf-a0d9-b99c32299027" containerName="maas-api"
Sep 08 15:32:20.890058 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:20.889960    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="040d06c3-df46-4196-8ef8-d30bf517b88b" containerName="payload-processing"
Sep 08 15:32:20.894745 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:20.894723    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"
Sep 08 15:32:20.896783 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:20.896762    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-34fed8\"/\"kube-root-ca.crt\""
Sep 08 15:32:20.896886 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:20.896845    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-34fed8\"/\"default-dockercfg-rk49s\""
Sep 08 15:32:20.897284 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:20.897252    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-34fed8\"/\"openshift-service-ca.crt\""
Sep 08 15:32:20.897580 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:20.897560    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-34fed8\"/\"test-model-34fed8-kserve-self-signed-certs\""
Sep 08 15:32:20.905694 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:20.905670    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"]
Sep 08 15:32:20.975151 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:20.975119    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k48q7\" (UniqueName: \"kubernetes.io/projected/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-kube-api-access-k48q7\") pod \"test-model-34fed8-kserve-99475969c-rgmc4\" (UID: \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\") " pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"
Sep 08 15:32:20.975304 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:20.975174    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-home\") pod \"test-model-34fed8-kserve-99475969c-rgmc4\" (UID: \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\") " pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"
Sep 08 15:32:20.975304 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:20.975210    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-dshm\") pod \"test-model-34fed8-kserve-99475969c-rgmc4\" (UID: \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\") " pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"
Sep 08 15:32:20.975304 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:20.975292    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-model-cache\") pod \"test-model-34fed8-kserve-99475969c-rgmc4\" (UID: \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\") " pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"
Sep 08 15:32:20.975430 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:20.975318    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-tls-certs\") pod \"test-model-34fed8-kserve-99475969c-rgmc4\" (UID: \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\") " pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"
Sep 08 15:32:20.975430 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:20.975345    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-tmp-dir\") pod \"test-model-34fed8-kserve-99475969c-rgmc4\" (UID: \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\") " pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"
Sep 08 15:32:21.075435 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:21.075405    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:21.075793 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:21.075774    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k48q7\" (UniqueName: \"kubernetes.io/projected/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-kube-api-access-k48q7\") pod \"test-model-34fed8-kserve-99475969c-rgmc4\" (UID: \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\") " pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"
Sep 08 15:32:21.075845 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:21.075818    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-home\") pod \"test-model-34fed8-kserve-99475969c-rgmc4\" (UID: \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\") " pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"
Sep 08 15:32:21.076005 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:21.075984    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-dshm\") pod \"test-model-34fed8-kserve-99475969c-rgmc4\" (UID: \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\") " pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"
Sep 08 15:32:21.076058 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:21.076033    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-model-cache\") pod \"test-model-34fed8-kserve-99475969c-rgmc4\" (UID: \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\") " pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"
Sep 08 15:32:21.076146 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:21.076133    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-tls-certs\") pod \"test-model-34fed8-kserve-99475969c-rgmc4\" (UID: \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\") " pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"
Sep 08 15:32:21.076204 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:21.076164    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-tmp-dir\") pod \"test-model-34fed8-kserve-99475969c-rgmc4\" (UID: \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\") " pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"
Sep 08 15:32:21.076262 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:21.076163    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-home\") pod \"test-model-34fed8-kserve-99475969c-rgmc4\" (UID: \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\") " pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"
Sep 08 15:32:21.076392 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:21.076372    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-model-cache\") pod \"test-model-34fed8-kserve-99475969c-rgmc4\" (UID: \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\") " pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"
Sep 08 15:32:21.076448 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:21.076413    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-tmp-dir\") pod \"test-model-34fed8-kserve-99475969c-rgmc4\" (UID: \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\") " pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"
Sep 08 15:32:21.078152 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:21.078120    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-dshm\") pod \"test-model-34fed8-kserve-99475969c-rgmc4\" (UID: \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\") " pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"
Sep 08 15:32:21.078385 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:21.078369    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-tls-certs\") pod \"test-model-34fed8-kserve-99475969c-rgmc4\" (UID: \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\") " pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"
Sep 08 15:32:21.091746 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:21.091725    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k48q7\" (UniqueName: \"kubernetes.io/projected/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-kube-api-access-k48q7\") pod \"test-model-34fed8-kserve-99475969c-rgmc4\" (UID: \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\") " pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"
Sep 08 15:32:21.205086 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:21.205040    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"
Sep 08 15:32:21.290797 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:21.290708    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:21.393731 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:21.393691    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:21.481729 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:21.481706    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"]
Sep 08 15:32:21.483303 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:32:21.483279    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6a2c7454_f621_4bf4_ad4c_c14494fcf0d1.slice/crio-d9f3693287c440ad41f3ad4814715b042f8886bbab7b9a45996a79bb71f2e12f WatchSource:0}: Error finding container d9f3693287c440ad41f3ad4814715b042f8886bbab7b9a45996a79bb71f2e12f: Status 404 returned error can't find the container with id d9f3693287c440ad41f3ad4814715b042f8886bbab7b9a45996a79bb71f2e12f
Sep 08 15:32:21.489050 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:21.489026    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:21.940436 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:21.940396    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4" event={"ID":"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1","Type":"ContainerStarted","Data":"a99498269debfdc17ae046a9786a7cc3af0a7cbd429c032545f2f94fd0cd32d0"}
Sep 08 15:32:21.940898 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:21.940447    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4" event={"ID":"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1","Type":"ContainerStarted","Data":"d9f3693287c440ad41f3ad4814715b042f8886bbab7b9a45996a79bb71f2e12f"}
Sep 08 15:32:21.959784 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:21.959729    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4" podStartSLOduration=1.959713674 podStartE2EDuration="1.959713674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:32:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:32:21.95743243 +0000 UTC m=+894.669374921" watchObservedRunningTime="2026-09-08 15:32:21.959713674 +0000 UTC m=+894.671656196"
Sep 08 15:32:22.498303 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:22.498265    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:23.319936 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:23.319898    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m"]
Sep 08 15:32:23.323450 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:23.323425    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m"
Sep 08 15:32:23.325244 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:23.325222    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-4f2c6181-serving-cert\""
Sep 08 15:32:23.334143 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:23.334120    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m"]
Sep 08 15:32:23.345709 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:23.345679    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:23.396095 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:23.396047    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j487d\" (UniqueName: \"kubernetes.io/projected/61980147-693b-4b36-8585-42b56e61b1ef-kube-api-access-j487d\") pod \"maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m\" (UID: \"61980147-693b-4b36-8585-42b56e61b1ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m"
Sep 08 15:32:23.396248 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:23.396186    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61980147-693b-4b36-8585-42b56e61b1ef-openshift-service-ca\") pod \"maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m\" (UID: \"61980147-693b-4b36-8585-42b56e61b1ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m"
Sep 08 15:32:23.396308 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:23.396247    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61980147-693b-4b36-8585-42b56e61b1ef-metrics-certs\") pod \"maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m\" (UID: \"61980147-693b-4b36-8585-42b56e61b1ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m"
Sep 08 15:32:23.396347 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:23.396328    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61980147-693b-4b36-8585-42b56e61b1ef-maas-api-tls\") pod \"maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m\" (UID: \"61980147-693b-4b36-8585-42b56e61b1ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m"
Sep 08 15:32:23.438452 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:23.438407    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:23.497724 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:23.497694    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j487d\" (UniqueName: \"kubernetes.io/projected/61980147-693b-4b36-8585-42b56e61b1ef-kube-api-access-j487d\") pod \"maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m\" (UID: \"61980147-693b-4b36-8585-42b56e61b1ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m"
Sep 08 15:32:23.497882 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:23.497750    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61980147-693b-4b36-8585-42b56e61b1ef-openshift-service-ca\") pod \"maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m\" (UID: \"61980147-693b-4b36-8585-42b56e61b1ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m"
Sep 08 15:32:23.497882 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:23.497788    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61980147-693b-4b36-8585-42b56e61b1ef-metrics-certs\") pod \"maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m\" (UID: \"61980147-693b-4b36-8585-42b56e61b1ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m"
Sep 08 15:32:23.497882 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:23.497814    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61980147-693b-4b36-8585-42b56e61b1ef-maas-api-tls\") pod \"maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m\" (UID: \"61980147-693b-4b36-8585-42b56e61b1ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m"
Sep 08 15:32:23.498010 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:32:23.497940    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-serving-cert: secret "maas-api-e2e-derive-4f2c6181-serving-cert" not found
Sep 08 15:32:23.498010 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:32:23.498004    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61980147-693b-4b36-8585-42b56e61b1ef-maas-api-tls podName:61980147-693b-4b36-8585-42b56e61b1ef nodeName:}" failed. No retries permitted until 2026-09-08 15:32:23.997984954 +0000 UTC m=+896.709927443 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/61980147-693b-4b36-8585-42b56e61b1ef-maas-api-tls") pod "maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m" (UID: "61980147-693b-4b36-8585-42b56e61b1ef") : secret "maas-api-e2e-derive-4f2c6181-serving-cert" not found
Sep 08 15:32:23.498501 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:23.498482    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61980147-693b-4b36-8585-42b56e61b1ef-openshift-service-ca\") pod \"maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m\" (UID: \"61980147-693b-4b36-8585-42b56e61b1ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m"
Sep 08 15:32:23.500449 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:23.500428    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61980147-693b-4b36-8585-42b56e61b1ef-metrics-certs\") pod \"maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m\" (UID: \"61980147-693b-4b36-8585-42b56e61b1ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m"
Sep 08 15:32:23.507200 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:23.507176    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j487d\" (UniqueName: \"kubernetes.io/projected/61980147-693b-4b36-8585-42b56e61b1ef-kube-api-access-j487d\") pod \"maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m\" (UID: \"61980147-693b-4b36-8585-42b56e61b1ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m"
Sep 08 15:32:23.677869 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:23.677832    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:23.935369 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:23.935294    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:24.001734 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.001703    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61980147-693b-4b36-8585-42b56e61b1ef-maas-api-tls\") pod \"maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m\" (UID: \"61980147-693b-4b36-8585-42b56e61b1ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m"
Sep 08 15:32:24.004169 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.004136    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61980147-693b-4b36-8585-42b56e61b1ef-maas-api-tls\") pod \"maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m\" (UID: \"61980147-693b-4b36-8585-42b56e61b1ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m"
Sep 08 15:32:24.056137 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.056033    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:24.158132 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.158099    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm"]
Sep 08 15:32:24.161863 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.161844    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm"
Sep 08 15:32:24.164164 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.164132    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-4f2c6181\""
Sep 08 15:32:24.164276 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.164183    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-4f2c6181-dockercfg-7b87x\""
Sep 08 15:32:24.174174 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.174151    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm"]
Sep 08 15:32:24.207851 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.207789    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76"]
Sep 08 15:32:24.211262 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.211245    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76"
Sep 08 15:32:24.222405 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.222383    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76"]
Sep 08 15:32:24.235545 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.235521    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m"
Sep 08 15:32:24.306450 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.306375    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6n6vh\" (UniqueName: \"kubernetes.io/projected/5abc56a1-dc9d-4216-b03f-aea6efad8417-kube-api-access-6n6vh\") pod \"payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm\" (UID: \"5abc56a1-dc9d-4216-b03f-aea6efad8417\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm"
Sep 08 15:32:24.306450 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.306441    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqfrp\" (UniqueName: \"kubernetes.io/projected/d809fbae-e8d0-4c0e-8f91-8f67df0b6498-kube-api-access-rqfrp\") pod \"payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76\" (UID: \"d809fbae-e8d0-4c0e-8f91-8f67df0b6498\") " pod="openshift-ingress/payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76"
Sep 08 15:32:24.306650 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.306545    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d809fbae-e8d0-4c0e-8f91-8f67df0b6498-plugins-config-volume\") pod \"payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76\" (UID: \"d809fbae-e8d0-4c0e-8f91-8f67df0b6498\") " pod="openshift-ingress/payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76"
Sep 08 15:32:24.306650 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.306582    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5abc56a1-dc9d-4216-b03f-aea6efad8417-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm\" (UID: \"5abc56a1-dc9d-4216-b03f-aea6efad8417\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm"
Sep 08 15:32:24.407038 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.407006    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d809fbae-e8d0-4c0e-8f91-8f67df0b6498-plugins-config-volume\") pod \"payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76\" (UID: \"d809fbae-e8d0-4c0e-8f91-8f67df0b6498\") " pod="openshift-ingress/payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76"
Sep 08 15:32:24.407038 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.407042    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5abc56a1-dc9d-4216-b03f-aea6efad8417-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm\" (UID: \"5abc56a1-dc9d-4216-b03f-aea6efad8417\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm"
Sep 08 15:32:24.407575 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.407131    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6n6vh\" (UniqueName: \"kubernetes.io/projected/5abc56a1-dc9d-4216-b03f-aea6efad8417-kube-api-access-6n6vh\") pod \"payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm\" (UID: \"5abc56a1-dc9d-4216-b03f-aea6efad8417\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm"
Sep 08 15:32:24.407575 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.407161    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqfrp\" (UniqueName: \"kubernetes.io/projected/d809fbae-e8d0-4c0e-8f91-8f67df0b6498-kube-api-access-rqfrp\") pod \"payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76\" (UID: \"d809fbae-e8d0-4c0e-8f91-8f67df0b6498\") " pod="openshift-ingress/payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76"
Sep 08 15:32:24.407842 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.407819    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5abc56a1-dc9d-4216-b03f-aea6efad8417-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm\" (UID: \"5abc56a1-dc9d-4216-b03f-aea6efad8417\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm"
Sep 08 15:32:24.407913 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.407875    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d809fbae-e8d0-4c0e-8f91-8f67df0b6498-plugins-config-volume\") pod \"payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76\" (UID: \"d809fbae-e8d0-4c0e-8f91-8f67df0b6498\") " pod="openshift-ingress/payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76"
Sep 08 15:32:24.421930 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.421905    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqfrp\" (UniqueName: \"kubernetes.io/projected/d809fbae-e8d0-4c0e-8f91-8f67df0b6498-kube-api-access-rqfrp\") pod \"payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76\" (UID: \"d809fbae-e8d0-4c0e-8f91-8f67df0b6498\") " pod="openshift-ingress/payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76"
Sep 08 15:32:24.422060 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.422005    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6n6vh\" (UniqueName: \"kubernetes.io/projected/5abc56a1-dc9d-4216-b03f-aea6efad8417-kube-api-access-6n6vh\") pod \"payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm\" (UID: \"5abc56a1-dc9d-4216-b03f-aea6efad8417\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm"
Sep 08 15:32:24.473907 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.473832    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm"
Sep 08 15:32:24.497699 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.497671    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:24.522166 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.522135    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m"]
Sep 08 15:32:24.522351 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.522192    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76"
Sep 08 15:32:24.527257 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:32:24.526978    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61980147_693b_4b36_8585_42b56e61b1ef.slice/crio-b84333225c6a827025e40fff5e948fff2c32c36017bad8641bd5dd364252ad32 WatchSource:0}: Error finding container b84333225c6a827025e40fff5e948fff2c32c36017bad8641bd5dd364252ad32: Status 404 returned error can't find the container with id b84333225c6a827025e40fff5e948fff2c32c36017bad8641bd5dd364252ad32
Sep 08 15:32:24.691879 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.691825    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:24.765545 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.765518    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm"]
Sep 08 15:32:24.766589 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:32:24.766560    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5abc56a1_dc9d_4216_b03f_aea6efad8417.slice/crio-1d161a63bc55b2fed0ed658182b5057c2b51ad4f6fb408e9b9e01716a28b7163 WatchSource:0}: Error finding container 1d161a63bc55b2fed0ed658182b5057c2b51ad4f6fb408e9b9e01716a28b7163: Status 404 returned error can't find the container with id 1d161a63bc55b2fed0ed658182b5057c2b51ad4f6fb408e9b9e01716a28b7163
Sep 08 15:32:24.818694 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.818672    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76"]
Sep 08 15:32:24.820867 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:32:24.820842    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd809fbae_e8d0_4c0e_8f91_8f67df0b6498.slice/crio-65945423ff0a61fc4c6cfd319f72ce3f4491faebf7d44b976bb5303ad0fe2615 WatchSource:0}: Error finding container 65945423ff0a61fc4c6cfd319f72ce3f4491faebf7d44b976bb5303ad0fe2615: Status 404 returned error can't find the container with id 65945423ff0a61fc4c6cfd319f72ce3f4491faebf7d44b976bb5303ad0fe2615
Sep 08 15:32:24.962783 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.962749    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76" event={"ID":"d809fbae-e8d0-4c0e-8f91-8f67df0b6498","Type":"ContainerStarted","Data":"3b8cc854bfb827dd0018514c5f4f1ca39fb68dde30695ea6f591108dd8b75e0d"}
Sep 08 15:32:24.962968 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.962811    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76" event={"ID":"d809fbae-e8d0-4c0e-8f91-8f67df0b6498","Type":"ContainerStarted","Data":"65945423ff0a61fc4c6cfd319f72ce3f4491faebf7d44b976bb5303ad0fe2615"}
Sep 08 15:32:24.962968 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.962845    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76"
Sep 08 15:32:24.964343 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.964308    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m" event={"ID":"61980147-693b-4b36-8585-42b56e61b1ef","Type":"ContainerStarted","Data":"b84333225c6a827025e40fff5e948fff2c32c36017bad8641bd5dd364252ad32"}
Sep 08 15:32:24.966097 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.966048    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm" event={"ID":"5abc56a1-dc9d-4216-b03f-aea6efad8417","Type":"ContainerStarted","Data":"3edcc0eaa94de7b647b1057443c33a12adf294600a21583b328089ef2e2c02b4"}
Sep 08 15:32:24.966198 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.966116    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm"
Sep 08 15:32:24.966198 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.966145    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm" event={"ID":"5abc56a1-dc9d-4216-b03f-aea6efad8417","Type":"ContainerStarted","Data":"1d161a63bc55b2fed0ed658182b5057c2b51ad4f6fb408e9b9e01716a28b7163"}
Sep 08 15:32:24.987767 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:24.987708    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76" podStartSLOduration=0.987687477 podStartE2EDuration="987.687477ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:32:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:32:24.985541475 +0000 UTC m=+897.697483981" watchObservedRunningTime="2026-09-08 15:32:24.987687477 +0000 UTC m=+897.699629968"
Sep 08 15:32:25.011193 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:25.011145    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm" podStartSLOduration=1.011127821 podStartE2EDuration="1.011127821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:32:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:32:25.009644899 +0000 UTC m=+897.721587415" watchObservedRunningTime="2026-09-08 15:32:25.011127821 +0000 UTC m=+897.723070312"
Sep 08 15:32:25.715957 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:25.715925    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:25.974125 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:25.974018    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m" event={"ID":"61980147-693b-4b36-8585-42b56e61b1ef","Type":"ContainerStarted","Data":"4eb9b5594c2206079f4711364a24f51fde8662ad30a77ee1f58d2767474c6f0a"}
Sep 08 15:32:25.974367 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:25.974346    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m"
Sep 08 15:32:26.007424 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:26.007375    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m" podStartSLOduration=2.5598707259999998 podStartE2EDuration="3.007361934s" totalImagesPullingTime="447.491208ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:32:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:32:26.00425312 +0000 UTC m=+898.716195613" watchObservedRunningTime="2026-09-08 15:32:26.007361934 +0000 UTC m=+898.719304469"
Sep 08 15:32:27.725733 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:27.725693    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vc5n9_17e30810-dfd0-4e6f-9204-731d1a6479a3/console-operator/2.log"
Sep 08 15:32:27.727043 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:27.727014    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vc5n9_17e30810-dfd0-4e6f-9204-731d1a6479a3/console-operator/2.log"
Sep 08 15:32:30.715094 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:30.715033    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:31.205959 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:31.205920    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"
Sep 08 15:32:31.206160 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:31.205974    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"
Sep 08 15:32:31.218415 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:31.218391    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"
Sep 08 15:32:31.775802 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:31.775760    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:31.984809 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:31.984779    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m"
Sep 08 15:32:32.015494 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:32.015453    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"
Sep 08 15:32:35.975932 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:35.975891    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm"
Sep 08 15:32:35.978042 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:35.976008    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76"
Sep 08 15:32:52.926347 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:52.926294    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m"]
Sep 08 15:32:52.928215 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:52.927847    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m" podUID="61980147-693b-4b36-8585-42b56e61b1ef" containerName="maas-api" containerID="cri-o://4eb9b5594c2206079f4711364a24f51fde8662ad30a77ee1f58d2767474c6f0a" gracePeriod=30
Sep 08 15:32:52.955778 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:52.955749    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:52.990383 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:52.990348    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76"]
Sep 08 15:32:52.990723 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:52.990587    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76" podUID="d809fbae-e8d0-4c0e-8f91-8f67df0b6498" containerName="payload-processing" containerID="cri-o://3b8cc854bfb827dd0018514c5f4f1ca39fb68dde30695ea6f591108dd8b75e0d" gracePeriod=30
Sep 08 15:32:53.000443 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.000389    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm"]
Sep 08 15:32:53.000702 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.000672    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm" podUID="5abc56a1-dc9d-4216-b03f-aea6efad8417" containerName="payload-pre-processing" containerID="cri-o://3edcc0eaa94de7b647b1057443c33a12adf294600a21583b328089ef2e2c02b4" gracePeriod=30
Sep 08 15:32:53.114284 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.114248    2560 generic.go:356] "Generic (PLEG): container finished" podID="d809fbae-e8d0-4c0e-8f91-8f67df0b6498" containerID="3b8cc854bfb827dd0018514c5f4f1ca39fb68dde30695ea6f591108dd8b75e0d" exitCode=0
Sep 08 15:32:53.114429 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.114321    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76" event={"ID":"d809fbae-e8d0-4c0e-8f91-8f67df0b6498","Type":"ContainerDied","Data":"3b8cc854bfb827dd0018514c5f4f1ca39fb68dde30695ea6f591108dd8b75e0d"}
Sep 08 15:32:53.116266 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.116230    2560 generic.go:356] "Generic (PLEG): container finished" podID="61980147-693b-4b36-8585-42b56e61b1ef" containerID="4eb9b5594c2206079f4711364a24f51fde8662ad30a77ee1f58d2767474c6f0a" exitCode=0
Sep 08 15:32:53.116382 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.116307    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m" event={"ID":"61980147-693b-4b36-8585-42b56e61b1ef","Type":"ContainerDied","Data":"4eb9b5594c2206079f4711364a24f51fde8662ad30a77ee1f58d2767474c6f0a"}
Sep 08 15:32:53.278091 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.276911    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m"
Sep 08 15:32:53.329051 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.329023    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76"
Sep 08 15:32:53.366232 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.366129    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm"
Sep 08 15:32:53.369106 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.369064    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j487d\" (UniqueName: \"kubernetes.io/projected/61980147-693b-4b36-8585-42b56e61b1ef-kube-api-access-j487d\") pod \"61980147-693b-4b36-8585-42b56e61b1ef\" (UID: \"61980147-693b-4b36-8585-42b56e61b1ef\") "
Sep 08 15:32:53.369264 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.369136    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61980147-693b-4b36-8585-42b56e61b1ef-openshift-service-ca\") pod \"61980147-693b-4b36-8585-42b56e61b1ef\" (UID: \"61980147-693b-4b36-8585-42b56e61b1ef\") "
Sep 08 15:32:53.369264 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.369186    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rqfrp\" (UniqueName: \"kubernetes.io/projected/d809fbae-e8d0-4c0e-8f91-8f67df0b6498-kube-api-access-rqfrp\") pod \"d809fbae-e8d0-4c0e-8f91-8f67df0b6498\" (UID: \"d809fbae-e8d0-4c0e-8f91-8f67df0b6498\") "
Sep 08 15:32:53.369264 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.369233    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61980147-693b-4b36-8585-42b56e61b1ef-maas-api-tls\") pod \"61980147-693b-4b36-8585-42b56e61b1ef\" (UID: \"61980147-693b-4b36-8585-42b56e61b1ef\") "
Sep 08 15:32:53.369419 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.369261    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d809fbae-e8d0-4c0e-8f91-8f67df0b6498-plugins-config-volume\") pod \"d809fbae-e8d0-4c0e-8f91-8f67df0b6498\" (UID: \"d809fbae-e8d0-4c0e-8f91-8f67df0b6498\") "
Sep 08 15:32:53.369419 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.369318    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61980147-693b-4b36-8585-42b56e61b1ef-metrics-certs\") pod \"61980147-693b-4b36-8585-42b56e61b1ef\" (UID: \"61980147-693b-4b36-8585-42b56e61b1ef\") "
Sep 08 15:32:53.369519 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.369496    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61980147-693b-4b36-8585-42b56e61b1ef-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "61980147-693b-4b36-8585-42b56e61b1ef" (UID: "61980147-693b-4b36-8585-42b56e61b1ef"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:32:53.369661 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.369629    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61980147-693b-4b36-8585-42b56e61b1ef-openshift-service-ca\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:32:53.369799 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.369761    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d809fbae-e8d0-4c0e-8f91-8f67df0b6498-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d809fbae-e8d0-4c0e-8f91-8f67df0b6498" (UID: "d809fbae-e8d0-4c0e-8f91-8f67df0b6498"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:32:53.371808 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.371756    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61980147-693b-4b36-8585-42b56e61b1ef-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "61980147-693b-4b36-8585-42b56e61b1ef" (UID: "61980147-693b-4b36-8585-42b56e61b1ef"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:32:53.371912 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.371848    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61980147-693b-4b36-8585-42b56e61b1ef-kube-api-access-j487d" (OuterVolumeSpecName: "kube-api-access-j487d") pod "61980147-693b-4b36-8585-42b56e61b1ef" (UID: "61980147-693b-4b36-8585-42b56e61b1ef"). InnerVolumeSpecName "kube-api-access-j487d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:32:53.371912 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.371857    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d809fbae-e8d0-4c0e-8f91-8f67df0b6498-kube-api-access-rqfrp" (OuterVolumeSpecName: "kube-api-access-rqfrp") pod "d809fbae-e8d0-4c0e-8f91-8f67df0b6498" (UID: "d809fbae-e8d0-4c0e-8f91-8f67df0b6498"). InnerVolumeSpecName "kube-api-access-rqfrp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:32:53.372054 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.372036    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61980147-693b-4b36-8585-42b56e61b1ef-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "61980147-693b-4b36-8585-42b56e61b1ef" (UID: "61980147-693b-4b36-8585-42b56e61b1ef"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:32:53.470103 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.470004    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6n6vh\" (UniqueName: \"kubernetes.io/projected/5abc56a1-dc9d-4216-b03f-aea6efad8417-kube-api-access-6n6vh\") pod \"5abc56a1-dc9d-4216-b03f-aea6efad8417\" (UID: \"5abc56a1-dc9d-4216-b03f-aea6efad8417\") "
Sep 08 15:32:53.470103 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.470081    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5abc56a1-dc9d-4216-b03f-aea6efad8417-plugins-config-volume\") pod \"5abc56a1-dc9d-4216-b03f-aea6efad8417\" (UID: \"5abc56a1-dc9d-4216-b03f-aea6efad8417\") "
Sep 08 15:32:53.470365 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.470348    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j487d\" (UniqueName: \"kubernetes.io/projected/61980147-693b-4b36-8585-42b56e61b1ef-kube-api-access-j487d\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:32:53.470432 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.470370    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rqfrp\" (UniqueName: \"kubernetes.io/projected/d809fbae-e8d0-4c0e-8f91-8f67df0b6498-kube-api-access-rqfrp\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:32:53.470432 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.470385    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61980147-693b-4b36-8585-42b56e61b1ef-maas-api-tls\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:32:53.470432 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.470399    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d809fbae-e8d0-4c0e-8f91-8f67df0b6498-plugins-config-volume\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:32:53.470432 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.470414    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61980147-693b-4b36-8585-42b56e61b1ef-metrics-certs\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:32:53.470576 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.470443    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5abc56a1-dc9d-4216-b03f-aea6efad8417-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5abc56a1-dc9d-4216-b03f-aea6efad8417" (UID: "5abc56a1-dc9d-4216-b03f-aea6efad8417"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:32:53.472106 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.472084    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5abc56a1-dc9d-4216-b03f-aea6efad8417-kube-api-access-6n6vh" (OuterVolumeSpecName: "kube-api-access-6n6vh") pod "5abc56a1-dc9d-4216-b03f-aea6efad8417" (UID: "5abc56a1-dc9d-4216-b03f-aea6efad8417"). InnerVolumeSpecName "kube-api-access-6n6vh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:32:53.571656 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.571619    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6n6vh\" (UniqueName: \"kubernetes.io/projected/5abc56a1-dc9d-4216-b03f-aea6efad8417-kube-api-access-6n6vh\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:32:53.571656 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:53.571649    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5abc56a1-dc9d-4216-b03f-aea6efad8417-plugins-config-volume\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:32:54.123598 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:54.123518    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76"
Sep 08 15:32:54.124024 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:54.123521    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76" event={"ID":"d809fbae-e8d0-4c0e-8f91-8f67df0b6498","Type":"ContainerDied","Data":"65945423ff0a61fc4c6cfd319f72ce3f4491faebf7d44b976bb5303ad0fe2615"}
Sep 08 15:32:54.124024 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:54.123650    2560 scope.go:117] "RemoveContainer" containerID="3b8cc854bfb827dd0018514c5f4f1ca39fb68dde30695ea6f591108dd8b75e0d"
Sep 08 15:32:54.125139 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:54.125125    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m"
Sep 08 15:32:54.125236 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:54.125128    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m" event={"ID":"61980147-693b-4b36-8585-42b56e61b1ef","Type":"ContainerDied","Data":"b84333225c6a827025e40fff5e948fff2c32c36017bad8641bd5dd364252ad32"}
Sep 08 15:32:54.126739 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:54.126716    2560 generic.go:356] "Generic (PLEG): container finished" podID="5abc56a1-dc9d-4216-b03f-aea6efad8417" containerID="3edcc0eaa94de7b647b1057443c33a12adf294600a21583b328089ef2e2c02b4" exitCode=0
Sep 08 15:32:54.126846 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:54.126760    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm"
Sep 08 15:32:54.126846 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:54.126810    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm" event={"ID":"5abc56a1-dc9d-4216-b03f-aea6efad8417","Type":"ContainerDied","Data":"3edcc0eaa94de7b647b1057443c33a12adf294600a21583b328089ef2e2c02b4"}
Sep 08 15:32:54.126945 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:54.126863    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm" event={"ID":"5abc56a1-dc9d-4216-b03f-aea6efad8417","Type":"ContainerDied","Data":"1d161a63bc55b2fed0ed658182b5057c2b51ad4f6fb408e9b9e01716a28b7163"}
Sep 08 15:32:54.133675 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:54.133652    2560 scope.go:117] "RemoveContainer" containerID="4eb9b5594c2206079f4711364a24f51fde8662ad30a77ee1f58d2767474c6f0a"
Sep 08 15:32:54.147492 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:54.147460    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76"]
Sep 08 15:32:54.149302 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:54.149282    2560 scope.go:117] "RemoveContainer" containerID="3edcc0eaa94de7b647b1057443c33a12adf294600a21583b328089ef2e2c02b4"
Sep 08 15:32:54.149602 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:54.149575    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-4f2c6181-75fc747fbf-gbb76"]
Sep 08 15:32:54.157392 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:54.157376    2560 scope.go:117] "RemoveContainer" containerID="3edcc0eaa94de7b647b1057443c33a12adf294600a21583b328089ef2e2c02b4"
Sep 08 15:32:54.157641 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:32:54.157624    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3edcc0eaa94de7b647b1057443c33a12adf294600a21583b328089ef2e2c02b4\": container with ID starting with 3edcc0eaa94de7b647b1057443c33a12adf294600a21583b328089ef2e2c02b4 not found: ID does not exist" containerID="3edcc0eaa94de7b647b1057443c33a12adf294600a21583b328089ef2e2c02b4"
Sep 08 15:32:54.157703 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:54.157647    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3edcc0eaa94de7b647b1057443c33a12adf294600a21583b328089ef2e2c02b4"} err="failed to get container status \"3edcc0eaa94de7b647b1057443c33a12adf294600a21583b328089ef2e2c02b4\": rpc error: code = NotFound desc = could not find container \"3edcc0eaa94de7b647b1057443c33a12adf294600a21583b328089ef2e2c02b4\": container with ID starting with 3edcc0eaa94de7b647b1057443c33a12adf294600a21583b328089ef2e2c02b4 not found: ID does not exist"
Sep 08 15:32:54.167827 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:54.167800    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm"]
Sep 08 15:32:54.169608 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:54.169585    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4f2c6181-74446684cd-wn2rm"]
Sep 08 15:32:54.180096 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:54.180047    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m"]
Sep 08 15:32:54.184989 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:54.184970    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4f2c6181-b89fc7f5c-6xb9m"]
Sep 08 15:32:54.910422 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:54.910368    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:55.737775 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:55.737740    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:55.800061 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:55.800021    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5abc56a1-dc9d-4216-b03f-aea6efad8417" path="/var/lib/kubelet/pods/5abc56a1-dc9d-4216-b03f-aea6efad8417/volumes"
Sep 08 15:32:55.800523 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:55.800502    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61980147-693b-4b36-8585-42b56e61b1ef" path="/var/lib/kubelet/pods/61980147-693b-4b36-8585-42b56e61b1ef/volumes"
Sep 08 15:32:55.800904 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:55.800886    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d809fbae-e8d0-4c0e-8f91-8f67df0b6498" path="/var/lib/kubelet/pods/d809fbae-e8d0-4c0e-8f91-8f67df0b6498/volumes"
Sep 08 15:32:59.303328 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:59.303294    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:59.403748 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:32:59.403711    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:02.182776 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:02.182738    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:02.262704 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:02.262671    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:02.356464 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:02.356425    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:02.457886 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:02.457777    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:02.909732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:02.909611    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:11.929094 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:11.929022    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:28.875489 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:28.875457    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:29.717380 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:29.717351    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:32.020098 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:32.020038    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:32.358382 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:32.358305    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:35.587791 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:35.587756    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:35.692754 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:35.692690    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:35.789548 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:35.789482    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:42.806057 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:42.806018    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:42.833403 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:42.833372    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6"]
Sep 08 15:33:42.833645 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:42.833599    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6" podUID="d4c1069f-5271-4154-8250-83d9572b4f83" containerName="payload-pre-processing" containerID="cri-o://47f4f2f6d9c71c8815df006588e5acbe389559eaaf417f9191a8c89ded7dec3b" gracePeriod=30
Sep 08 15:33:42.903701 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:42.903671    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:43.002836 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:43.002808    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:43.069790 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:43.069761    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:43.091651 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:43.091629    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6"
Sep 08 15:33:43.129904 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:43.129874    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4c1069f-5271-4154-8250-83d9572b4f83-plugins-config-volume\") pod \"d4c1069f-5271-4154-8250-83d9572b4f83\" (UID: \"d4c1069f-5271-4154-8250-83d9572b4f83\") "
Sep 08 15:33:43.130100 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:43.129921    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjxg9\" (UniqueName: \"kubernetes.io/projected/d4c1069f-5271-4154-8250-83d9572b4f83-kube-api-access-rjxg9\") pod \"d4c1069f-5271-4154-8250-83d9572b4f83\" (UID: \"d4c1069f-5271-4154-8250-83d9572b4f83\") "
Sep 08 15:33:43.130243 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:43.130220    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4c1069f-5271-4154-8250-83d9572b4f83-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d4c1069f-5271-4154-8250-83d9572b4f83" (UID: "d4c1069f-5271-4154-8250-83d9572b4f83"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:33:43.132010 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:43.131972    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4c1069f-5271-4154-8250-83d9572b4f83-kube-api-access-rjxg9" (OuterVolumeSpecName: "kube-api-access-rjxg9") pod "d4c1069f-5271-4154-8250-83d9572b4f83" (UID: "d4c1069f-5271-4154-8250-83d9572b4f83"). InnerVolumeSpecName "kube-api-access-rjxg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:33:43.231573 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:43.231539    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4c1069f-5271-4154-8250-83d9572b4f83-plugins-config-volume\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:33:43.231573 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:43.231570    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjxg9\" (UniqueName: \"kubernetes.io/projected/d4c1069f-5271-4154-8250-83d9572b4f83-kube-api-access-rjxg9\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:33:43.369676 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:43.369596    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:43.377778 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:43.377753    2560 generic.go:356] "Generic (PLEG): container finished" podID="d4c1069f-5271-4154-8250-83d9572b4f83" containerID="47f4f2f6d9c71c8815df006588e5acbe389559eaaf417f9191a8c89ded7dec3b" exitCode=0
Sep 08 15:33:43.377899 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:43.377795    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6"
Sep 08 15:33:43.377899 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:43.377846    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6" event={"ID":"d4c1069f-5271-4154-8250-83d9572b4f83","Type":"ContainerDied","Data":"47f4f2f6d9c71c8815df006588e5acbe389559eaaf417f9191a8c89ded7dec3b"}
Sep 08 15:33:43.377976 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:43.377899    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6" event={"ID":"d4c1069f-5271-4154-8250-83d9572b4f83","Type":"ContainerDied","Data":"7b295ed61a8ad3b5166e2b9f525ddeab9cb64e2f1ff6e0d34150245177904427"}
Sep 08 15:33:43.377976 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:43.377924    2560 scope.go:117] "RemoveContainer" containerID="47f4f2f6d9c71c8815df006588e5acbe389559eaaf417f9191a8c89ded7dec3b"
Sep 08 15:33:43.386740 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:43.386724    2560 scope.go:117] "RemoveContainer" containerID="47f4f2f6d9c71c8815df006588e5acbe389559eaaf417f9191a8c89ded7dec3b"
Sep 08 15:33:43.386993 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:33:43.386972    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47f4f2f6d9c71c8815df006588e5acbe389559eaaf417f9191a8c89ded7dec3b\": container with ID starting with 47f4f2f6d9c71c8815df006588e5acbe389559eaaf417f9191a8c89ded7dec3b not found: ID does not exist" containerID="47f4f2f6d9c71c8815df006588e5acbe389559eaaf417f9191a8c89ded7dec3b"
Sep 08 15:33:43.387039 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:43.387001    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47f4f2f6d9c71c8815df006588e5acbe389559eaaf417f9191a8c89ded7dec3b"} err="failed to get container status \"47f4f2f6d9c71c8815df006588e5acbe389559eaaf417f9191a8c89ded7dec3b\": rpc error: code = NotFound desc = could not find container \"47f4f2f6d9c71c8815df006588e5acbe389559eaaf417f9191a8c89ded7dec3b\": container with ID starting with 47f4f2f6d9c71c8815df006588e5acbe389559eaaf417f9191a8c89ded7dec3b not found: ID does not exist"
Sep 08 15:33:43.401513 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:43.401490    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6"]
Sep 08 15:33:43.405871 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:43.405836    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-356350-68f8957d7d-btdb6"]
Sep 08 15:33:43.807383 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:43.807342    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4c1069f-5271-4154-8250-83d9572b4f83" path="/var/lib/kubelet/pods/d4c1069f-5271-4154-8250-83d9572b4f83/volumes"
Sep 08 15:33:49.462790 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:49.462751    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:49.525997 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:49.525965    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:49.724135 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:49.724026    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:50.224509 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:50.224470    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:50.572707 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:50.572629    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:50.665904 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:50.665870    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:51.611814 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:51.611768    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:51.700516 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:51.700483    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:51.799992 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:51.799951    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:51.902880 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:51.902788    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:57.649200 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:57.649170    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:58.051479 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:58.051446    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:58.157364 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:58.157327    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:58.251821 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:33:58.251789    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:08.786902 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:08.786864    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:09.612854 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:09.612820    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:15.500860 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:15.500818    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:15.604041 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:15.603994    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:15.702503 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:15.702470    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:15.801536 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:15.801458    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:15.938901 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:15.938862    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:16.466996 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:16.466966    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:16.563475 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:16.563444    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:16.663284 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:16.663252    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:18.173813 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:18.173778    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:24.157730 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:24.157694    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:24.226611 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:24.226573    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:24.880207 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:24.880174    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:26.146954 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:26.146920    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:26.270132 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:26.270101    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:26.505930 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:26.505901    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:26.626300 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:26.626267    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:31.546543 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:31.546484    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:31.633326 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:31.633294    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:31.731998 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:31.731962    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:31.832337 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:31.832255    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:32.145611 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:32.145526    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:32.636861 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:32.636822    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:35.493341 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:35.493293    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:35.588238 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:35.588200    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:36.637851 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.637813    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7"]
Sep 08 15:34:36.638329 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.638274    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4c1069f-5271-4154-8250-83d9572b4f83" containerName="payload-pre-processing"
Sep 08 15:34:36.638329 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.638294    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4c1069f-5271-4154-8250-83d9572b4f83" containerName="payload-pre-processing"
Sep 08 15:34:36.638329 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.638319    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d809fbae-e8d0-4c0e-8f91-8f67df0b6498" containerName="payload-processing"
Sep 08 15:34:36.638329 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.638328    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d809fbae-e8d0-4c0e-8f91-8f67df0b6498" containerName="payload-processing"
Sep 08 15:34:36.638573 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.638341    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61980147-693b-4b36-8585-42b56e61b1ef" containerName="maas-api"
Sep 08 15:34:36.638573 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.638349    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="61980147-693b-4b36-8585-42b56e61b1ef" containerName="maas-api"
Sep 08 15:34:36.638573 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.638362    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5abc56a1-dc9d-4216-b03f-aea6efad8417" containerName="payload-pre-processing"
Sep 08 15:34:36.638573 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.638371    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5abc56a1-dc9d-4216-b03f-aea6efad8417" containerName="payload-pre-processing"
Sep 08 15:34:36.638573 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.638456    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d809fbae-e8d0-4c0e-8f91-8f67df0b6498" containerName="payload-processing"
Sep 08 15:34:36.638573 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.638466    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="61980147-693b-4b36-8585-42b56e61b1ef" containerName="maas-api"
Sep 08 15:34:36.638573 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.638476    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4c1069f-5271-4154-8250-83d9572b4f83" containerName="payload-pre-processing"
Sep 08 15:34:36.638573 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.638514    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5abc56a1-dc9d-4216-b03f-aea6efad8417" containerName="payload-pre-processing"
Sep 08 15:34:36.643206 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.643184    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7"
Sep 08 15:34:36.645881 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.645694    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-9fbfa439-serving-cert\""
Sep 08 15:34:36.654601 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.654577    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7"]
Sep 08 15:34:36.683777 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.683750    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:36.725593 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.725558    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6e9737c-6e09-4f7d-83c3-5113eb3f1793-openshift-service-ca\") pod \"maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7\" (UID: \"f6e9737c-6e09-4f7d-83c3-5113eb3f1793\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7"
Sep 08 15:34:36.725756 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.725671    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6e9737c-6e09-4f7d-83c3-5113eb3f1793-metrics-certs\") pod \"maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7\" (UID: \"f6e9737c-6e09-4f7d-83c3-5113eb3f1793\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7"
Sep 08 15:34:36.725756 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.725711    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsk9r\" (UniqueName: \"kubernetes.io/projected/f6e9737c-6e09-4f7d-83c3-5113eb3f1793-kube-api-access-dsk9r\") pod \"maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7\" (UID: \"f6e9737c-6e09-4f7d-83c3-5113eb3f1793\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7"
Sep 08 15:34:36.725881 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.725753    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6e9737c-6e09-4f7d-83c3-5113eb3f1793-maas-api-tls\") pod \"maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7\" (UID: \"f6e9737c-6e09-4f7d-83c3-5113eb3f1793\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7"
Sep 08 15:34:36.739480 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.739448    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"]
Sep 08 15:34:36.827101 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.826968    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6e9737c-6e09-4f7d-83c3-5113eb3f1793-metrics-certs\") pod \"maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7\" (UID: \"f6e9737c-6e09-4f7d-83c3-5113eb3f1793\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7"
Sep 08 15:34:36.827101 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.827024    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsk9r\" (UniqueName: \"kubernetes.io/projected/f6e9737c-6e09-4f7d-83c3-5113eb3f1793-kube-api-access-dsk9r\") pod \"maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7\" (UID: \"f6e9737c-6e09-4f7d-83c3-5113eb3f1793\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7"
Sep 08 15:34:36.827650 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.827082    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6e9737c-6e09-4f7d-83c3-5113eb3f1793-maas-api-tls\") pod \"maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7\" (UID: \"f6e9737c-6e09-4f7d-83c3-5113eb3f1793\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7"
Sep 08 15:34:36.828167 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.828135    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6e9737c-6e09-4f7d-83c3-5113eb3f1793-openshift-service-ca\") pod \"maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7\" (UID: \"f6e9737c-6e09-4f7d-83c3-5113eb3f1793\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7"
Sep 08 15:34:36.828280 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:34:36.827607    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-serving-cert: secret "maas-api-e2e-mt-9fbfa439-serving-cert" not found
Sep 08 15:34:36.828338 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:34:36.828294    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6e9737c-6e09-4f7d-83c3-5113eb3f1793-maas-api-tls podName:f6e9737c-6e09-4f7d-83c3-5113eb3f1793 nodeName:}" failed. No retries permitted until 2026-09-08 15:34:37.32826717 +0000 UTC m=+1030.040209659 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f6e9737c-6e09-4f7d-83c3-5113eb3f1793-maas-api-tls") pod "maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7" (UID: "f6e9737c-6e09-4f7d-83c3-5113eb3f1793") : secret "maas-api-e2e-mt-9fbfa439-serving-cert" not found
Sep 08 15:34:36.829112 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.829032    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6e9737c-6e09-4f7d-83c3-5113eb3f1793-openshift-service-ca\") pod \"maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7\" (UID: \"f6e9737c-6e09-4f7d-83c3-5113eb3f1793\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7"
Sep 08 15:34:36.831229 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.831211    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6e9737c-6e09-4f7d-83c3-5113eb3f1793-metrics-certs\") pod \"maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7\" (UID: \"f6e9737c-6e09-4f7d-83c3-5113eb3f1793\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7"
Sep 08 15:34:36.841980 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.841020    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsk9r\" (UniqueName: \"kubernetes.io/projected/f6e9737c-6e09-4f7d-83c3-5113eb3f1793-kube-api-access-dsk9r\") pod \"maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7\" (UID: \"f6e9737c-6e09-4f7d-83c3-5113eb3f1793\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7"
Sep 08 15:34:36.893217 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:36.893137    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:37.080879 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:37.080844    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:37.272603 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:37.272569    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:37.333242 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:37.333208    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6e9737c-6e09-4f7d-83c3-5113eb3f1793-maas-api-tls\") pod \"maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7\" (UID: \"f6e9737c-6e09-4f7d-83c3-5113eb3f1793\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7"
Sep 08 15:34:37.335703 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:37.335681    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6e9737c-6e09-4f7d-83c3-5113eb3f1793-maas-api-tls\") pod \"maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7\" (UID: \"f6e9737c-6e09-4f7d-83c3-5113eb3f1793\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7"
Sep 08 15:34:37.452438 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:37.452395    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:37.557078 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:37.556988    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7"
Sep 08 15:34:37.833232 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:37.833204    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7"]
Sep 08 15:34:37.835587 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:34:37.835556    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6e9737c_6e09_4f7d_83c3_5113eb3f1793.slice/crio-a51e1479582c09fd5500244036880731dc961438b3fda01314a5f1dec42359d4 WatchSource:0}: Error finding container a51e1479582c09fd5500244036880731dc961438b3fda01314a5f1dec42359d4: Status 404 returned error can't find the container with id a51e1479582c09fd5500244036880731dc961438b3fda01314a5f1dec42359d4
Sep 08 15:34:38.665746 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:38.665720    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7" event={"ID":"f6e9737c-6e09-4f7d-83c3-5113eb3f1793","Type":"ContainerStarted","Data":"a51e1479582c09fd5500244036880731dc961438b3fda01314a5f1dec42359d4"}
Sep 08 15:34:39.604935 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:39.604896    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:39.674744 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:39.674711    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7" event={"ID":"f6e9737c-6e09-4f7d-83c3-5113eb3f1793","Type":"ContainerStarted","Data":"d8311e9f91ac6b0c0027fd710368a3e72859ecf8df40910919a97671178076c9"}
Sep 08 15:34:39.674939 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:39.674820    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7"
Sep 08 15:34:39.695733 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:39.695669    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7" podStartSLOduration=2.916309657 podStartE2EDuration="3.695645968s" totalImagesPullingTime="779.336311ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:34:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:34:39.693029902 +0000 UTC m=+1032.404972390" watchObservedRunningTime="2026-09-08 15:34:39.695645968 +0000 UTC m=+1032.407588460"
Sep 08 15:34:39.851948 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:39.851914    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:45.687190 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:45.687157    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7"
Sep 08 15:34:50.402140 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.402103    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v"]
Sep 08 15:34:50.402637 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.402379    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v" podUID="6796befc-a590-4901-979e-070a4678ffc8" containerName="maas-api" containerID="cri-o://8802737cd69128549fe0e5b5abe2bb0f74a2863fcfb9a964b70b1ab5b75dca35" gracePeriod=30
Sep 08 15:34:50.431277 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.431246    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:50.463384 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.463353    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl"]
Sep 08 15:34:50.463579 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.463551    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl" podUID="1c7d4493-b471-4433-9644-0ef10ae92544" containerName="payload-processing" containerID="cri-o://78eddefc3612c9ba23b337e1a7e80f624eb01758103c576e589e9257915e763f" gracePeriod=30
Sep 08 15:34:50.472528 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.472504    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb"]
Sep 08 15:34:50.472714 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.472686    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb" podUID="42f1de90-30c3-4706-b5fc-6d85f6ca6186" containerName="payload-pre-processing" containerID="cri-o://fe143af4c95281ecc7b2217a02696b35576c42477043eb2f7779f6ddbb458811" gracePeriod=30
Sep 08 15:34:50.504998 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.504969    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:50.607052 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.606979    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:50.684659 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.684586    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:50.737987 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.737748    2560 generic.go:356] "Generic (PLEG): container finished" podID="1c7d4493-b471-4433-9644-0ef10ae92544" containerID="78eddefc3612c9ba23b337e1a7e80f624eb01758103c576e589e9257915e763f" exitCode=0
Sep 08 15:34:50.737987 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.737861    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl" event={"ID":"1c7d4493-b471-4433-9644-0ef10ae92544","Type":"ContainerDied","Data":"78eddefc3612c9ba23b337e1a7e80f624eb01758103c576e589e9257915e763f"}
Sep 08 15:34:50.740329 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.740302    2560 generic.go:356] "Generic (PLEG): container finished" podID="6796befc-a590-4901-979e-070a4678ffc8" containerID="8802737cd69128549fe0e5b5abe2bb0f74a2863fcfb9a964b70b1ab5b75dca35" exitCode=0
Sep 08 15:34:50.740485 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.740465    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v" event={"ID":"6796befc-a590-4901-979e-070a4678ffc8","Type":"ContainerDied","Data":"8802737cd69128549fe0e5b5abe2bb0f74a2863fcfb9a964b70b1ab5b75dca35"}
Sep 08 15:34:50.743293 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.743254    2560 generic.go:356] "Generic (PLEG): container finished" podID="42f1de90-30c3-4706-b5fc-6d85f6ca6186" containerID="fe143af4c95281ecc7b2217a02696b35576c42477043eb2f7779f6ddbb458811" exitCode=0
Sep 08 15:34:50.743426 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.743324    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb" event={"ID":"42f1de90-30c3-4706-b5fc-6d85f6ca6186","Type":"ContainerDied","Data":"fe143af4c95281ecc7b2217a02696b35576c42477043eb2f7779f6ddbb458811"}
Sep 08 15:34:50.766988 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.766962    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl"
Sep 08 15:34:50.782322 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.782304    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v"
Sep 08 15:34:50.796830 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.796800    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb"
Sep 08 15:34:50.864592 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.864558    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6796befc-a590-4901-979e-070a4678ffc8-metrics-certs\") pod \"6796befc-a590-4901-979e-070a4678ffc8\" (UID: \"6796befc-a590-4901-979e-070a4678ffc8\") "
Sep 08 15:34:50.864732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.864626    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6796befc-a590-4901-979e-070a4678ffc8-openshift-service-ca\") pod \"6796befc-a590-4901-979e-070a4678ffc8\" (UID: \"6796befc-a590-4901-979e-070a4678ffc8\") "
Sep 08 15:34:50.864732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.864657    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6796befc-a590-4901-979e-070a4678ffc8-maas-api-tls\") pod \"6796befc-a590-4901-979e-070a4678ffc8\" (UID: \"6796befc-a590-4901-979e-070a4678ffc8\") "
Sep 08 15:34:50.864732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.864703    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7mnq\" (UniqueName: \"kubernetes.io/projected/1c7d4493-b471-4433-9644-0ef10ae92544-kube-api-access-r7mnq\") pod \"1c7d4493-b471-4433-9644-0ef10ae92544\" (UID: \"1c7d4493-b471-4433-9644-0ef10ae92544\") "
Sep 08 15:34:50.864858 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.864748    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gm8cp\" (UniqueName: \"kubernetes.io/projected/6796befc-a590-4901-979e-070a4678ffc8-kube-api-access-gm8cp\") pod \"6796befc-a590-4901-979e-070a4678ffc8\" (UID: \"6796befc-a590-4901-979e-070a4678ffc8\") "
Sep 08 15:34:50.864858 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.864778    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c7d4493-b471-4433-9644-0ef10ae92544-plugins-config-volume\") pod \"1c7d4493-b471-4433-9644-0ef10ae92544\" (UID: \"1c7d4493-b471-4433-9644-0ef10ae92544\") "
Sep 08 15:34:50.865326 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.865266    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c7d4493-b471-4433-9644-0ef10ae92544-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1c7d4493-b471-4433-9644-0ef10ae92544" (UID: "1c7d4493-b471-4433-9644-0ef10ae92544"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:34:50.865326 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.865266    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6796befc-a590-4901-979e-070a4678ffc8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6796befc-a590-4901-979e-070a4678ffc8" (UID: "6796befc-a590-4901-979e-070a4678ffc8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:34:50.867327 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.867293    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6796befc-a590-4901-979e-070a4678ffc8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6796befc-a590-4901-979e-070a4678ffc8" (UID: "6796befc-a590-4901-979e-070a4678ffc8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:34:50.867327 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.867299    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c7d4493-b471-4433-9644-0ef10ae92544-kube-api-access-r7mnq" (OuterVolumeSpecName: "kube-api-access-r7mnq") pod "1c7d4493-b471-4433-9644-0ef10ae92544" (UID: "1c7d4493-b471-4433-9644-0ef10ae92544"). InnerVolumeSpecName "kube-api-access-r7mnq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:34:50.867462 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.867389    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6796befc-a590-4901-979e-070a4678ffc8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6796befc-a590-4901-979e-070a4678ffc8" (UID: "6796befc-a590-4901-979e-070a4678ffc8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:34:50.867462 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.867397    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6796befc-a590-4901-979e-070a4678ffc8-kube-api-access-gm8cp" (OuterVolumeSpecName: "kube-api-access-gm8cp") pod "6796befc-a590-4901-979e-070a4678ffc8" (UID: "6796befc-a590-4901-979e-070a4678ffc8"). InnerVolumeSpecName "kube-api-access-gm8cp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:34:50.966194 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.966106    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42f1de90-30c3-4706-b5fc-6d85f6ca6186-plugins-config-volume\") pod \"42f1de90-30c3-4706-b5fc-6d85f6ca6186\" (UID: \"42f1de90-30c3-4706-b5fc-6d85f6ca6186\") "
Sep 08 15:34:50.966367 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.966209    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tc792\" (UniqueName: \"kubernetes.io/projected/42f1de90-30c3-4706-b5fc-6d85f6ca6186-kube-api-access-tc792\") pod \"42f1de90-30c3-4706-b5fc-6d85f6ca6186\" (UID: \"42f1de90-30c3-4706-b5fc-6d85f6ca6186\") "
Sep 08 15:34:50.966473 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.966449    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42f1de90-30c3-4706-b5fc-6d85f6ca6186-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "42f1de90-30c3-4706-b5fc-6d85f6ca6186" (UID: "42f1de90-30c3-4706-b5fc-6d85f6ca6186"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:34:50.966542 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.966529    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6796befc-a590-4901-979e-070a4678ffc8-openshift-service-ca\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:34:50.966583 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.966545    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6796befc-a590-4901-979e-070a4678ffc8-maas-api-tls\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:34:50.966617 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.966581    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42f1de90-30c3-4706-b5fc-6d85f6ca6186-plugins-config-volume\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:34:50.966617 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.966591    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7mnq\" (UniqueName: \"kubernetes.io/projected/1c7d4493-b471-4433-9644-0ef10ae92544-kube-api-access-r7mnq\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:34:50.966617 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.966599    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gm8cp\" (UniqueName: \"kubernetes.io/projected/6796befc-a590-4901-979e-070a4678ffc8-kube-api-access-gm8cp\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:34:50.966617 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.966608    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c7d4493-b471-4433-9644-0ef10ae92544-plugins-config-volume\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:34:50.966617 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.966616    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6796befc-a590-4901-979e-070a4678ffc8-metrics-certs\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:34:50.968218 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:50.968196    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42f1de90-30c3-4706-b5fc-6d85f6ca6186-kube-api-access-tc792" (OuterVolumeSpecName: "kube-api-access-tc792") pod "42f1de90-30c3-4706-b5fc-6d85f6ca6186" (UID: "42f1de90-30c3-4706-b5fc-6d85f6ca6186"). InnerVolumeSpecName "kube-api-access-tc792". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:34:51.039563 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:51.039506    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:51.067336 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:51.067308    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tc792\" (UniqueName: \"kubernetes.io/projected/42f1de90-30c3-4706-b5fc-6d85f6ca6186-kube-api-access-tc792\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:34:51.751020 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:51.750985    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl" event={"ID":"1c7d4493-b471-4433-9644-0ef10ae92544","Type":"ContainerDied","Data":"311e73d9d90978f5a4cd01f3686275398be3d4d188f323cf4bcc69238dd35e11"}
Sep 08 15:34:51.751492 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:51.751043    2560 scope.go:117] "RemoveContainer" containerID="78eddefc3612c9ba23b337e1a7e80f624eb01758103c576e589e9257915e763f"
Sep 08 15:34:51.751492 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:51.750995    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl"
Sep 08 15:34:51.752594 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:51.752570    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v"
Sep 08 15:34:51.752722 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:51.752646    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v" event={"ID":"6796befc-a590-4901-979e-070a4678ffc8","Type":"ContainerDied","Data":"495227613ee23abafd26f916ed461152e0883891a189f14db956382562181938"}
Sep 08 15:34:51.754341 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:51.754310    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb" event={"ID":"42f1de90-30c3-4706-b5fc-6d85f6ca6186","Type":"ContainerDied","Data":"21d260f9547f2b9a68b1c5ddf89106bac20cd95d567ff254157b57a0c7d81a7b"}
Sep 08 15:34:51.754465 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:51.754339    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb"
Sep 08 15:34:51.760505 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:51.760272    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4" podUID="6a2c7454-f621-4bf4-ad4c-c14494fcf0d1" containerName="main" containerID="cri-o://a99498269debfdc17ae046a9786a7cc3af0a7cbd429c032545f2f94fd0cd32d0" gracePeriod=45
Sep 08 15:34:51.762161 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:51.762143    2560 scope.go:117] "RemoveContainer" containerID="8802737cd69128549fe0e5b5abe2bb0f74a2863fcfb9a964b70b1ab5b75dca35"
Sep 08 15:34:51.770915 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:51.770899    2560 scope.go:117] "RemoveContainer" containerID="fe143af4c95281ecc7b2217a02696b35576c42477043eb2f7779f6ddbb458811"
Sep 08 15:34:51.778744 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:51.778721    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl"]
Sep 08 15:34:51.783641 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:51.783619    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-34fed8-86df68cc-97nxl"]
Sep 08 15:34:51.792543 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:51.792524    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v"]
Sep 08 15:34:51.801868 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:51.801844    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c7d4493-b471-4433-9644-0ef10ae92544" path="/var/lib/kubelet/pods/1c7d4493-b471-4433-9644-0ef10ae92544/volumes"
Sep 08 15:34:51.802142 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:51.802128    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-34fed8-df7446444-tlk9v"]
Sep 08 15:34:51.807697 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:51.807676    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb"]
Sep 08 15:34:51.813861 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:51.813843    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-34fed8-76d7f7cb94-qd7rb"]
Sep 08 15:34:52.009152 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:52.009090    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"
Sep 08 15:34:52.177396 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:52.177360    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-home\") pod \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\" (UID: \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\") "
Sep 08 15:34:52.177542 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:52.177407    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-model-cache\") pod \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\" (UID: \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\") "
Sep 08 15:34:52.177542 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:52.177429    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-tmp-dir\") pod \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\" (UID: \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\") "
Sep 08 15:34:52.177542 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:52.177477    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-tls-certs\") pod \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\" (UID: \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\") "
Sep 08 15:34:52.177542 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:52.177502    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-dshm\") pod \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\" (UID: \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\") "
Sep 08 15:34:52.177763 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:52.177546    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k48q7\" (UniqueName: \"kubernetes.io/projected/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-kube-api-access-k48q7\") pod \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\" (UID: \"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1\") "
Sep 08 15:34:52.177763 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:52.177682    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-model-cache" (OuterVolumeSpecName: "model-cache") pod "6a2c7454-f621-4bf4-ad4c-c14494fcf0d1" (UID: "6a2c7454-f621-4bf4-ad4c-c14494fcf0d1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:34:52.177763 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:52.177713    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-home" (OuterVolumeSpecName: "home") pod "6a2c7454-f621-4bf4-ad4c-c14494fcf0d1" (UID: "6a2c7454-f621-4bf4-ad4c-c14494fcf0d1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:34:52.177920 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:52.177773    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6a2c7454-f621-4bf4-ad4c-c14494fcf0d1" (UID: "6a2c7454-f621-4bf4-ad4c-c14494fcf0d1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:34:52.177920 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:52.177861    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-home\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:34:52.177920 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:52.177880    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-model-cache\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:34:52.177920 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:52.177893    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-tmp-dir\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:34:52.179694 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:52.179660    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6a2c7454-f621-4bf4-ad4c-c14494fcf0d1" (UID: "6a2c7454-f621-4bf4-ad4c-c14494fcf0d1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:34:52.179797 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:52.179703    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-dshm" (OuterVolumeSpecName: "dshm") pod "6a2c7454-f621-4bf4-ad4c-c14494fcf0d1" (UID: "6a2c7454-f621-4bf4-ad4c-c14494fcf0d1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:34:52.179797 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:52.179724    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-kube-api-access-k48q7" (OuterVolumeSpecName: "kube-api-access-k48q7") pod "6a2c7454-f621-4bf4-ad4c-c14494fcf0d1" (UID: "6a2c7454-f621-4bf4-ad4c-c14494fcf0d1"). InnerVolumeSpecName "kube-api-access-k48q7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:34:52.278610 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:52.278518    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-tls-certs\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:34:52.278610 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:52.278552    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-dshm\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:34:52.278610 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:52.278562    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k48q7\" (UniqueName: \"kubernetes.io/projected/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1-kube-api-access-k48q7\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:34:52.764608 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:52.764575    2560 generic.go:356] "Generic (PLEG): container finished" podID="6a2c7454-f621-4bf4-ad4c-c14494fcf0d1" containerID="a99498269debfdc17ae046a9786a7cc3af0a7cbd429c032545f2f94fd0cd32d0" exitCode=0
Sep 08 15:34:52.764608 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:52.764613    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4" event={"ID":"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1","Type":"ContainerDied","Data":"a99498269debfdc17ae046a9786a7cc3af0a7cbd429c032545f2f94fd0cd32d0"}
Sep 08 15:34:52.765118 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:52.764624    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"
Sep 08 15:34:52.765118 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:52.764638    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4" event={"ID":"6a2c7454-f621-4bf4-ad4c-c14494fcf0d1","Type":"ContainerDied","Data":"d9f3693287c440ad41f3ad4814715b042f8886bbab7b9a45996a79bb71f2e12f"}
Sep 08 15:34:52.765118 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:52.764654    2560 scope.go:117] "RemoveContainer" containerID="a99498269debfdc17ae046a9786a7cc3af0a7cbd429c032545f2f94fd0cd32d0"
Sep 08 15:34:52.773835 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:52.773819    2560 scope.go:117] "RemoveContainer" containerID="a99498269debfdc17ae046a9786a7cc3af0a7cbd429c032545f2f94fd0cd32d0"
Sep 08 15:34:52.774115 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:34:52.774092    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a99498269debfdc17ae046a9786a7cc3af0a7cbd429c032545f2f94fd0cd32d0\": container with ID starting with a99498269debfdc17ae046a9786a7cc3af0a7cbd429c032545f2f94fd0cd32d0 not found: ID does not exist" containerID="a99498269debfdc17ae046a9786a7cc3af0a7cbd429c032545f2f94fd0cd32d0"
Sep 08 15:34:52.774177 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:52.774122    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a99498269debfdc17ae046a9786a7cc3af0a7cbd429c032545f2f94fd0cd32d0"} err="failed to get container status \"a99498269debfdc17ae046a9786a7cc3af0a7cbd429c032545f2f94fd0cd32d0\": rpc error: code = NotFound desc = could not find container \"a99498269debfdc17ae046a9786a7cc3af0a7cbd429c032545f2f94fd0cd32d0\": container with ID starting with a99498269debfdc17ae046a9786a7cc3af0a7cbd429c032545f2f94fd0cd32d0 not found: ID does not exist"
Sep 08 15:34:52.783977 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:52.783953    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"]
Sep 08 15:34:52.789592 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:52.789574    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4"]
Sep 08 15:34:53.800287 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:53.800250    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42f1de90-30c3-4706-b5fc-6d85f6ca6186" path="/var/lib/kubelet/pods/42f1de90-30c3-4706-b5fc-6d85f6ca6186/volumes"
Sep 08 15:34:53.800652 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:53.800638    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6796befc-a590-4901-979e-070a4678ffc8" path="/var/lib/kubelet/pods/6796befc-a590-4901-979e-070a4678ffc8/volumes"
Sep 08 15:34:53.800967 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:53.800954    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a2c7454-f621-4bf4-ad4c-c14494fcf0d1" path="/var/lib/kubelet/pods/6a2c7454-f621-4bf4-ad4c-c14494fcf0d1/volumes"
Sep 08 15:34:57.005344 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:57.005292    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-inf-a-34fed8/test-model-34fed8-kserve-99475969c-rgmc4" podUID="6a2c7454-f621-4bf4-ad4c-c14494fcf0d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Sep 08 15:34:57.831362 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:57.831324    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:57.930922 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:57.930884    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:58.126759 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:58.126679    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:58.720233 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:58.720205    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:59.625034 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:34:59.624997    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:00.417047 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:00.417009    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:01.044430 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:01.044398    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:01.138682 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:01.138641    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:01.470562 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:01.470528    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:07.798456 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:07.798418    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:07.897546 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:07.897507    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:08.991681 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:08.991644    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:09.049727 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:09.049695    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5"]
Sep 08 15:35:09.049919 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:09.049894    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5" podUID="8c9cbb9b-2f0e-443d-803f-c92512a4bcef" containerName="payload-pre-processing" containerID="cri-o://8343e417432b195e7385a0971c4e573b31420477e3fef20ab260b18a8a43063c" gracePeriod=30
Sep 08 15:35:09.191670 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:09.191616    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:09.288441 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:09.288409    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:09.307511 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:09.307485    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5"
Sep 08 15:35:09.392895 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:09.392867    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:09.436163 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:09.436129    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m49rt\" (UniqueName: \"kubernetes.io/projected/8c9cbb9b-2f0e-443d-803f-c92512a4bcef-kube-api-access-m49rt\") pod \"8c9cbb9b-2f0e-443d-803f-c92512a4bcef\" (UID: \"8c9cbb9b-2f0e-443d-803f-c92512a4bcef\") "
Sep 08 15:35:09.436339 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:09.436194    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c9cbb9b-2f0e-443d-803f-c92512a4bcef-plugins-config-volume\") pod \"8c9cbb9b-2f0e-443d-803f-c92512a4bcef\" (UID: \"8c9cbb9b-2f0e-443d-803f-c92512a4bcef\") "
Sep 08 15:35:09.436599 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:09.436575    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c9cbb9b-2f0e-443d-803f-c92512a4bcef-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8c9cbb9b-2f0e-443d-803f-c92512a4bcef" (UID: "8c9cbb9b-2f0e-443d-803f-c92512a4bcef"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:35:09.438297 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:09.438269    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c9cbb9b-2f0e-443d-803f-c92512a4bcef-kube-api-access-m49rt" (OuterVolumeSpecName: "kube-api-access-m49rt") pod "8c9cbb9b-2f0e-443d-803f-c92512a4bcef" (UID: "8c9cbb9b-2f0e-443d-803f-c92512a4bcef"). InnerVolumeSpecName "kube-api-access-m49rt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:35:09.537484 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:09.537403    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m49rt\" (UniqueName: \"kubernetes.io/projected/8c9cbb9b-2f0e-443d-803f-c92512a4bcef-kube-api-access-m49rt\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:35:09.537484 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:09.537439    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c9cbb9b-2f0e-443d-803f-c92512a4bcef-plugins-config-volume\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:35:09.788936 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:09.788861    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:09.852746 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:09.852711    2560 generic.go:356] "Generic (PLEG): container finished" podID="8c9cbb9b-2f0e-443d-803f-c92512a4bcef" containerID="8343e417432b195e7385a0971c4e573b31420477e3fef20ab260b18a8a43063c" exitCode=0
Sep 08 15:35:09.852906 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:09.852760    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5"
Sep 08 15:35:09.852906 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:09.852797    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5" event={"ID":"8c9cbb9b-2f0e-443d-803f-c92512a4bcef","Type":"ContainerDied","Data":"8343e417432b195e7385a0971c4e573b31420477e3fef20ab260b18a8a43063c"}
Sep 08 15:35:09.852906 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:09.852856    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5" event={"ID":"8c9cbb9b-2f0e-443d-803f-c92512a4bcef","Type":"ContainerDied","Data":"96c0bc802e38088bcb51ecf3d29ef59da0da99cd8e3c68786bec51095eb8996d"}
Sep 08 15:35:09.852906 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:09.852886    2560 scope.go:117] "RemoveContainer" containerID="8343e417432b195e7385a0971c4e573b31420477e3fef20ab260b18a8a43063c"
Sep 08 15:35:09.861833 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:09.861812    2560 scope.go:117] "RemoveContainer" containerID="8343e417432b195e7385a0971c4e573b31420477e3fef20ab260b18a8a43063c"
Sep 08 15:35:09.862137 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:35:09.862112    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8343e417432b195e7385a0971c4e573b31420477e3fef20ab260b18a8a43063c\": container with ID starting with 8343e417432b195e7385a0971c4e573b31420477e3fef20ab260b18a8a43063c not found: ID does not exist" containerID="8343e417432b195e7385a0971c4e573b31420477e3fef20ab260b18a8a43063c"
Sep 08 15:35:09.862210 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:09.862149    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8343e417432b195e7385a0971c4e573b31420477e3fef20ab260b18a8a43063c"} err="failed to get container status \"8343e417432b195e7385a0971c4e573b31420477e3fef20ab260b18a8a43063c\": rpc error: code = NotFound desc = could not find container \"8343e417432b195e7385a0971c4e573b31420477e3fef20ab260b18a8a43063c\": container with ID starting with 8343e417432b195e7385a0971c4e573b31420477e3fef20ab260b18a8a43063c not found: ID does not exist"
Sep 08 15:35:09.891842 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:09.891815    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5"]
Sep 08 15:35:09.928473 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:09.925636    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e39762-544ddb47d6-f9kw5"]
Sep 08 15:35:10.858110 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:10.858055    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:11.799664 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:11.799631    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c9cbb9b-2f0e-443d-803f-c92512a4bcef" path="/var/lib/kubelet/pods/8c9cbb9b-2f0e-443d-803f-c92512a4bcef/volumes"
Sep 08 15:35:13.263899 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:13.263864    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:13.365279 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:13.365246    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:15.990847 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:15.990809    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:16.090014 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:16.089981    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:16.214960 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:16.214913    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:16.755671 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:16.755635    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:17.129095 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:17.128979    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:17.840512 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:17.840477    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:18.847280 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:18.847246    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7"]
Sep 08 15:35:18.847807 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:18.847517    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7" podUID="f6e9737c-6e09-4f7d-83c3-5113eb3f1793" containerName="maas-api" containerID="cri-o://d8311e9f91ac6b0c0027fd710368a3e72859ecf8df40910919a97671178076c9" gracePeriod=30
Sep 08 15:35:18.916346 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:18.916312    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:19.084523 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:19.084495    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:19.110456 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:19.110389    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7"
Sep 08 15:35:19.200607 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:19.200559    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:19.226675 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:19.226642    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsk9r\" (UniqueName: \"kubernetes.io/projected/f6e9737c-6e09-4f7d-83c3-5113eb3f1793-kube-api-access-dsk9r\") pod \"f6e9737c-6e09-4f7d-83c3-5113eb3f1793\" (UID: \"f6e9737c-6e09-4f7d-83c3-5113eb3f1793\") "
Sep 08 15:35:19.226880 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:19.226711    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6e9737c-6e09-4f7d-83c3-5113eb3f1793-metrics-certs\") pod \"f6e9737c-6e09-4f7d-83c3-5113eb3f1793\" (UID: \"f6e9737c-6e09-4f7d-83c3-5113eb3f1793\") "
Sep 08 15:35:19.226880 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:19.226770    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6e9737c-6e09-4f7d-83c3-5113eb3f1793-openshift-service-ca\") pod \"f6e9737c-6e09-4f7d-83c3-5113eb3f1793\" (UID: \"f6e9737c-6e09-4f7d-83c3-5113eb3f1793\") "
Sep 08 15:35:19.226880 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:19.226786    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6e9737c-6e09-4f7d-83c3-5113eb3f1793-maas-api-tls\") pod \"f6e9737c-6e09-4f7d-83c3-5113eb3f1793\" (UID: \"f6e9737c-6e09-4f7d-83c3-5113eb3f1793\") "
Sep 08 15:35:19.227209 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:19.227174    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6e9737c-6e09-4f7d-83c3-5113eb3f1793-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f6e9737c-6e09-4f7d-83c3-5113eb3f1793" (UID: "f6e9737c-6e09-4f7d-83c3-5113eb3f1793"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:35:19.228922 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:19.228897    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6e9737c-6e09-4f7d-83c3-5113eb3f1793-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f6e9737c-6e09-4f7d-83c3-5113eb3f1793" (UID: "f6e9737c-6e09-4f7d-83c3-5113eb3f1793"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:35:19.229097 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:19.229052    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6e9737c-6e09-4f7d-83c3-5113eb3f1793-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f6e9737c-6e09-4f7d-83c3-5113eb3f1793" (UID: "f6e9737c-6e09-4f7d-83c3-5113eb3f1793"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:35:19.229097 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:19.229057    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6e9737c-6e09-4f7d-83c3-5113eb3f1793-kube-api-access-dsk9r" (OuterVolumeSpecName: "kube-api-access-dsk9r") pod "f6e9737c-6e09-4f7d-83c3-5113eb3f1793" (UID: "f6e9737c-6e09-4f7d-83c3-5113eb3f1793"). InnerVolumeSpecName "kube-api-access-dsk9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:35:19.327767 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:19.327725    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6e9737c-6e09-4f7d-83c3-5113eb3f1793-openshift-service-ca\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:35:19.327767 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:19.327763    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6e9737c-6e09-4f7d-83c3-5113eb3f1793-maas-api-tls\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:35:19.328039 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:19.327778    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsk9r\" (UniqueName: \"kubernetes.io/projected/f6e9737c-6e09-4f7d-83c3-5113eb3f1793-kube-api-access-dsk9r\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:35:19.328039 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:19.327788    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6e9737c-6e09-4f7d-83c3-5113eb3f1793-metrics-certs\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:35:19.409441 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:19.409352    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:19.496727 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:19.496696    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:19.808725 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:19.808696    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:19.905450 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:19.905416    2560 generic.go:356] "Generic (PLEG): container finished" podID="f6e9737c-6e09-4f7d-83c3-5113eb3f1793" containerID="d8311e9f91ac6b0c0027fd710368a3e72859ecf8df40910919a97671178076c9" exitCode=0
Sep 08 15:35:19.905450 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:19.905456    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7"
Sep 08 15:35:19.905938 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:19.905502    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7" event={"ID":"f6e9737c-6e09-4f7d-83c3-5113eb3f1793","Type":"ContainerDied","Data":"d8311e9f91ac6b0c0027fd710368a3e72859ecf8df40910919a97671178076c9"}
Sep 08 15:35:19.905938 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:19.905556    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7" event={"ID":"f6e9737c-6e09-4f7d-83c3-5113eb3f1793","Type":"ContainerDied","Data":"a51e1479582c09fd5500244036880731dc961438b3fda01314a5f1dec42359d4"}
Sep 08 15:35:19.905938 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:19.905581    2560 scope.go:117] "RemoveContainer" containerID="d8311e9f91ac6b0c0027fd710368a3e72859ecf8df40910919a97671178076c9"
Sep 08 15:35:19.914548 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:19.914530    2560 scope.go:117] "RemoveContainer" containerID="d8311e9f91ac6b0c0027fd710368a3e72859ecf8df40910919a97671178076c9"
Sep 08 15:35:19.914802 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:35:19.914783    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8311e9f91ac6b0c0027fd710368a3e72859ecf8df40910919a97671178076c9\": container with ID starting with d8311e9f91ac6b0c0027fd710368a3e72859ecf8df40910919a97671178076c9 not found: ID does not exist" containerID="d8311e9f91ac6b0c0027fd710368a3e72859ecf8df40910919a97671178076c9"
Sep 08 15:35:19.914850 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:19.914810    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8311e9f91ac6b0c0027fd710368a3e72859ecf8df40910919a97671178076c9"} err="failed to get container status \"d8311e9f91ac6b0c0027fd710368a3e72859ecf8df40910919a97671178076c9\": rpc error: code = NotFound desc = could not find container \"d8311e9f91ac6b0c0027fd710368a3e72859ecf8df40910919a97671178076c9\": container with ID starting with d8311e9f91ac6b0c0027fd710368a3e72859ecf8df40910919a97671178076c9 not found: ID does not exist"
Sep 08 15:35:19.923320 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:19.923295    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7"]
Sep 08 15:35:19.936705 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:19.936661    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9fbfa439-5ddc5cfc68-2kll7"]
Sep 08 15:35:20.659899 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:20.659860    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:20.771081 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:20.771030    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:21.799947 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:21.799913    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6e9737c-6e09-4f7d-83c3-5113eb3f1793" path="/var/lib/kubelet/pods/f6e9737c-6e09-4f7d-83c3-5113eb3f1793/volumes"
Sep 08 15:35:21.906324 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:21.906291    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw"]
Sep 08 15:35:21.906716 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:21.906698    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6796befc-a590-4901-979e-070a4678ffc8" containerName="maas-api"
Sep 08 15:35:21.906802 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:21.906720    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6796befc-a590-4901-979e-070a4678ffc8" containerName="maas-api"
Sep 08 15:35:21.906802 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:21.906736    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42f1de90-30c3-4706-b5fc-6d85f6ca6186" containerName="payload-pre-processing"
Sep 08 15:35:21.906802 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:21.906753    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="42f1de90-30c3-4706-b5fc-6d85f6ca6186" containerName="payload-pre-processing"
Sep 08 15:35:21.906802 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:21.906765    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6e9737c-6e09-4f7d-83c3-5113eb3f1793" containerName="maas-api"
Sep 08 15:35:21.906802 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:21.906773    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6e9737c-6e09-4f7d-83c3-5113eb3f1793" containerName="maas-api"
Sep 08 15:35:21.906802 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:21.906791    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c9cbb9b-2f0e-443d-803f-c92512a4bcef" containerName="payload-pre-processing"
Sep 08 15:35:21.906802 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:21.906799    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c9cbb9b-2f0e-443d-803f-c92512a4bcef" containerName="payload-pre-processing"
Sep 08 15:35:21.907194 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:21.906817    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c7d4493-b471-4433-9644-0ef10ae92544" containerName="payload-processing"
Sep 08 15:35:21.907194 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:21.906826    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c7d4493-b471-4433-9644-0ef10ae92544" containerName="payload-processing"
Sep 08 15:35:21.907194 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:21.906851    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a2c7454-f621-4bf4-ad4c-c14494fcf0d1" containerName="main"
Sep 08 15:35:21.907194 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:21.906860    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a2c7454-f621-4bf4-ad4c-c14494fcf0d1" containerName="main"
Sep 08 15:35:21.907194 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:21.906942    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6796befc-a590-4901-979e-070a4678ffc8" containerName="maas-api"
Sep 08 15:35:21.907194 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:21.906954    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6e9737c-6e09-4f7d-83c3-5113eb3f1793" containerName="maas-api"
Sep 08 15:35:21.907194 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:21.906965    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c9cbb9b-2f0e-443d-803f-c92512a4bcef" containerName="payload-pre-processing"
Sep 08 15:35:21.907194 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:21.906974    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c7d4493-b471-4433-9644-0ef10ae92544" containerName="payload-processing"
Sep 08 15:35:21.907194 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:21.906986    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="42f1de90-30c3-4706-b5fc-6d85f6ca6186" containerName="payload-pre-processing"
Sep 08 15:35:21.907194 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:21.906998    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a2c7454-f621-4bf4-ad4c-c14494fcf0d1" containerName="main"
Sep 08 15:35:21.911604 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:21.911580    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw"
Sep 08 15:35:21.916548 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:21.913810    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-9ca1fc-serving-cert\""
Sep 08 15:35:21.929406 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:21.929365    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw"]
Sep 08 15:35:21.998092 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:21.998040    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:22.053449 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.053369    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/265fd771-aa9f-4b17-a164-b05ae1f0faa1-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw\" (UID: \"265fd771-aa9f-4b17-a164-b05ae1f0faa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw"
Sep 08 15:35:22.053449 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.053430    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/265fd771-aa9f-4b17-a164-b05ae1f0faa1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw\" (UID: \"265fd771-aa9f-4b17-a164-b05ae1f0faa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw"
Sep 08 15:35:22.053627 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.053482    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbcfs\" (UniqueName: \"kubernetes.io/projected/265fd771-aa9f-4b17-a164-b05ae1f0faa1-kube-api-access-hbcfs\") pod \"maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw\" (UID: \"265fd771-aa9f-4b17-a164-b05ae1f0faa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw"
Sep 08 15:35:22.053627 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.053570    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/265fd771-aa9f-4b17-a164-b05ae1f0faa1-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw\" (UID: \"265fd771-aa9f-4b17-a164-b05ae1f0faa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw"
Sep 08 15:35:22.099962 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.099928    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:22.154005 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.153968    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbcfs\" (UniqueName: \"kubernetes.io/projected/265fd771-aa9f-4b17-a164-b05ae1f0faa1-kube-api-access-hbcfs\") pod \"maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw\" (UID: \"265fd771-aa9f-4b17-a164-b05ae1f0faa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw"
Sep 08 15:35:22.154179 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.154014    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/265fd771-aa9f-4b17-a164-b05ae1f0faa1-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw\" (UID: \"265fd771-aa9f-4b17-a164-b05ae1f0faa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw"
Sep 08 15:35:22.154179 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.154062    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/265fd771-aa9f-4b17-a164-b05ae1f0faa1-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw\" (UID: \"265fd771-aa9f-4b17-a164-b05ae1f0faa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw"
Sep 08 15:35:22.154179 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.154133    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/265fd771-aa9f-4b17-a164-b05ae1f0faa1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw\" (UID: \"265fd771-aa9f-4b17-a164-b05ae1f0faa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw"
Sep 08 15:35:22.154732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.154706    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/265fd771-aa9f-4b17-a164-b05ae1f0faa1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw\" (UID: \"265fd771-aa9f-4b17-a164-b05ae1f0faa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw"
Sep 08 15:35:22.156560 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.156537    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/265fd771-aa9f-4b17-a164-b05ae1f0faa1-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw\" (UID: \"265fd771-aa9f-4b17-a164-b05ae1f0faa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw"
Sep 08 15:35:22.156654 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.156562    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/265fd771-aa9f-4b17-a164-b05ae1f0faa1-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw\" (UID: \"265fd771-aa9f-4b17-a164-b05ae1f0faa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw"
Sep 08 15:35:22.165001 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.164983    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbcfs\" (UniqueName: \"kubernetes.io/projected/265fd771-aa9f-4b17-a164-b05ae1f0faa1-kube-api-access-hbcfs\") pod \"maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw\" (UID: \"265fd771-aa9f-4b17-a164-b05ae1f0faa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw"
Sep 08 15:35:22.202148 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.202115    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:22.234158 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.234122    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw"
Sep 08 15:35:22.511976 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.511948    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw"]
Sep 08 15:35:22.515230 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:35:22.515202    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod265fd771_aa9f_4b17_a164_b05ae1f0faa1.slice/crio-0f79cdb65d24ad999f3dc99e05f4ff4e0e46c0e62e3fe19103a15d9cdf05623e WatchSource:0}: Error finding container 0f79cdb65d24ad999f3dc99e05f4ff4e0e46c0e62e3fe19103a15d9cdf05623e: Status 404 returned error can't find the container with id 0f79cdb65d24ad999f3dc99e05f4ff4e0e46c0e62e3fe19103a15d9cdf05623e
Sep 08 15:35:22.516559 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.516537    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 15:35:22.531973 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.531950    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:22.633679 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.633646    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:22.698228 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.698198    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm"]
Sep 08 15:35:22.702628 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.702611    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm"
Sep 08 15:35:22.704427 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.704408    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-9ca1fc\""
Sep 08 15:35:22.704549 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.704532    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-9ca1fc-dockercfg-fhc88\""
Sep 08 15:35:22.713588 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.713564    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm"]
Sep 08 15:35:22.861479 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.861387    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdlfd\" (UniqueName: \"kubernetes.io/projected/9ffccf45-ca2a-435b-9464-3f4aae9f6678-kube-api-access-gdlfd\") pod \"payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm\" (UID: \"9ffccf45-ca2a-435b-9464-3f4aae9f6678\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm"
Sep 08 15:35:22.861479 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.861457    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ffccf45-ca2a-435b-9464-3f4aae9f6678-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm\" (UID: \"9ffccf45-ca2a-435b-9464-3f4aae9f6678\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm"
Sep 08 15:35:22.940264 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.940225    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw" event={"ID":"265fd771-aa9f-4b17-a164-b05ae1f0faa1","Type":"ContainerStarted","Data":"1b28102dcb0e8f0add9890674387f29cecf70a8d8d16558b034080430d914b47"}
Sep 08 15:35:22.940433 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.940295    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw" event={"ID":"265fd771-aa9f-4b17-a164-b05ae1f0faa1","Type":"ContainerStarted","Data":"0f79cdb65d24ad999f3dc99e05f4ff4e0e46c0e62e3fe19103a15d9cdf05623e"}
Sep 08 15:35:22.940837 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.940779    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw"
Sep 08 15:35:22.960105 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.960042    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw" podStartSLOduration=1.6035228209999999 podStartE2EDuration="1.960025788s" totalImagesPullingTime="356.502967ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:35:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:35:22.95757753 +0000 UTC m=+1075.669520024" watchObservedRunningTime="2026-09-08 15:35:22.960025788 +0000 UTC m=+1075.671968317"
Sep 08 15:35:22.962454 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.962427    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdlfd\" (UniqueName: \"kubernetes.io/projected/9ffccf45-ca2a-435b-9464-3f4aae9f6678-kube-api-access-gdlfd\") pod \"payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm\" (UID: \"9ffccf45-ca2a-435b-9464-3f4aae9f6678\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm"
Sep 08 15:35:22.962550 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.962488    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ffccf45-ca2a-435b-9464-3f4aae9f6678-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm\" (UID: \"9ffccf45-ca2a-435b-9464-3f4aae9f6678\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm"
Sep 08 15:35:22.963165 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.963147    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ffccf45-ca2a-435b-9464-3f4aae9f6678-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm\" (UID: \"9ffccf45-ca2a-435b-9464-3f4aae9f6678\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm"
Sep 08 15:35:22.973352 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:22.973330    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdlfd\" (UniqueName: \"kubernetes.io/projected/9ffccf45-ca2a-435b-9464-3f4aae9f6678-kube-api-access-gdlfd\") pod \"payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm\" (UID: \"9ffccf45-ca2a-435b-9464-3f4aae9f6678\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm"
Sep 08 15:35:23.014280 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:23.014251    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm"
Sep 08 15:35:23.288972 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:23.288640    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm"]
Sep 08 15:35:23.291640 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:35:23.291612    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ffccf45_ca2a_435b_9464_3f4aae9f6678.slice/crio-3308c9debaef89d031b6f254a3d2e7f6c4b1a9c1ea0df4579930a841670bd780 WatchSource:0}: Error finding container 3308c9debaef89d031b6f254a3d2e7f6c4b1a9c1ea0df4579930a841670bd780: Status 404 returned error can't find the container with id 3308c9debaef89d031b6f254a3d2e7f6c4b1a9c1ea0df4579930a841670bd780
Sep 08 15:35:23.948216 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:23.948178    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm" event={"ID":"9ffccf45-ca2a-435b-9464-3f4aae9f6678","Type":"ContainerStarted","Data":"7dd6aa53722cdcf053b2d49d3b6d5b3c7342d6907c962168d21adf0430a6032d"}
Sep 08 15:35:23.948663 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:23.948241    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm" event={"ID":"9ffccf45-ca2a-435b-9464-3f4aae9f6678","Type":"ContainerStarted","Data":"3308c9debaef89d031b6f254a3d2e7f6c4b1a9c1ea0df4579930a841670bd780"}
Sep 08 15:35:23.981713 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:23.981662    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm" podStartSLOduration=1.981646907 podStartE2EDuration="1.981646907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:35:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:35:23.980758808 +0000 UTC m=+1076.692701323" watchObservedRunningTime="2026-09-08 15:35:23.981646907 +0000 UTC m=+1076.693589397"
Sep 08 15:35:24.953842 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:24.953813    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm"
Sep 08 15:35:26.287945 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:26.287906    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:26.392088 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:26.392044    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:26.678384 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:26.678350    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:26.938343 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:26.938261    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:27.039434 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:27.039400    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:27.145046 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:27.145014    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:28.954061 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:28.954034    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw"
Sep 08 15:35:33.607876 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:33.607844    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:33.718416 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:33.718372    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:33.810020 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:33.809984    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:33.957459 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:33.957423    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:35.966587 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:35.966556    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm"
Sep 08 15:35:39.854927 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:39.854893    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:39.959280 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:39.959236    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:40.762301 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:40.762268    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:41.123268 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:41.123191    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:41.592330 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:41.592291    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:44.004545 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:44.004512    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:44.096572 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:44.096539    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:45.212241 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:45.212210    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6"]
Sep 08 15:35:45.216585 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:45.216564    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6"
Sep 08 15:35:45.218718 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:45.218700    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-03e8df-serving-cert\""
Sep 08 15:35:45.229735 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:45.229708    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6"]
Sep 08 15:35:45.252961 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:45.252932    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:45.271985 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:45.271962    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e606c7e2-90e3-45ef-932d-afc43ab58e12-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6\" (UID: \"e606c7e2-90e3-45ef-932d-afc43ab58e12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6"
Sep 08 15:35:45.272096 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:45.271993    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k66gv\" (UniqueName: \"kubernetes.io/projected/e606c7e2-90e3-45ef-932d-afc43ab58e12-kube-api-access-k66gv\") pod \"maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6\" (UID: \"e606c7e2-90e3-45ef-932d-afc43ab58e12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6"
Sep 08 15:35:45.272096 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:45.272014    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e606c7e2-90e3-45ef-932d-afc43ab58e12-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6\" (UID: \"e606c7e2-90e3-45ef-932d-afc43ab58e12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6"
Sep 08 15:35:45.272096 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:45.272063    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e606c7e2-90e3-45ef-932d-afc43ab58e12-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6\" (UID: \"e606c7e2-90e3-45ef-932d-afc43ab58e12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6"
Sep 08 15:35:45.355738 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:45.355695    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:45.372977 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:45.372944    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k66gv\" (UniqueName: \"kubernetes.io/projected/e606c7e2-90e3-45ef-932d-afc43ab58e12-kube-api-access-k66gv\") pod \"maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6\" (UID: \"e606c7e2-90e3-45ef-932d-afc43ab58e12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6"
Sep 08 15:35:45.373142 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:45.372983    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e606c7e2-90e3-45ef-932d-afc43ab58e12-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6\" (UID: \"e606c7e2-90e3-45ef-932d-afc43ab58e12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6"
Sep 08 15:35:45.373142 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:45.373026    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e606c7e2-90e3-45ef-932d-afc43ab58e12-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6\" (UID: \"e606c7e2-90e3-45ef-932d-afc43ab58e12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6"
Sep 08 15:35:45.373142 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:45.373131    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e606c7e2-90e3-45ef-932d-afc43ab58e12-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6\" (UID: \"e606c7e2-90e3-45ef-932d-afc43ab58e12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6"
Sep 08 15:35:45.373311 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:35:45.373218    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-serving-cert: secret "maas-api-e2e-shared-a-w2-03e8df-serving-cert" not found
Sep 08 15:35:45.373311 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:35:45.373278    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e606c7e2-90e3-45ef-932d-afc43ab58e12-maas-api-tls podName:e606c7e2-90e3-45ef-932d-afc43ab58e12 nodeName:}" failed. No retries permitted until 2026-09-08 15:35:45.873261774 +0000 UTC m=+1098.585204243 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e606c7e2-90e3-45ef-932d-afc43ab58e12-maas-api-tls") pod "maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6" (UID: "e606c7e2-90e3-45ef-932d-afc43ab58e12") : secret "maas-api-e2e-shared-a-w2-03e8df-serving-cert" not found
Sep 08 15:35:45.373858 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:45.373833    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e606c7e2-90e3-45ef-932d-afc43ab58e12-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6\" (UID: \"e606c7e2-90e3-45ef-932d-afc43ab58e12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6"
Sep 08 15:35:45.375635 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:45.375602    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e606c7e2-90e3-45ef-932d-afc43ab58e12-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6\" (UID: \"e606c7e2-90e3-45ef-932d-afc43ab58e12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6"
Sep 08 15:35:45.384925 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:45.384901    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k66gv\" (UniqueName: \"kubernetes.io/projected/e606c7e2-90e3-45ef-932d-afc43ab58e12-kube-api-access-k66gv\") pod \"maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6\" (UID: \"e606c7e2-90e3-45ef-932d-afc43ab58e12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6"
Sep 08 15:35:45.450723 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:45.450681    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:45.550593 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:45.550494    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:45.668941 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:45.668906    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:45.879738 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:45.879660    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e606c7e2-90e3-45ef-932d-afc43ab58e12-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6\" (UID: \"e606c7e2-90e3-45ef-932d-afc43ab58e12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6"
Sep 08 15:35:45.882430 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:45.882406    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e606c7e2-90e3-45ef-932d-afc43ab58e12-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6\" (UID: \"e606c7e2-90e3-45ef-932d-afc43ab58e12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6"
Sep 08 15:35:45.979092 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:45.979033    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:46.074484 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:46.074446    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:46.126262 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:46.126219    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6"
Sep 08 15:35:46.148906 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:46.148870    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8"]
Sep 08 15:35:46.153209 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:46.153189    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8"
Sep 08 15:35:46.155569 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:46.155498    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-03e8df\""
Sep 08 15:35:46.155732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:46.155712    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-03e8df-dockercfg-fkgbf\""
Sep 08 15:35:46.164880 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:46.164849    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8"]
Sep 08 15:35:46.182392 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:46.182222    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgndk\" (UniqueName: \"kubernetes.io/projected/47b63d67-2d05-4020-bc72-2592f6b21710-kube-api-access-fgndk\") pod \"payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8\" (UID: \"47b63d67-2d05-4020-bc72-2592f6b21710\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8"
Sep 08 15:35:46.182392 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:46.182305    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47b63d67-2d05-4020-bc72-2592f6b21710-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8\" (UID: \"47b63d67-2d05-4020-bc72-2592f6b21710\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8"
Sep 08 15:35:46.283448 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:46.283406    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47b63d67-2d05-4020-bc72-2592f6b21710-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8\" (UID: \"47b63d67-2d05-4020-bc72-2592f6b21710\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8"
Sep 08 15:35:46.283889 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:46.283536    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgndk\" (UniqueName: \"kubernetes.io/projected/47b63d67-2d05-4020-bc72-2592f6b21710-kube-api-access-fgndk\") pod \"payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8\" (UID: \"47b63d67-2d05-4020-bc72-2592f6b21710\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8"
Sep 08 15:35:46.284185 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:46.284158    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47b63d67-2d05-4020-bc72-2592f6b21710-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8\" (UID: \"47b63d67-2d05-4020-bc72-2592f6b21710\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8"
Sep 08 15:35:46.292535 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:46.292500    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgndk\" (UniqueName: \"kubernetes.io/projected/47b63d67-2d05-4020-bc72-2592f6b21710-kube-api-access-fgndk\") pod \"payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8\" (UID: \"47b63d67-2d05-4020-bc72-2592f6b21710\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8"
Sep 08 15:35:46.415173 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:46.415146    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6"]
Sep 08 15:35:46.417226 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:35:46.417195    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode606c7e2_90e3_45ef_932d_afc43ab58e12.slice/crio-1320343276d526624b45206645609301c0c96068b6772828524bcf6abb66f3e3 WatchSource:0}: Error finding container 1320343276d526624b45206645609301c0c96068b6772828524bcf6abb66f3e3: Status 404 returned error can't find the container with id 1320343276d526624b45206645609301c0c96068b6772828524bcf6abb66f3e3
Sep 08 15:35:46.432160 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:46.432132    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:46.465052 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:46.465021    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8"
Sep 08 15:35:46.745506 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:46.745474    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8"]
Sep 08 15:35:46.748116 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:35:46.748078    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47b63d67_2d05_4020_bc72_2592f6b21710.slice/crio-8dc1b276fff8af0284de4f5fb2bb80a71992ce02ffb5b1901b83f60e4fe5cd5b WatchSource:0}: Error finding container 8dc1b276fff8af0284de4f5fb2bb80a71992ce02ffb5b1901b83f60e4fe5cd5b: Status 404 returned error can't find the container with id 8dc1b276fff8af0284de4f5fb2bb80a71992ce02ffb5b1901b83f60e4fe5cd5b
Sep 08 15:35:47.068455 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:47.068355    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8" event={"ID":"47b63d67-2d05-4020-bc72-2592f6b21710","Type":"ContainerStarted","Data":"4c070c91c3d2739cf83f2c00e1108b169c467e6ff7502c6711e64dcba87b92c7"}
Sep 08 15:35:47.068455 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:47.068421    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8" event={"ID":"47b63d67-2d05-4020-bc72-2592f6b21710","Type":"ContainerStarted","Data":"8dc1b276fff8af0284de4f5fb2bb80a71992ce02ffb5b1901b83f60e4fe5cd5b"}
Sep 08 15:35:47.068698 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:47.068516    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8"
Sep 08 15:35:47.070137 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:47.070111    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6" event={"ID":"e606c7e2-90e3-45ef-932d-afc43ab58e12","Type":"ContainerStarted","Data":"16ed1cb6e212d81c212a31729962d87bbebde339eaf1ffe96bf4139d5ba99b0f"}
Sep 08 15:35:47.070250 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:47.070170    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6" event={"ID":"e606c7e2-90e3-45ef-932d-afc43ab58e12","Type":"ContainerStarted","Data":"1320343276d526624b45206645609301c0c96068b6772828524bcf6abb66f3e3"}
Sep 08 15:35:47.070250 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:47.070203    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6"
Sep 08 15:35:47.100149 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:47.100088    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8" podStartSLOduration=1.100061158 podStartE2EDuration="1.100061158s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:35:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:35:47.099729264 +0000 UTC m=+1099.811671765" watchObservedRunningTime="2026-09-08 15:35:47.100061158 +0000 UTC m=+1099.812003649"
Sep 08 15:35:49.132749 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:49.132717    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:49.234393 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:49.234357    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:50.544652 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:50.544616    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:50.651501 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:50.651470    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:51.432510 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:51.432473    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:51.530239 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:51.530200    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:51.589923 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:51.589870    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6" podStartSLOduration=6.25144264 podStartE2EDuration="6.589852042s" totalImagesPullingTime="338.409402ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:35:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:35:47.137101728 +0000 UTC m=+1099.849044213" watchObservedRunningTime="2026-09-08 15:35:51.589852042 +0000 UTC m=+1104.301794533"
Sep 08 15:35:51.591397 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:51.591376    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4"]
Sep 08 15:35:51.594838 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:51.594819    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4"
Sep 08 15:35:51.596924 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:51.596901    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-9fb31f-dockercfg-zvbs4\""
Sep 08 15:35:51.597418 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:51.597401    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-9fb31f\""
Sep 08 15:35:51.608454 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:51.608430    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4"]
Sep 08 15:35:51.629931 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:51.629902    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl2kp\" (UniqueName: \"kubernetes.io/projected/cdca0a4b-6927-48dc-90b6-bb712a50d2cf-kube-api-access-wl2kp\") pod \"payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4\" (UID: \"cdca0a4b-6927-48dc-90b6-bb712a50d2cf\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4"
Sep 08 15:35:51.630056 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:51.629983    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdca0a4b-6927-48dc-90b6-bb712a50d2cf-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4\" (UID: \"cdca0a4b-6927-48dc-90b6-bb712a50d2cf\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4"
Sep 08 15:35:51.731477 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:51.731375    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl2kp\" (UniqueName: \"kubernetes.io/projected/cdca0a4b-6927-48dc-90b6-bb712a50d2cf-kube-api-access-wl2kp\") pod \"payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4\" (UID: \"cdca0a4b-6927-48dc-90b6-bb712a50d2cf\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4"
Sep 08 15:35:51.731646 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:51.731474    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdca0a4b-6927-48dc-90b6-bb712a50d2cf-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4\" (UID: \"cdca0a4b-6927-48dc-90b6-bb712a50d2cf\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4"
Sep 08 15:35:51.732279 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:51.732255    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdca0a4b-6927-48dc-90b6-bb712a50d2cf-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4\" (UID: \"cdca0a4b-6927-48dc-90b6-bb712a50d2cf\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4"
Sep 08 15:35:51.741305 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:51.741281    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl2kp\" (UniqueName: \"kubernetes.io/projected/cdca0a4b-6927-48dc-90b6-bb712a50d2cf-kube-api-access-wl2kp\") pod \"payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4\" (UID: \"cdca0a4b-6927-48dc-90b6-bb712a50d2cf\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4"
Sep 08 15:35:51.905329 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:51.905291    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4"
Sep 08 15:35:52.178949 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:52.178860    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4"]
Sep 08 15:35:52.181454 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:35:52.181426    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdca0a4b_6927_48dc_90b6_bb712a50d2cf.slice/crio-63b824f7f93b72557fc3fb9194343c7fd37fecd51911b590543a328c3979beed WatchSource:0}: Error finding container 63b824f7f93b72557fc3fb9194343c7fd37fecd51911b590543a328c3979beed: Status 404 returned error can't find the container with id 63b824f7f93b72557fc3fb9194343c7fd37fecd51911b590543a328c3979beed
Sep 08 15:35:53.081442 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:53.081416    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6"
Sep 08 15:35:53.102841 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:53.102807    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4" event={"ID":"cdca0a4b-6927-48dc-90b6-bb712a50d2cf","Type":"ContainerStarted","Data":"2b0ef22f5e7ae017e13451cd143931ba8fa3e4e4c479a5c0f252755184ff36f7"}
Sep 08 15:35:53.103005 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:53.102854    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4" event={"ID":"cdca0a4b-6927-48dc-90b6-bb712a50d2cf","Type":"ContainerStarted","Data":"63b824f7f93b72557fc3fb9194343c7fd37fecd51911b590543a328c3979beed"}
Sep 08 15:35:53.103005 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:53.102965    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4"
Sep 08 15:35:53.119097 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:53.119020    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4" podStartSLOduration=2.119000921 podStartE2EDuration="2.119000921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:35:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:35:53.118002674 +0000 UTC m=+1105.829945164" watchObservedRunningTime="2026-09-08 15:35:53.119000921 +0000 UTC m=+1105.830943442"
Sep 08 15:35:58.078781 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:35:58.078748    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8"
Sep 08 15:36:04.110253 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:04.110219    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4"
Sep 08 15:36:07.785244 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:07.785203    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:08.934399 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:08.934366    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:09.260336 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:09.260245    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:09.593384 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:09.593298    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:09.890158 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:09.890050    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:11.960274 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:11.960229    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:12.262286 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:12.262199    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:12.353063 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:12.353027    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:13.955858 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:13.955825    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:14.016740 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:14.016707    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:14.132352 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:14.132318    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:14.509800 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:14.509766    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:14.697430 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:14.697390    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:14.844995 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:14.844920    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:14.973880 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:14.973846    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8"]
Sep 08 15:36:14.977466 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:14.977449    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8"
Sep 08 15:36:14.980175 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:14.980149    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-a7ae4e-dockercfg-qlhwm\""
Sep 08 15:36:14.980284 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:14.980170    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-a7ae4e\""
Sep 08 15:36:14.991738 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:14.991714    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8"]
Sep 08 15:36:15.148519 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:15.148442    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24bc2a8e-f67f-4289-81d4-0f582de6299d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8\" (UID: \"24bc2a8e-f67f-4289-81d4-0f582de6299d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8"
Sep 08 15:36:15.148519 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:15.148476    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj6rg\" (UniqueName: \"kubernetes.io/projected/24bc2a8e-f67f-4289-81d4-0f582de6299d-kube-api-access-lj6rg\") pod \"payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8\" (UID: \"24bc2a8e-f67f-4289-81d4-0f582de6299d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8"
Sep 08 15:36:15.249457 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:15.249423    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24bc2a8e-f67f-4289-81d4-0f582de6299d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8\" (UID: \"24bc2a8e-f67f-4289-81d4-0f582de6299d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8"
Sep 08 15:36:15.249457 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:15.249466    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lj6rg\" (UniqueName: \"kubernetes.io/projected/24bc2a8e-f67f-4289-81d4-0f582de6299d-kube-api-access-lj6rg\") pod \"payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8\" (UID: \"24bc2a8e-f67f-4289-81d4-0f582de6299d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8"
Sep 08 15:36:15.250219 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:15.250198    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24bc2a8e-f67f-4289-81d4-0f582de6299d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8\" (UID: \"24bc2a8e-f67f-4289-81d4-0f582de6299d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8"
Sep 08 15:36:15.266667 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:15.266640    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj6rg\" (UniqueName: \"kubernetes.io/projected/24bc2a8e-f67f-4289-81d4-0f582de6299d-kube-api-access-lj6rg\") pod \"payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8\" (UID: \"24bc2a8e-f67f-4289-81d4-0f582de6299d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8"
Sep 08 15:36:15.287509 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:15.287485    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8"
Sep 08 15:36:15.361787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:15.361758    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:15.562836 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:15.562799    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8"]
Sep 08 15:36:15.565854 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:36:15.565825    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24bc2a8e_f67f_4289_81d4_0f582de6299d.slice/crio-a95c3f30b395dea6b9ac8baf48cb39fa29b2d55ab6b51dfcc0a02c808ecd9f7b WatchSource:0}: Error finding container a95c3f30b395dea6b9ac8baf48cb39fa29b2d55ab6b51dfcc0a02c808ecd9f7b: Status 404 returned error can't find the container with id a95c3f30b395dea6b9ac8baf48cb39fa29b2d55ab6b51dfcc0a02c808ecd9f7b
Sep 08 15:36:16.221636 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:16.221600    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8" event={"ID":"24bc2a8e-f67f-4289-81d4-0f582de6299d","Type":"ContainerStarted","Data":"e1dfdc65a5d8193275c1f3655e9be31b6ad50d2328e9b23eb113d1a0ad91029d"}
Sep 08 15:36:16.222035 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:16.221655    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8" event={"ID":"24bc2a8e-f67f-4289-81d4-0f582de6299d","Type":"ContainerStarted","Data":"a95c3f30b395dea6b9ac8baf48cb39fa29b2d55ab6b51dfcc0a02c808ecd9f7b"}
Sep 08 15:36:16.222035 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:16.221679    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8"
Sep 08 15:36:16.237573 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:16.237546    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:16.241875 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:16.241834    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8" podStartSLOduration=2.241820868 podStartE2EDuration="2.241820868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:36:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:36:16.240204628 +0000 UTC m=+1128.952147120" watchObservedRunningTime="2026-09-08 15:36:16.241820868 +0000 UTC m=+1128.953763358"
Sep 08 15:36:21.095813 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:21.095778    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:27.229375 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:27.229342    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8"
Sep 08 15:36:32.804746 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:32.804709    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:32.867212 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:32.867171    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:37.239946 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:37.239911    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:40.750428 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:40.750387    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:48.826915 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:48.826876    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:49.198103 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:49.198038    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:49.440642 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:49.440605    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:52.618016 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:52.617936    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:52.717892 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:52.717856    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:53.820830 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:53.820796    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s"]
Sep 08 15:36:53.826140 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:53.826113    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s"
Sep 08 15:36:53.827927 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:53.827909    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-4e499427-serving-cert\""
Sep 08 15:36:53.833600 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:53.833570    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s"]
Sep 08 15:36:53.842534 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:53.842506    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:53.899959 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:53.899931    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsmxw\" (UniqueName: \"kubernetes.io/projected/df2c2386-37dc-462c-8c29-587d153f6bd7-kube-api-access-vsmxw\") pod \"maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s\" (UID: \"df2c2386-37dc-462c-8c29-587d153f6bd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s"
Sep 08 15:36:53.900141 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:53.899983    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df2c2386-37dc-462c-8c29-587d153f6bd7-openshift-service-ca\") pod \"maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s\" (UID: \"df2c2386-37dc-462c-8c29-587d153f6bd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s"
Sep 08 15:36:53.900141 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:53.900017    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df2c2386-37dc-462c-8c29-587d153f6bd7-maas-api-tls\") pod \"maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s\" (UID: \"df2c2386-37dc-462c-8c29-587d153f6bd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s"
Sep 08 15:36:53.900141 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:53.900034    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df2c2386-37dc-462c-8c29-587d153f6bd7-metrics-certs\") pod \"maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s\" (UID: \"df2c2386-37dc-462c-8c29-587d153f6bd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s"
Sep 08 15:36:53.909954 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:53.909921    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:54.000914 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.000880    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df2c2386-37dc-462c-8c29-587d153f6bd7-openshift-service-ca\") pod \"maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s\" (UID: \"df2c2386-37dc-462c-8c29-587d153f6bd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s"
Sep 08 15:36:54.001122 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.000927    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df2c2386-37dc-462c-8c29-587d153f6bd7-maas-api-tls\") pod \"maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s\" (UID: \"df2c2386-37dc-462c-8c29-587d153f6bd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s"
Sep 08 15:36:54.001122 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.001026    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df2c2386-37dc-462c-8c29-587d153f6bd7-metrics-certs\") pod \"maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s\" (UID: \"df2c2386-37dc-462c-8c29-587d153f6bd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s"
Sep 08 15:36:54.001234 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.001139    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsmxw\" (UniqueName: \"kubernetes.io/projected/df2c2386-37dc-462c-8c29-587d153f6bd7-kube-api-access-vsmxw\") pod \"maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s\" (UID: \"df2c2386-37dc-462c-8c29-587d153f6bd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s"
Sep 08 15:36:54.001651 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.001626    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df2c2386-37dc-462c-8c29-587d153f6bd7-openshift-service-ca\") pod \"maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s\" (UID: \"df2c2386-37dc-462c-8c29-587d153f6bd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s"
Sep 08 15:36:54.003534 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.003514    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df2c2386-37dc-462c-8c29-587d153f6bd7-metrics-certs\") pod \"maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s\" (UID: \"df2c2386-37dc-462c-8c29-587d153f6bd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s"
Sep 08 15:36:54.003642 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.003572    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df2c2386-37dc-462c-8c29-587d153f6bd7-maas-api-tls\") pod \"maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s\" (UID: \"df2c2386-37dc-462c-8c29-587d153f6bd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s"
Sep 08 15:36:54.008502 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.008474    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:54.011110 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.011051    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsmxw\" (UniqueName: \"kubernetes.io/projected/df2c2386-37dc-462c-8c29-587d153f6bd7-kube-api-access-vsmxw\") pod \"maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s\" (UID: \"df2c2386-37dc-462c-8c29-587d153f6bd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s"
Sep 08 15:36:54.138669 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.138577    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s"
Sep 08 15:36:54.416605 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.416580    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s"]
Sep 08 15:36:54.419210 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:36:54.419181    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf2c2386_37dc_462c_8c29_587d153f6bd7.slice/crio-9c6a476fb4496e6b3e9978cbfdb31f7e6a6a40c2d569dd96fba36a9d6d320509 WatchSource:0}: Error finding container 9c6a476fb4496e6b3e9978cbfdb31f7e6a6a40c2d569dd96fba36a9d6d320509: Status 404 returned error can't find the container with id 9c6a476fb4496e6b3e9978cbfdb31f7e6a6a40c2d569dd96fba36a9d6d320509
Sep 08 15:36:54.498120 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.498026    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:54.597841 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.597809    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:54.608615 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.608588    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs"]
Sep 08 15:36:54.613242 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.613222    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs"
Sep 08 15:36:54.615052 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.615034    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-4e499427\""
Sep 08 15:36:54.615173 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.615157    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-4e499427-dockercfg-nhhk7\""
Sep 08 15:36:54.622593 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.622569    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx"]
Sep 08 15:36:54.626341 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.626321    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs"]
Sep 08 15:36:54.626439 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.626402    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx"
Sep 08 15:36:54.640909 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.640883    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx"]
Sep 08 15:36:54.706486 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.706449    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c2f38bd-9cc3-4731-8cc5-1be941f14b33-plugins-config-volume\") pod \"payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx\" (UID: \"1c2f38bd-9cc3-4731-8cc5-1be941f14b33\") " pod="openshift-ingress/payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx"
Sep 08 15:36:54.706695 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.706503    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba7fb594-e64c-4a4b-98a3-c8a57a74ed18-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs\" (UID: \"ba7fb594-e64c-4a4b-98a3-c8a57a74ed18\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs"
Sep 08 15:36:54.706695 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.706568    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42ssc\" (UniqueName: \"kubernetes.io/projected/1c2f38bd-9cc3-4731-8cc5-1be941f14b33-kube-api-access-42ssc\") pod \"payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx\" (UID: \"1c2f38bd-9cc3-4731-8cc5-1be941f14b33\") " pod="openshift-ingress/payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx"
Sep 08 15:36:54.706695 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.706661    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbs46\" (UniqueName: \"kubernetes.io/projected/ba7fb594-e64c-4a4b-98a3-c8a57a74ed18-kube-api-access-kbs46\") pod \"payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs\" (UID: \"ba7fb594-e64c-4a4b-98a3-c8a57a74ed18\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs"
Sep 08 15:36:54.807572 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.807537    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c2f38bd-9cc3-4731-8cc5-1be941f14b33-plugins-config-volume\") pod \"payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx\" (UID: \"1c2f38bd-9cc3-4731-8cc5-1be941f14b33\") " pod="openshift-ingress/payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx"
Sep 08 15:36:54.807767 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.807585    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba7fb594-e64c-4a4b-98a3-c8a57a74ed18-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs\" (UID: \"ba7fb594-e64c-4a4b-98a3-c8a57a74ed18\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs"
Sep 08 15:36:54.807767 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.807641    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42ssc\" (UniqueName: \"kubernetes.io/projected/1c2f38bd-9cc3-4731-8cc5-1be941f14b33-kube-api-access-42ssc\") pod \"payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx\" (UID: \"1c2f38bd-9cc3-4731-8cc5-1be941f14b33\") " pod="openshift-ingress/payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx"
Sep 08 15:36:54.807767 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.807727    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbs46\" (UniqueName: \"kubernetes.io/projected/ba7fb594-e64c-4a4b-98a3-c8a57a74ed18-kube-api-access-kbs46\") pod \"payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs\" (UID: \"ba7fb594-e64c-4a4b-98a3-c8a57a74ed18\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs"
Sep 08 15:36:54.808313 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.808292    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba7fb594-e64c-4a4b-98a3-c8a57a74ed18-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs\" (UID: \"ba7fb594-e64c-4a4b-98a3-c8a57a74ed18\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs"
Sep 08 15:36:54.808386 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.808336    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c2f38bd-9cc3-4731-8cc5-1be941f14b33-plugins-config-volume\") pod \"payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx\" (UID: \"1c2f38bd-9cc3-4731-8cc5-1be941f14b33\") " pod="openshift-ingress/payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx"
Sep 08 15:36:54.815683 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.815654    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbs46\" (UniqueName: \"kubernetes.io/projected/ba7fb594-e64c-4a4b-98a3-c8a57a74ed18-kube-api-access-kbs46\") pod \"payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs\" (UID: \"ba7fb594-e64c-4a4b-98a3-c8a57a74ed18\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs"
Sep 08 15:36:54.815769 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.815714    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42ssc\" (UniqueName: \"kubernetes.io/projected/1c2f38bd-9cc3-4731-8cc5-1be941f14b33-kube-api-access-42ssc\") pod \"payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx\" (UID: \"1c2f38bd-9cc3-4731-8cc5-1be941f14b33\") " pod="openshift-ingress/payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx"
Sep 08 15:36:54.924870 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.924839    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs"
Sep 08 15:36:54.936604 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:54.936582    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx"
Sep 08 15:36:55.207040 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:55.206818    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs"]
Sep 08 15:36:55.209651 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:36:55.209617    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba7fb594_e64c_4a4b_98a3_c8a57a74ed18.slice/crio-3969795ee869779d6414d29c3f39f21e57692dea7b48871dc776b461ac66c8da WatchSource:0}: Error finding container 3969795ee869779d6414d29c3f39f21e57692dea7b48871dc776b461ac66c8da: Status 404 returned error can't find the container with id 3969795ee869779d6414d29c3f39f21e57692dea7b48871dc776b461ac66c8da
Sep 08 15:36:55.224876 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:55.224847    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx"]
Sep 08 15:36:55.228371 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:36:55.228346    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c2f38bd_9cc3_4731_8cc5_1be941f14b33.slice/crio-5c19f0bf631ac176d2cab9c7fcda01f9f74d6b8c5ef685bfa246834f49aee0ac WatchSource:0}: Error finding container 5c19f0bf631ac176d2cab9c7fcda01f9f74d6b8c5ef685bfa246834f49aee0ac: Status 404 returned error can't find the container with id 5c19f0bf631ac176d2cab9c7fcda01f9f74d6b8c5ef685bfa246834f49aee0ac
Sep 08 15:36:55.417394 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:55.417351    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs" event={"ID":"ba7fb594-e64c-4a4b-98a3-c8a57a74ed18","Type":"ContainerStarted","Data":"bf9b424aa9d8c90eb208694ec30a7fce37a5d6d2156be81a9264112b386bf64e"}
Sep 08 15:36:55.417585 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:55.417445    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs" event={"ID":"ba7fb594-e64c-4a4b-98a3-c8a57a74ed18","Type":"ContainerStarted","Data":"3969795ee869779d6414d29c3f39f21e57692dea7b48871dc776b461ac66c8da"}
Sep 08 15:36:55.417585 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:55.417475    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs"
Sep 08 15:36:55.419080 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:55.419038    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx" event={"ID":"1c2f38bd-9cc3-4731-8cc5-1be941f14b33","Type":"ContainerStarted","Data":"3ff7337e4de0c03779feb8195bd4483b3f1b5b705ccbbd18562f0ee3864a15e1"}
Sep 08 15:36:55.419204 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:55.419112    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx" event={"ID":"1c2f38bd-9cc3-4731-8cc5-1be941f14b33","Type":"ContainerStarted","Data":"5c19f0bf631ac176d2cab9c7fcda01f9f74d6b8c5ef685bfa246834f49aee0ac"}
Sep 08 15:36:55.419301 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:55.419285    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx"
Sep 08 15:36:55.420971 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:55.420944    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s" event={"ID":"df2c2386-37dc-462c-8c29-587d153f6bd7","Type":"ContainerStarted","Data":"16f26f6ff9c6bc3f2f78251aa62b4ebdd0f8218b76518cb6802c0ada94e3e85a"}
Sep 08 15:36:55.421101 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:55.420982    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s" event={"ID":"df2c2386-37dc-462c-8c29-587d153f6bd7","Type":"ContainerStarted","Data":"9c6a476fb4496e6b3e9978cbfdb31f7e6a6a40c2d569dd96fba36a9d6d320509"}
Sep 08 15:36:55.421179 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:55.421115    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s"
Sep 08 15:36:55.435517 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:55.435453    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs" podStartSLOduration=1.435434007 podStartE2EDuration="1.435434007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:36:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:36:55.434540862 +0000 UTC m=+1168.146483365" watchObservedRunningTime="2026-09-08 15:36:55.435434007 +0000 UTC m=+1168.147376499"
Sep 08 15:36:55.454966 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:55.454916    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx" podStartSLOduration=1.454885143 podStartE2EDuration="1.454885143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:36:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:36:55.450616517 +0000 UTC m=+1168.162559009" watchObservedRunningTime="2026-09-08 15:36:55.454885143 +0000 UTC m=+1168.166827640"
Sep 08 15:36:55.472150 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:36:55.472050    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s" podStartSLOduration=1.876332359 podStartE2EDuration="2.472030309s" totalImagesPullingTime="595.69795ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:36:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:36:55.468922724 +0000 UTC m=+1168.180865217" watchObservedRunningTime="2026-09-08 15:36:55.472030309 +0000 UTC m=+1168.183972801"
Sep 08 15:37:01.431000 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:37:01.430967    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s"
Sep 08 15:37:06.428143 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:37:06.428110    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx"
Sep 08 15:37:06.428577 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:37:06.428199    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs"
Sep 08 15:37:17.581710 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:37:17.581655    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:22.444567 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:37:22.444523    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:27.759825 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:37:27.759794    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vc5n9_17e30810-dfd0-4e6f-9204-731d1a6479a3/console-operator/2.log"
Sep 08 15:37:27.761346 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:37:27.761320    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vc5n9_17e30810-dfd0-4e6f-9204-731d1a6479a3/console-operator/2.log"
Sep 08 15:37:43.915032 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:37:43.914993    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:11.745361 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:11.745322    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:18.814728 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:18.814677    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:19.197834 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:19.197802    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:25.823363 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:25.823277    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:27.856191 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:27.856148    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s"]
Sep 08 15:38:27.857594 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:27.857538    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s" podUID="df2c2386-37dc-462c-8c29-587d153f6bd7" containerName="maas-api" containerID="cri-o://16f26f6ff9c6bc3f2f78251aa62b4ebdd0f8218b76518cb6802c0ada94e3e85a" gracePeriod=30
Sep 08 15:38:27.885664 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:27.885626    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:27.924717 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:27.924678    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx"]
Sep 08 15:38:27.924974 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:27.924942    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx" podUID="1c2f38bd-9cc3-4731-8cc5-1be941f14b33" containerName="payload-processing" containerID="cri-o://3ff7337e4de0c03779feb8195bd4483b3f1b5b705ccbbd18562f0ee3864a15e1" gracePeriod=30
Sep 08 15:38:27.935792 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:27.935761    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs"]
Sep 08 15:38:27.936008 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:27.935982    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs" podUID="ba7fb594-e64c-4a4b-98a3-c8a57a74ed18" containerName="payload-pre-processing" containerID="cri-o://bf9b424aa9d8c90eb208694ec30a7fce37a5d6d2156be81a9264112b386bf64e" gracePeriod=30
Sep 08 15:38:27.957897 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:27.957865    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:28.060769 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.060735    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:28.178091 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.177684    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:28.217941 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.217909    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s"
Sep 08 15:38:28.268568 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.268521    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:28.271555 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.271536    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs"
Sep 08 15:38:28.274895 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.274873    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx"
Sep 08 15:38:28.361978 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.361946    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df2c2386-37dc-462c-8c29-587d153f6bd7-openshift-service-ca\") pod \"df2c2386-37dc-462c-8c29-587d153f6bd7\" (UID: \"df2c2386-37dc-462c-8c29-587d153f6bd7\") "
Sep 08 15:38:28.362192 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.361987    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vsmxw\" (UniqueName: \"kubernetes.io/projected/df2c2386-37dc-462c-8c29-587d153f6bd7-kube-api-access-vsmxw\") pod \"df2c2386-37dc-462c-8c29-587d153f6bd7\" (UID: \"df2c2386-37dc-462c-8c29-587d153f6bd7\") "
Sep 08 15:38:28.362192 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.362054    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba7fb594-e64c-4a4b-98a3-c8a57a74ed18-plugins-config-volume\") pod \"ba7fb594-e64c-4a4b-98a3-c8a57a74ed18\" (UID: \"ba7fb594-e64c-4a4b-98a3-c8a57a74ed18\") "
Sep 08 15:38:28.362192 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.362083    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df2c2386-37dc-462c-8c29-587d153f6bd7-maas-api-tls\") pod \"df2c2386-37dc-462c-8c29-587d153f6bd7\" (UID: \"df2c2386-37dc-462c-8c29-587d153f6bd7\") "
Sep 08 15:38:28.362192 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.362099    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42ssc\" (UniqueName: \"kubernetes.io/projected/1c2f38bd-9cc3-4731-8cc5-1be941f14b33-kube-api-access-42ssc\") pod \"1c2f38bd-9cc3-4731-8cc5-1be941f14b33\" (UID: \"1c2f38bd-9cc3-4731-8cc5-1be941f14b33\") "
Sep 08 15:38:28.362192 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.362117    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df2c2386-37dc-462c-8c29-587d153f6bd7-metrics-certs\") pod \"df2c2386-37dc-462c-8c29-587d153f6bd7\" (UID: \"df2c2386-37dc-462c-8c29-587d153f6bd7\") "
Sep 08 15:38:28.362192 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.362170    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c2f38bd-9cc3-4731-8cc5-1be941f14b33-plugins-config-volume\") pod \"1c2f38bd-9cc3-4731-8cc5-1be941f14b33\" (UID: \"1c2f38bd-9cc3-4731-8cc5-1be941f14b33\") "
Sep 08 15:38:28.362192 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.362193    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbs46\" (UniqueName: \"kubernetes.io/projected/ba7fb594-e64c-4a4b-98a3-c8a57a74ed18-kube-api-access-kbs46\") pod \"ba7fb594-e64c-4a4b-98a3-c8a57a74ed18\" (UID: \"ba7fb594-e64c-4a4b-98a3-c8a57a74ed18\") "
Sep 08 15:38:28.362603 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.362368    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df2c2386-37dc-462c-8c29-587d153f6bd7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "df2c2386-37dc-462c-8c29-587d153f6bd7" (UID: "df2c2386-37dc-462c-8c29-587d153f6bd7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:38:28.362603 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.362466    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba7fb594-e64c-4a4b-98a3-c8a57a74ed18-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ba7fb594-e64c-4a4b-98a3-c8a57a74ed18" (UID: "ba7fb594-e64c-4a4b-98a3-c8a57a74ed18"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:38:28.362745 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.362719    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c2f38bd-9cc3-4731-8cc5-1be941f14b33-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1c2f38bd-9cc3-4731-8cc5-1be941f14b33" (UID: "1c2f38bd-9cc3-4731-8cc5-1be941f14b33"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:38:28.362850 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.362832    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba7fb594-e64c-4a4b-98a3-c8a57a74ed18-plugins-config-volume\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:38:28.362915 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.362857    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c2f38bd-9cc3-4731-8cc5-1be941f14b33-plugins-config-volume\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:38:28.362915 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.362871    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df2c2386-37dc-462c-8c29-587d153f6bd7-openshift-service-ca\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:38:28.364378 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.364350    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c2f38bd-9cc3-4731-8cc5-1be941f14b33-kube-api-access-42ssc" (OuterVolumeSpecName: "kube-api-access-42ssc") pod "1c2f38bd-9cc3-4731-8cc5-1be941f14b33" (UID: "1c2f38bd-9cc3-4731-8cc5-1be941f14b33"). InnerVolumeSpecName "kube-api-access-42ssc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:38:28.364493 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.364378    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df2c2386-37dc-462c-8c29-587d153f6bd7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "df2c2386-37dc-462c-8c29-587d153f6bd7" (UID: "df2c2386-37dc-462c-8c29-587d153f6bd7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:38:28.364493 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.364400    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df2c2386-37dc-462c-8c29-587d153f6bd7-kube-api-access-vsmxw" (OuterVolumeSpecName: "kube-api-access-vsmxw") pod "df2c2386-37dc-462c-8c29-587d153f6bd7" (UID: "df2c2386-37dc-462c-8c29-587d153f6bd7"). InnerVolumeSpecName "kube-api-access-vsmxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:38:28.364689 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.364674    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df2c2386-37dc-462c-8c29-587d153f6bd7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "df2c2386-37dc-462c-8c29-587d153f6bd7" (UID: "df2c2386-37dc-462c-8c29-587d153f6bd7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:38:28.364822 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.364795    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba7fb594-e64c-4a4b-98a3-c8a57a74ed18-kube-api-access-kbs46" (OuterVolumeSpecName: "kube-api-access-kbs46") pod "ba7fb594-e64c-4a4b-98a3-c8a57a74ed18" (UID: "ba7fb594-e64c-4a4b-98a3-c8a57a74ed18"). InnerVolumeSpecName "kube-api-access-kbs46". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:38:28.463584 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.463546    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbs46\" (UniqueName: \"kubernetes.io/projected/ba7fb594-e64c-4a4b-98a3-c8a57a74ed18-kube-api-access-kbs46\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:38:28.463584 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.463577    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vsmxw\" (UniqueName: \"kubernetes.io/projected/df2c2386-37dc-462c-8c29-587d153f6bd7-kube-api-access-vsmxw\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:38:28.463584 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.463587    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df2c2386-37dc-462c-8c29-587d153f6bd7-maas-api-tls\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:38:28.463833 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.463596    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42ssc\" (UniqueName: \"kubernetes.io/projected/1c2f38bd-9cc3-4731-8cc5-1be941f14b33-kube-api-access-42ssc\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:38:28.463833 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.463605    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df2c2386-37dc-462c-8c29-587d153f6bd7-metrics-certs\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:38:28.899512 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.899477    2560 generic.go:356] "Generic (PLEG): container finished" podID="ba7fb594-e64c-4a4b-98a3-c8a57a74ed18" containerID="bf9b424aa9d8c90eb208694ec30a7fce37a5d6d2156be81a9264112b386bf64e" exitCode=0
Sep 08 15:38:28.899940 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.899525    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs"
Sep 08 15:38:28.899940 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.899557    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs" event={"ID":"ba7fb594-e64c-4a4b-98a3-c8a57a74ed18","Type":"ContainerDied","Data":"bf9b424aa9d8c90eb208694ec30a7fce37a5d6d2156be81a9264112b386bf64e"}
Sep 08 15:38:28.899940 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.899604    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs" event={"ID":"ba7fb594-e64c-4a4b-98a3-c8a57a74ed18","Type":"ContainerDied","Data":"3969795ee869779d6414d29c3f39f21e57692dea7b48871dc776b461ac66c8da"}
Sep 08 15:38:28.899940 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.899629    2560 scope.go:117] "RemoveContainer" containerID="bf9b424aa9d8c90eb208694ec30a7fce37a5d6d2156be81a9264112b386bf64e"
Sep 08 15:38:28.901001 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.900875    2560 generic.go:356] "Generic (PLEG): container finished" podID="1c2f38bd-9cc3-4731-8cc5-1be941f14b33" containerID="3ff7337e4de0c03779feb8195bd4483b3f1b5b705ccbbd18562f0ee3864a15e1" exitCode=0
Sep 08 15:38:28.901001 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.900926    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx"
Sep 08 15:38:28.901001 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.900957    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx" event={"ID":"1c2f38bd-9cc3-4731-8cc5-1be941f14b33","Type":"ContainerDied","Data":"3ff7337e4de0c03779feb8195bd4483b3f1b5b705ccbbd18562f0ee3864a15e1"}
Sep 08 15:38:28.901232 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.901004    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx" event={"ID":"1c2f38bd-9cc3-4731-8cc5-1be941f14b33","Type":"ContainerDied","Data":"5c19f0bf631ac176d2cab9c7fcda01f9f74d6b8c5ef685bfa246834f49aee0ac"}
Sep 08 15:38:28.902516 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.902489    2560 generic.go:356] "Generic (PLEG): container finished" podID="df2c2386-37dc-462c-8c29-587d153f6bd7" containerID="16f26f6ff9c6bc3f2f78251aa62b4ebdd0f8218b76518cb6802c0ada94e3e85a" exitCode=0
Sep 08 15:38:28.902634 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.902532    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s" event={"ID":"df2c2386-37dc-462c-8c29-587d153f6bd7","Type":"ContainerDied","Data":"16f26f6ff9c6bc3f2f78251aa62b4ebdd0f8218b76518cb6802c0ada94e3e85a"}
Sep 08 15:38:28.902634 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.902571    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s" event={"ID":"df2c2386-37dc-462c-8c29-587d153f6bd7","Type":"ContainerDied","Data":"9c6a476fb4496e6b3e9978cbfdb31f7e6a6a40c2d569dd96fba36a9d6d320509"}
Sep 08 15:38:28.902634 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.902538    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s"
Sep 08 15:38:28.910092 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.910055    2560 scope.go:117] "RemoveContainer" containerID="bf9b424aa9d8c90eb208694ec30a7fce37a5d6d2156be81a9264112b386bf64e"
Sep 08 15:38:28.910390 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:38:28.910371    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf9b424aa9d8c90eb208694ec30a7fce37a5d6d2156be81a9264112b386bf64e\": container with ID starting with bf9b424aa9d8c90eb208694ec30a7fce37a5d6d2156be81a9264112b386bf64e not found: ID does not exist" containerID="bf9b424aa9d8c90eb208694ec30a7fce37a5d6d2156be81a9264112b386bf64e"
Sep 08 15:38:28.910482 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.910400    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf9b424aa9d8c90eb208694ec30a7fce37a5d6d2156be81a9264112b386bf64e"} err="failed to get container status \"bf9b424aa9d8c90eb208694ec30a7fce37a5d6d2156be81a9264112b386bf64e\": rpc error: code = NotFound desc = could not find container \"bf9b424aa9d8c90eb208694ec30a7fce37a5d6d2156be81a9264112b386bf64e\": container with ID starting with bf9b424aa9d8c90eb208694ec30a7fce37a5d6d2156be81a9264112b386bf64e not found: ID does not exist"
Sep 08 15:38:28.910482 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.910422    2560 scope.go:117] "RemoveContainer" containerID="3ff7337e4de0c03779feb8195bd4483b3f1b5b705ccbbd18562f0ee3864a15e1"
Sep 08 15:38:28.919518 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.919500    2560 scope.go:117] "RemoveContainer" containerID="3ff7337e4de0c03779feb8195bd4483b3f1b5b705ccbbd18562f0ee3864a15e1"
Sep 08 15:38:28.919826 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:38:28.919809    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ff7337e4de0c03779feb8195bd4483b3f1b5b705ccbbd18562f0ee3864a15e1\": container with ID starting with 3ff7337e4de0c03779feb8195bd4483b3f1b5b705ccbbd18562f0ee3864a15e1 not found: ID does not exist" containerID="3ff7337e4de0c03779feb8195bd4483b3f1b5b705ccbbd18562f0ee3864a15e1"
Sep 08 15:38:28.919878 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.919833    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ff7337e4de0c03779feb8195bd4483b3f1b5b705ccbbd18562f0ee3864a15e1"} err="failed to get container status \"3ff7337e4de0c03779feb8195bd4483b3f1b5b705ccbbd18562f0ee3864a15e1\": rpc error: code = NotFound desc = could not find container \"3ff7337e4de0c03779feb8195bd4483b3f1b5b705ccbbd18562f0ee3864a15e1\": container with ID starting with 3ff7337e4de0c03779feb8195bd4483b3f1b5b705ccbbd18562f0ee3864a15e1 not found: ID does not exist"
Sep 08 15:38:28.919878 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.919850    2560 scope.go:117] "RemoveContainer" containerID="16f26f6ff9c6bc3f2f78251aa62b4ebdd0f8218b76518cb6802c0ada94e3e85a"
Sep 08 15:38:28.926678 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.926653    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs"]
Sep 08 15:38:28.929115 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.929096    2560 scope.go:117] "RemoveContainer" containerID="16f26f6ff9c6bc3f2f78251aa62b4ebdd0f8218b76518cb6802c0ada94e3e85a"
Sep 08 15:38:28.929467 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:38:28.929445    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16f26f6ff9c6bc3f2f78251aa62b4ebdd0f8218b76518cb6802c0ada94e3e85a\": container with ID starting with 16f26f6ff9c6bc3f2f78251aa62b4ebdd0f8218b76518cb6802c0ada94e3e85a not found: ID does not exist" containerID="16f26f6ff9c6bc3f2f78251aa62b4ebdd0f8218b76518cb6802c0ada94e3e85a"
Sep 08 15:38:28.929595 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.929476    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16f26f6ff9c6bc3f2f78251aa62b4ebdd0f8218b76518cb6802c0ada94e3e85a"} err="failed to get container status \"16f26f6ff9c6bc3f2f78251aa62b4ebdd0f8218b76518cb6802c0ada94e3e85a\": rpc error: code = NotFound desc = could not find container \"16f26f6ff9c6bc3f2f78251aa62b4ebdd0f8218b76518cb6802c0ada94e3e85a\": container with ID starting with 16f26f6ff9c6bc3f2f78251aa62b4ebdd0f8218b76518cb6802c0ada94e3e85a not found: ID does not exist"
Sep 08 15:38:28.930253 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.930235    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4e499427-6ff66f6db-f4nrs"]
Sep 08 15:38:28.940309 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.940285    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx"]
Sep 08 15:38:28.950724 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.950702    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4e499427-58b69954cc-8lxgx"]
Sep 08 15:38:28.961686 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.961660    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s"]
Sep 08 15:38:28.964604 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:28.964584    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4e499427-548c9ddf76-j7v7s"]
Sep 08 15:38:29.799942 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:29.799906    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c2f38bd-9cc3-4731-8cc5-1be941f14b33" path="/var/lib/kubelet/pods/1c2f38bd-9cc3-4731-8cc5-1be941f14b33/volumes"
Sep 08 15:38:29.800410 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:29.800386    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba7fb594-e64c-4a4b-98a3-c8a57a74ed18" path="/var/lib/kubelet/pods/ba7fb594-e64c-4a4b-98a3-c8a57a74ed18/volumes"
Sep 08 15:38:29.800815 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:29.800797    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df2c2386-37dc-462c-8c29-587d153f6bd7" path="/var/lib/kubelet/pods/df2c2386-37dc-462c-8c29-587d153f6bd7/volumes"
Sep 08 15:38:31.347856 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:31.347823    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:36.398205 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:38:36.398171    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:06.354902 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:39:06.354858    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:06.416572 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:39:06.416528    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:13.378018 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:39:13.377983    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:13.564841 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:39:13.564808    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:16.852396 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:39:16.852347    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:17.820144 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:39:17.820106    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:19.489212 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:39:19.489172    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:19.595907 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:39:19.595857    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:19.761688 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:39:19.761591    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:21.063482 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:39:21.063449    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:21.160515 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:39:21.160477    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:21.270260 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:39:21.270209    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:21.870166 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:39:21.870128    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:21.967554 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:39:21.967518    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:22.070763 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:39:22.070722    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:23.312554 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:39:23.312523    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:29.292094 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:39:29.292040    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:29.391291 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:39:29.391257    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:29.981107 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:39:29.981051    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:34.171927 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:39:34.171892    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:36.383858 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:39:36.383821    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:36.465313 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:39:36.465272    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:36.596127 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:39:36.596091    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:36.679084 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:39:36.679044    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:47.211691 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:39:47.211658    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:47.459854 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:39:47.459819    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:08.407433 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:08.407394    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:14.336263 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:14.336222    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:14.436885 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:14.436849    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:14.692824 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:14.692790    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:15.001198 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:15.001104    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:15.131958 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:15.131929    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:17.985673 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:17.985602    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:18.072781 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:18.072735    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:19.123709 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.123671    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj"]
Sep 08 15:40:19.124133 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.124118    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c2f38bd-9cc3-4731-8cc5-1be941f14b33" containerName="payload-processing"
Sep 08 15:40:19.124185 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.124134    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c2f38bd-9cc3-4731-8cc5-1be941f14b33" containerName="payload-processing"
Sep 08 15:40:19.124185 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.124155    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df2c2386-37dc-462c-8c29-587d153f6bd7" containerName="maas-api"
Sep 08 15:40:19.124185 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.124164    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="df2c2386-37dc-462c-8c29-587d153f6bd7" containerName="maas-api"
Sep 08 15:40:19.124185 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.124172    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba7fb594-e64c-4a4b-98a3-c8a57a74ed18" containerName="payload-pre-processing"
Sep 08 15:40:19.124185 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.124177    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba7fb594-e64c-4a4b-98a3-c8a57a74ed18" containerName="payload-pre-processing"
Sep 08 15:40:19.124340 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.124246    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="df2c2386-37dc-462c-8c29-587d153f6bd7" containerName="maas-api"
Sep 08 15:40:19.124340 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.124259    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c2f38bd-9cc3-4731-8cc5-1be941f14b33" containerName="payload-processing"
Sep 08 15:40:19.124340 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.124265    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba7fb594-e64c-4a4b-98a3-c8a57a74ed18" containerName="payload-pre-processing"
Sep 08 15:40:19.127726 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.127705    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj"
Sep 08 15:40:19.129685 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.129664    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-630e4d-serving-cert\""
Sep 08 15:40:19.138552 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.138530    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj"]
Sep 08 15:40:19.155399 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.155372    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:19.262618 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.262585    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:19.268811 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.268784    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31fcda52-b9ff-4ab7-820e-1055c4fc0baa-maas-api-tls\") pod \"maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj\" (UID: \"31fcda52-b9ff-4ab7-820e-1055c4fc0baa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj"
Sep 08 15:40:19.268937 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.268816    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31fcda52-b9ff-4ab7-820e-1055c4fc0baa-metrics-certs\") pod \"maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj\" (UID: \"31fcda52-b9ff-4ab7-820e-1055c4fc0baa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj"
Sep 08 15:40:19.268937 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.268844    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9tcc\" (UniqueName: \"kubernetes.io/projected/31fcda52-b9ff-4ab7-820e-1055c4fc0baa-kube-api-access-x9tcc\") pod \"maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj\" (UID: \"31fcda52-b9ff-4ab7-820e-1055c4fc0baa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj"
Sep 08 15:40:19.268937 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.268884    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31fcda52-b9ff-4ab7-820e-1055c4fc0baa-openshift-service-ca\") pod \"maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj\" (UID: \"31fcda52-b9ff-4ab7-820e-1055c4fc0baa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj"
Sep 08 15:40:19.356348 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.356310    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:19.369499 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.369459    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31fcda52-b9ff-4ab7-820e-1055c4fc0baa-openshift-service-ca\") pod \"maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj\" (UID: \"31fcda52-b9ff-4ab7-820e-1055c4fc0baa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj"
Sep 08 15:40:19.369654 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.369579    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31fcda52-b9ff-4ab7-820e-1055c4fc0baa-maas-api-tls\") pod \"maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj\" (UID: \"31fcda52-b9ff-4ab7-820e-1055c4fc0baa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj"
Sep 08 15:40:19.369654 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.369603    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31fcda52-b9ff-4ab7-820e-1055c4fc0baa-metrics-certs\") pod \"maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj\" (UID: \"31fcda52-b9ff-4ab7-820e-1055c4fc0baa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj"
Sep 08 15:40:19.369654 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.369629    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9tcc\" (UniqueName: \"kubernetes.io/projected/31fcda52-b9ff-4ab7-820e-1055c4fc0baa-kube-api-access-x9tcc\") pod \"maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj\" (UID: \"31fcda52-b9ff-4ab7-820e-1055c4fc0baa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj"
Sep 08 15:40:19.370205 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.370181    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31fcda52-b9ff-4ab7-820e-1055c4fc0baa-openshift-service-ca\") pod \"maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj\" (UID: \"31fcda52-b9ff-4ab7-820e-1055c4fc0baa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj"
Sep 08 15:40:19.372110 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.372088    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31fcda52-b9ff-4ab7-820e-1055c4fc0baa-maas-api-tls\") pod \"maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj\" (UID: \"31fcda52-b9ff-4ab7-820e-1055c4fc0baa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj"
Sep 08 15:40:19.372215 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.372125    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31fcda52-b9ff-4ab7-820e-1055c4fc0baa-metrics-certs\") pod \"maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj\" (UID: \"31fcda52-b9ff-4ab7-820e-1055c4fc0baa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj"
Sep 08 15:40:19.379716 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.379656    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9tcc\" (UniqueName: \"kubernetes.io/projected/31fcda52-b9ff-4ab7-820e-1055c4fc0baa-kube-api-access-x9tcc\") pod \"maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj\" (UID: \"31fcda52-b9ff-4ab7-820e-1055c4fc0baa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj"
Sep 08 15:40:19.439655 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.439621    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj"
Sep 08 15:40:19.717578 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.717551    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj"]
Sep 08 15:40:19.719757 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:40:19.719719    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31fcda52_b9ff_4ab7_820e_1055c4fc0baa.slice/crio-edf795d1b72d6e15f60043314e4872aa13f4925d0fcc58ea75f09b52a87d46b4 WatchSource:0}: Error finding container edf795d1b72d6e15f60043314e4872aa13f4925d0fcc58ea75f09b52a87d46b4: Status 404 returned error can't find the container with id edf795d1b72d6e15f60043314e4872aa13f4925d0fcc58ea75f09b52a87d46b4
Sep 08 15:40:19.789558 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.789522    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7"]
Sep 08 15:40:19.794409 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.794383    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7"
Sep 08 15:40:19.796409 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.796388    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-630e4d\""
Sep 08 15:40:19.796513 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.796388    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-630e4d-dockercfg-fm7bn\""
Sep 08 15:40:19.805261 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.805238    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7"]
Sep 08 15:40:19.822780 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.822749    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7"]
Sep 08 15:40:19.825852 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.825789    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:19.827321 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.827299    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7"
Sep 08 15:40:19.829284 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.829262    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7"]
Sep 08 15:40:19.924871 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.924830    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:19.976124 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.976007    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57ea2742-18d0-4ef6-836a-1c407684916a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7\" (UID: \"57ea2742-18d0-4ef6-836a-1c407684916a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7"
Sep 08 15:40:19.976124 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.976064    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f4e4551-9695-4108-a670-bf8e9d52f9bf-plugins-config-volume\") pod \"payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7\" (UID: \"4f4e4551-9695-4108-a670-bf8e9d52f9bf\") " pod="openshift-ingress/payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7"
Sep 08 15:40:19.976360 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.976180    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njsn4\" (UniqueName: \"kubernetes.io/projected/57ea2742-18d0-4ef6-836a-1c407684916a-kube-api-access-njsn4\") pod \"payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7\" (UID: \"57ea2742-18d0-4ef6-836a-1c407684916a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7"
Sep 08 15:40:19.976360 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:19.976283    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s59tr\" (UniqueName: \"kubernetes.io/projected/4f4e4551-9695-4108-a670-bf8e9d52f9bf-kube-api-access-s59tr\") pod \"payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7\" (UID: \"4f4e4551-9695-4108-a670-bf8e9d52f9bf\") " pod="openshift-ingress/payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7"
Sep 08 15:40:20.077888 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:20.077848    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57ea2742-18d0-4ef6-836a-1c407684916a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7\" (UID: \"57ea2742-18d0-4ef6-836a-1c407684916a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7"
Sep 08 15:40:20.078103 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:20.077903    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f4e4551-9695-4108-a670-bf8e9d52f9bf-plugins-config-volume\") pod \"payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7\" (UID: \"4f4e4551-9695-4108-a670-bf8e9d52f9bf\") " pod="openshift-ingress/payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7"
Sep 08 15:40:20.078103 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:20.077948    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njsn4\" (UniqueName: \"kubernetes.io/projected/57ea2742-18d0-4ef6-836a-1c407684916a-kube-api-access-njsn4\") pod \"payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7\" (UID: \"57ea2742-18d0-4ef6-836a-1c407684916a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7"
Sep 08 15:40:20.078103 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:20.077994    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s59tr\" (UniqueName: \"kubernetes.io/projected/4f4e4551-9695-4108-a670-bf8e9d52f9bf-kube-api-access-s59tr\") pod \"payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7\" (UID: \"4f4e4551-9695-4108-a670-bf8e9d52f9bf\") " pod="openshift-ingress/payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7"
Sep 08 15:40:20.078865 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:20.078833    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f4e4551-9695-4108-a670-bf8e9d52f9bf-plugins-config-volume\") pod \"payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7\" (UID: \"4f4e4551-9695-4108-a670-bf8e9d52f9bf\") " pod="openshift-ingress/payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7"
Sep 08 15:40:20.078959 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:20.078846    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57ea2742-18d0-4ef6-836a-1c407684916a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7\" (UID: \"57ea2742-18d0-4ef6-836a-1c407684916a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7"
Sep 08 15:40:20.087429 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:20.087406    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s59tr\" (UniqueName: \"kubernetes.io/projected/4f4e4551-9695-4108-a670-bf8e9d52f9bf-kube-api-access-s59tr\") pod \"payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7\" (UID: \"4f4e4551-9695-4108-a670-bf8e9d52f9bf\") " pod="openshift-ingress/payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7"
Sep 08 15:40:20.087572 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:20.087516    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njsn4\" (UniqueName: \"kubernetes.io/projected/57ea2742-18d0-4ef6-836a-1c407684916a-kube-api-access-njsn4\") pod \"payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7\" (UID: \"57ea2742-18d0-4ef6-836a-1c407684916a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7"
Sep 08 15:40:20.108383 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:20.108348    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7"
Sep 08 15:40:20.138991 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:20.138759    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7"
Sep 08 15:40:20.394395 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:20.394366    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7"]
Sep 08 15:40:20.397065 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:40:20.397037    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57ea2742_18d0_4ef6_836a_1c407684916a.slice/crio-d5a4937c840ee2b4bbb63168d5356fd532f54225243f5fb4e3132c0150a001ca WatchSource:0}: Error finding container d5a4937c840ee2b4bbb63168d5356fd532f54225243f5fb4e3132c0150a001ca: Status 404 returned error can't find the container with id d5a4937c840ee2b4bbb63168d5356fd532f54225243f5fb4e3132c0150a001ca
Sep 08 15:40:20.424623 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:20.424588    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7"]
Sep 08 15:40:20.436319 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:40:20.436295    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f4e4551_9695_4108_a670_bf8e9d52f9bf.slice/crio-e2122af395c3340f60b11474a0b3d203e6b0d1c7e7d5c157c15d058e02111d2a WatchSource:0}: Error finding container e2122af395c3340f60b11474a0b3d203e6b0d1c7e7d5c157c15d058e02111d2a: Status 404 returned error can't find the container with id e2122af395c3340f60b11474a0b3d203e6b0d1c7e7d5c157c15d058e02111d2a
Sep 08 15:40:20.485322 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:20.485291    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7" event={"ID":"57ea2742-18d0-4ef6-836a-1c407684916a","Type":"ContainerStarted","Data":"d5a4937c840ee2b4bbb63168d5356fd532f54225243f5fb4e3132c0150a001ca"}
Sep 08 15:40:20.487121 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:20.487055    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7" event={"ID":"4f4e4551-9695-4108-a670-bf8e9d52f9bf","Type":"ContainerStarted","Data":"e2122af395c3340f60b11474a0b3d203e6b0d1c7e7d5c157c15d058e02111d2a"}
Sep 08 15:40:20.489282 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:20.489257    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj" event={"ID":"31fcda52-b9ff-4ab7-820e-1055c4fc0baa","Type":"ContainerStarted","Data":"bdb3d6d56a0e265baf219421e3650af37976d5100f9b40a0b565d9dbf43aa647"}
Sep 08 15:40:20.489422 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:20.489308    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj" event={"ID":"31fcda52-b9ff-4ab7-820e-1055c4fc0baa","Type":"ContainerStarted","Data":"edf795d1b72d6e15f60043314e4872aa13f4925d0fcc58ea75f09b52a87d46b4"}
Sep 08 15:40:20.489422 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:20.489402    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj"
Sep 08 15:40:20.509266 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:20.509208    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj" podStartSLOduration=0.977015793 podStartE2EDuration="1.509190638s" totalImagesPullingTime="532.174845ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:40:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:40:20.507922051 +0000 UTC m=+1373.219864546" watchObservedRunningTime="2026-09-08 15:40:20.509190638 +0000 UTC m=+1373.221133128"
Sep 08 15:40:21.496600 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:21.496565    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7" event={"ID":"57ea2742-18d0-4ef6-836a-1c407684916a","Type":"ContainerStarted","Data":"679bfe5719914dca652904cce7d4d75126ea95dfd81023e25b90166dbcb52a5e"}
Sep 08 15:40:21.497044 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:21.496629    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7"
Sep 08 15:40:21.498191 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:21.498160    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7" event={"ID":"4f4e4551-9695-4108-a670-bf8e9d52f9bf","Type":"ContainerStarted","Data":"75a83bc31cbf44588c41ddf601afdffe1e5368146bdbceea7e9fdd2ac1dbf5ed"}
Sep 08 15:40:21.516795 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:21.516734    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7" podStartSLOduration=2.516718653 podStartE2EDuration="2.516718653s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:40:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:40:21.514196554 +0000 UTC m=+1374.226139045" watchObservedRunningTime="2026-09-08 15:40:21.516718653 +0000 UTC m=+1374.228661171"
Sep 08 15:40:21.532469 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:21.532408    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7" podStartSLOduration=2.532394002 podStartE2EDuration="2.532394002s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:40:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:40:21.531599155 +0000 UTC m=+1374.243541647" watchObservedRunningTime="2026-09-08 15:40:21.532394002 +0000 UTC m=+1374.244336489"
Sep 08 15:40:22.503874 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:22.503840    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7"
Sep 08 15:40:25.241673 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:25.241634    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:26.504037 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:26.503996    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj"
Sep 08 15:40:32.506982 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:32.506945    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7"
Sep 08 15:40:33.511972 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:33.511940    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7"
Sep 08 15:40:42.924044 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:42.923998    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:43.369967 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:43.369880    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:43.546246 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:43.546210    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:43.734123 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:43.734085    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:43.825087 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:43.825031    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:47.255700 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:47.255671    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:47.377725 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:47.377671    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:48.385614 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:48.385578    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7"]
Sep 08 15:40:48.389424 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:48.389402    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7"
Sep 08 15:40:48.391935 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:48.391912    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-bea11f-serving-cert\""
Sep 08 15:40:48.409610 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:48.409585    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7"]
Sep 08 15:40:48.434927 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:48.434891    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:48.527090 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:48.527032    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:48.544329 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:48.544292    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l54zb\" (UniqueName: \"kubernetes.io/projected/ee3c39b6-f4c7-4d81-90a5-75526fecc6fa-kube-api-access-l54zb\") pod \"maas-api-e2e-api-b-bea11f-6697559944-ztsx7\" (UID: \"ee3c39b6-f4c7-4d81-90a5-75526fecc6fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7"
Sep 08 15:40:48.544520 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:48.544427    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee3c39b6-f4c7-4d81-90a5-75526fecc6fa-maas-api-tls\") pod \"maas-api-e2e-api-b-bea11f-6697559944-ztsx7\" (UID: \"ee3c39b6-f4c7-4d81-90a5-75526fecc6fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7"
Sep 08 15:40:48.544520 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:48.544486    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee3c39b6-f4c7-4d81-90a5-75526fecc6fa-metrics-certs\") pod \"maas-api-e2e-api-b-bea11f-6697559944-ztsx7\" (UID: \"ee3c39b6-f4c7-4d81-90a5-75526fecc6fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7"
Sep 08 15:40:48.544645 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:48.544589    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee3c39b6-f4c7-4d81-90a5-75526fecc6fa-openshift-service-ca\") pod \"maas-api-e2e-api-b-bea11f-6697559944-ztsx7\" (UID: \"ee3c39b6-f4c7-4d81-90a5-75526fecc6fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7"
Sep 08 15:40:48.645190 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:48.645095    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee3c39b6-f4c7-4d81-90a5-75526fecc6fa-openshift-service-ca\") pod \"maas-api-e2e-api-b-bea11f-6697559944-ztsx7\" (UID: \"ee3c39b6-f4c7-4d81-90a5-75526fecc6fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7"
Sep 08 15:40:48.645368 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:48.645200    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l54zb\" (UniqueName: \"kubernetes.io/projected/ee3c39b6-f4c7-4d81-90a5-75526fecc6fa-kube-api-access-l54zb\") pod \"maas-api-e2e-api-b-bea11f-6697559944-ztsx7\" (UID: \"ee3c39b6-f4c7-4d81-90a5-75526fecc6fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7"
Sep 08 15:40:48.645368 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:48.645250    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee3c39b6-f4c7-4d81-90a5-75526fecc6fa-maas-api-tls\") pod \"maas-api-e2e-api-b-bea11f-6697559944-ztsx7\" (UID: \"ee3c39b6-f4c7-4d81-90a5-75526fecc6fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7"
Sep 08 15:40:48.645368 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:48.645294    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee3c39b6-f4c7-4d81-90a5-75526fecc6fa-metrics-certs\") pod \"maas-api-e2e-api-b-bea11f-6697559944-ztsx7\" (UID: \"ee3c39b6-f4c7-4d81-90a5-75526fecc6fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7"
Sep 08 15:40:48.645882 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:48.645838    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee3c39b6-f4c7-4d81-90a5-75526fecc6fa-openshift-service-ca\") pod \"maas-api-e2e-api-b-bea11f-6697559944-ztsx7\" (UID: \"ee3c39b6-f4c7-4d81-90a5-75526fecc6fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7"
Sep 08 15:40:48.647887 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:48.647860    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee3c39b6-f4c7-4d81-90a5-75526fecc6fa-metrics-certs\") pod \"maas-api-e2e-api-b-bea11f-6697559944-ztsx7\" (UID: \"ee3c39b6-f4c7-4d81-90a5-75526fecc6fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7"
Sep 08 15:40:48.648009 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:48.647923    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee3c39b6-f4c7-4d81-90a5-75526fecc6fa-maas-api-tls\") pod \"maas-api-e2e-api-b-bea11f-6697559944-ztsx7\" (UID: \"ee3c39b6-f4c7-4d81-90a5-75526fecc6fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7"
Sep 08 15:40:48.655359 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:48.655334    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l54zb\" (UniqueName: \"kubernetes.io/projected/ee3c39b6-f4c7-4d81-90a5-75526fecc6fa-kube-api-access-l54zb\") pod \"maas-api-e2e-api-b-bea11f-6697559944-ztsx7\" (UID: \"ee3c39b6-f4c7-4d81-90a5-75526fecc6fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7"
Sep 08 15:40:48.683923 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:48.683900    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:48.699081 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:48.699053    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7"
Sep 08 15:40:48.995667 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:48.994693    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7"]
Sep 08 15:40:48.999319 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:40:48.999138    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee3c39b6_f4c7_4d81_90a5_75526fecc6fa.slice/crio-0363f129d8681298919bf087a16fb00d349eb7bb362203e96c843bdb4287d1c7 WatchSource:0}: Error finding container 0363f129d8681298919bf087a16fb00d349eb7bb362203e96c843bdb4287d1c7: Status 404 returned error can't find the container with id 0363f129d8681298919bf087a16fb00d349eb7bb362203e96c843bdb4287d1c7
Sep 08 15:40:49.000769 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:49.000745    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 15:40:49.034451 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:49.034420    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:49.134283 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:49.134252    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:49.655185 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:49.655097    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7" event={"ID":"ee3c39b6-f4c7-4d81-90a5-75526fecc6fa","Type":"ContainerStarted","Data":"bb4f98129d946148b200bce42ced80162e2e051b6204ba1cf7afea4a36d3ad26"}
Sep 08 15:40:49.655630 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:49.655197    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7" event={"ID":"ee3c39b6-f4c7-4d81-90a5-75526fecc6fa","Type":"ContainerStarted","Data":"0363f129d8681298919bf087a16fb00d349eb7bb362203e96c843bdb4287d1c7"}
Sep 08 15:40:49.655630 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:49.655228    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7"
Sep 08 15:40:49.675086 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:49.675010    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7" podStartSLOduration=1.300306794 podStartE2EDuration="1.674996589s" totalImagesPullingTime="374.689795ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:40:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:40:49.673719788 +0000 UTC m=+1402.385662278" watchObservedRunningTime="2026-09-08 15:40:49.674996589 +0000 UTC m=+1402.386939079"
Sep 08 15:40:55.666918 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:40:55.666884    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7"
Sep 08 15:41:08.646405 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:08.646369    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:09.470111 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:09.470079    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:12.093563 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:12.093495    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:13.088551 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:13.088518    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:15.354174 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:15.354136    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:26.166680 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:26.166602    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:27.303985 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:27.303953    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:30.858810 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:30.858554    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj"]
Sep 08 15:41:30.859330 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:30.858810    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj" podUID="31fcda52-b9ff-4ab7-820e-1055c4fc0baa" containerName="maas-api" containerID="cri-o://bdb3d6d56a0e265baf219421e3650af37976d5100f9b40a0b565d9dbf43aa647" gracePeriod=30
Sep 08 15:41:30.922780 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:30.922734    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7"]
Sep 08 15:41:30.923493 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:30.923411    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7" podUID="4f4e4551-9695-4108-a670-bf8e9d52f9bf" containerName="payload-processing" containerID="cri-o://75a83bc31cbf44588c41ddf601afdffe1e5368146bdbceea7e9fdd2ac1dbf5ed" gracePeriod=30
Sep 08 15:41:30.933834 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:30.933619    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7"]
Sep 08 15:41:30.933978 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:30.933945    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7" podUID="57ea2742-18d0-4ef6-836a-1c407684916a" containerName="payload-pre-processing" containerID="cri-o://679bfe5719914dca652904cce7d4d75126ea95dfd81023e25b90166dbcb52a5e" gracePeriod=30
Sep 08 15:41:31.140594 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.140508    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:31.203510 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.200257    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:31.213646 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.213619    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj"
Sep 08 15:41:31.252851 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.252826    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7"
Sep 08 15:41:31.255521 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.255499    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7"
Sep 08 15:41:31.350680 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.350640    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57ea2742-18d0-4ef6-836a-1c407684916a-plugins-config-volume\") pod \"57ea2742-18d0-4ef6-836a-1c407684916a\" (UID: \"57ea2742-18d0-4ef6-836a-1c407684916a\") "
Sep 08 15:41:31.350870 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.350688    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9tcc\" (UniqueName: \"kubernetes.io/projected/31fcda52-b9ff-4ab7-820e-1055c4fc0baa-kube-api-access-x9tcc\") pod \"31fcda52-b9ff-4ab7-820e-1055c4fc0baa\" (UID: \"31fcda52-b9ff-4ab7-820e-1055c4fc0baa\") "
Sep 08 15:41:31.350870 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.350720    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s59tr\" (UniqueName: \"kubernetes.io/projected/4f4e4551-9695-4108-a670-bf8e9d52f9bf-kube-api-access-s59tr\") pod \"4f4e4551-9695-4108-a670-bf8e9d52f9bf\" (UID: \"4f4e4551-9695-4108-a670-bf8e9d52f9bf\") "
Sep 08 15:41:31.350870 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.350791    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31fcda52-b9ff-4ab7-820e-1055c4fc0baa-maas-api-tls\") pod \"31fcda52-b9ff-4ab7-820e-1055c4fc0baa\" (UID: \"31fcda52-b9ff-4ab7-820e-1055c4fc0baa\") "
Sep 08 15:41:31.350870 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.350837    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f4e4551-9695-4108-a670-bf8e9d52f9bf-plugins-config-volume\") pod \"4f4e4551-9695-4108-a670-bf8e9d52f9bf\" (UID: \"4f4e4551-9695-4108-a670-bf8e9d52f9bf\") "
Sep 08 15:41:31.351112 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.350881    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njsn4\" (UniqueName: \"kubernetes.io/projected/57ea2742-18d0-4ef6-836a-1c407684916a-kube-api-access-njsn4\") pod \"57ea2742-18d0-4ef6-836a-1c407684916a\" (UID: \"57ea2742-18d0-4ef6-836a-1c407684916a\") "
Sep 08 15:41:31.351112 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.350949    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31fcda52-b9ff-4ab7-820e-1055c4fc0baa-metrics-certs\") pod \"31fcda52-b9ff-4ab7-820e-1055c4fc0baa\" (UID: \"31fcda52-b9ff-4ab7-820e-1055c4fc0baa\") "
Sep 08 15:41:31.351112 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.351006    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31fcda52-b9ff-4ab7-820e-1055c4fc0baa-openshift-service-ca\") pod \"31fcda52-b9ff-4ab7-820e-1055c4fc0baa\" (UID: \"31fcda52-b9ff-4ab7-820e-1055c4fc0baa\") "
Sep 08 15:41:31.351267 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.351136    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57ea2742-18d0-4ef6-836a-1c407684916a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "57ea2742-18d0-4ef6-836a-1c407684916a" (UID: "57ea2742-18d0-4ef6-836a-1c407684916a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:41:31.351377 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.351355    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57ea2742-18d0-4ef6-836a-1c407684916a-plugins-config-volume\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:41:31.351615 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.351557    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f4e4551-9695-4108-a670-bf8e9d52f9bf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4f4e4551-9695-4108-a670-bf8e9d52f9bf" (UID: "4f4e4551-9695-4108-a670-bf8e9d52f9bf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:41:31.351707 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.351644    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31fcda52-b9ff-4ab7-820e-1055c4fc0baa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "31fcda52-b9ff-4ab7-820e-1055c4fc0baa" (UID: "31fcda52-b9ff-4ab7-820e-1055c4fc0baa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:41:31.353567 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.353539    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31fcda52-b9ff-4ab7-820e-1055c4fc0baa-kube-api-access-x9tcc" (OuterVolumeSpecName: "kube-api-access-x9tcc") pod "31fcda52-b9ff-4ab7-820e-1055c4fc0baa" (UID: "31fcda52-b9ff-4ab7-820e-1055c4fc0baa"). InnerVolumeSpecName "kube-api-access-x9tcc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:41:31.353676 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.353577    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57ea2742-18d0-4ef6-836a-1c407684916a-kube-api-access-njsn4" (OuterVolumeSpecName: "kube-api-access-njsn4") pod "57ea2742-18d0-4ef6-836a-1c407684916a" (UID: "57ea2742-18d0-4ef6-836a-1c407684916a"). InnerVolumeSpecName "kube-api-access-njsn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:41:31.353676 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.353601    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f4e4551-9695-4108-a670-bf8e9d52f9bf-kube-api-access-s59tr" (OuterVolumeSpecName: "kube-api-access-s59tr") pod "4f4e4551-9695-4108-a670-bf8e9d52f9bf" (UID: "4f4e4551-9695-4108-a670-bf8e9d52f9bf"). InnerVolumeSpecName "kube-api-access-s59tr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:41:31.353676 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.353662    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31fcda52-b9ff-4ab7-820e-1055c4fc0baa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "31fcda52-b9ff-4ab7-820e-1055c4fc0baa" (UID: "31fcda52-b9ff-4ab7-820e-1055c4fc0baa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:41:31.353810 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.353772    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31fcda52-b9ff-4ab7-820e-1055c4fc0baa-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "31fcda52-b9ff-4ab7-820e-1055c4fc0baa" (UID: "31fcda52-b9ff-4ab7-820e-1055c4fc0baa"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:41:31.452096 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.452051    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31fcda52-b9ff-4ab7-820e-1055c4fc0baa-maas-api-tls\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:41:31.452096 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.452095    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f4e4551-9695-4108-a670-bf8e9d52f9bf-plugins-config-volume\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:41:31.452299 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.452109    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njsn4\" (UniqueName: \"kubernetes.io/projected/57ea2742-18d0-4ef6-836a-1c407684916a-kube-api-access-njsn4\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:41:31.452299 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.452119    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31fcda52-b9ff-4ab7-820e-1055c4fc0baa-metrics-certs\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:41:31.452299 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.452127    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31fcda52-b9ff-4ab7-820e-1055c4fc0baa-openshift-service-ca\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:41:31.452299 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.452135    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9tcc\" (UniqueName: \"kubernetes.io/projected/31fcda52-b9ff-4ab7-820e-1055c4fc0baa-kube-api-access-x9tcc\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:41:31.452299 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.452143    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s59tr\" (UniqueName: \"kubernetes.io/projected/4f4e4551-9695-4108-a670-bf8e9d52f9bf-kube-api-access-s59tr\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:41:31.467019 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.466964    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:31.877502 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.877455    2560 generic.go:356] "Generic (PLEG): container finished" podID="57ea2742-18d0-4ef6-836a-1c407684916a" containerID="679bfe5719914dca652904cce7d4d75126ea95dfd81023e25b90166dbcb52a5e" exitCode=0
Sep 08 15:41:31.877861 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.877521    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7"
Sep 08 15:41:31.877861 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.877530    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7" event={"ID":"57ea2742-18d0-4ef6-836a-1c407684916a","Type":"ContainerDied","Data":"679bfe5719914dca652904cce7d4d75126ea95dfd81023e25b90166dbcb52a5e"}
Sep 08 15:41:31.877861 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.877609    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7" event={"ID":"57ea2742-18d0-4ef6-836a-1c407684916a","Type":"ContainerDied","Data":"d5a4937c840ee2b4bbb63168d5356fd532f54225243f5fb4e3132c0150a001ca"}
Sep 08 15:41:31.877861 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.877639    2560 scope.go:117] "RemoveContainer" containerID="679bfe5719914dca652904cce7d4d75126ea95dfd81023e25b90166dbcb52a5e"
Sep 08 15:41:31.879116 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.879061    2560 generic.go:356] "Generic (PLEG): container finished" podID="4f4e4551-9695-4108-a670-bf8e9d52f9bf" containerID="75a83bc31cbf44588c41ddf601afdffe1e5368146bdbceea7e9fdd2ac1dbf5ed" exitCode=0
Sep 08 15:41:31.879195 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.879123    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7"
Sep 08 15:41:31.879195 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.879161    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7" event={"ID":"4f4e4551-9695-4108-a670-bf8e9d52f9bf","Type":"ContainerDied","Data":"75a83bc31cbf44588c41ddf601afdffe1e5368146bdbceea7e9fdd2ac1dbf5ed"}
Sep 08 15:41:31.879309 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.879222    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7" event={"ID":"4f4e4551-9695-4108-a670-bf8e9d52f9bf","Type":"ContainerDied","Data":"e2122af395c3340f60b11474a0b3d203e6b0d1c7e7d5c157c15d058e02111d2a"}
Sep 08 15:41:31.880765 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.880738    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj"
Sep 08 15:41:31.880765 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.880761    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj" event={"ID":"31fcda52-b9ff-4ab7-820e-1055c4fc0baa","Type":"ContainerDied","Data":"bdb3d6d56a0e265baf219421e3650af37976d5100f9b40a0b565d9dbf43aa647"}
Sep 08 15:41:31.880911 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.880743    2560 generic.go:356] "Generic (PLEG): container finished" podID="31fcda52-b9ff-4ab7-820e-1055c4fc0baa" containerID="bdb3d6d56a0e265baf219421e3650af37976d5100f9b40a0b565d9dbf43aa647" exitCode=0
Sep 08 15:41:31.880972 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.880915    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj" event={"ID":"31fcda52-b9ff-4ab7-820e-1055c4fc0baa","Type":"ContainerDied","Data":"edf795d1b72d6e15f60043314e4872aa13f4925d0fcc58ea75f09b52a87d46b4"}
Sep 08 15:41:31.890708 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.890693    2560 scope.go:117] "RemoveContainer" containerID="679bfe5719914dca652904cce7d4d75126ea95dfd81023e25b90166dbcb52a5e"
Sep 08 15:41:31.890977 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:41:31.890960    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"679bfe5719914dca652904cce7d4d75126ea95dfd81023e25b90166dbcb52a5e\": container with ID starting with 679bfe5719914dca652904cce7d4d75126ea95dfd81023e25b90166dbcb52a5e not found: ID does not exist" containerID="679bfe5719914dca652904cce7d4d75126ea95dfd81023e25b90166dbcb52a5e"
Sep 08 15:41:31.891018 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.890998    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"679bfe5719914dca652904cce7d4d75126ea95dfd81023e25b90166dbcb52a5e"} err="failed to get container status \"679bfe5719914dca652904cce7d4d75126ea95dfd81023e25b90166dbcb52a5e\": rpc error: code = NotFound desc = could not find container \"679bfe5719914dca652904cce7d4d75126ea95dfd81023e25b90166dbcb52a5e\": container with ID starting with 679bfe5719914dca652904cce7d4d75126ea95dfd81023e25b90166dbcb52a5e not found: ID does not exist"
Sep 08 15:41:31.891061 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.891016    2560 scope.go:117] "RemoveContainer" containerID="75a83bc31cbf44588c41ddf601afdffe1e5368146bdbceea7e9fdd2ac1dbf5ed"
Sep 08 15:41:31.898904 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.898879    2560 scope.go:117] "RemoveContainer" containerID="75a83bc31cbf44588c41ddf601afdffe1e5368146bdbceea7e9fdd2ac1dbf5ed"
Sep 08 15:41:31.899166 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:41:31.899138    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75a83bc31cbf44588c41ddf601afdffe1e5368146bdbceea7e9fdd2ac1dbf5ed\": container with ID starting with 75a83bc31cbf44588c41ddf601afdffe1e5368146bdbceea7e9fdd2ac1dbf5ed not found: ID does not exist" containerID="75a83bc31cbf44588c41ddf601afdffe1e5368146bdbceea7e9fdd2ac1dbf5ed"
Sep 08 15:41:31.899215 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.899164    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75a83bc31cbf44588c41ddf601afdffe1e5368146bdbceea7e9fdd2ac1dbf5ed"} err="failed to get container status \"75a83bc31cbf44588c41ddf601afdffe1e5368146bdbceea7e9fdd2ac1dbf5ed\": rpc error: code = NotFound desc = could not find container \"75a83bc31cbf44588c41ddf601afdffe1e5368146bdbceea7e9fdd2ac1dbf5ed\": container with ID starting with 75a83bc31cbf44588c41ddf601afdffe1e5368146bdbceea7e9fdd2ac1dbf5ed not found: ID does not exist"
Sep 08 15:41:31.899215 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.899179    2560 scope.go:117] "RemoveContainer" containerID="bdb3d6d56a0e265baf219421e3650af37976d5100f9b40a0b565d9dbf43aa647"
Sep 08 15:41:31.902740 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.902717    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj"]
Sep 08 15:41:31.906554 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.906532    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-630e4d-5d494cbddf-nwwtj"]
Sep 08 15:41:31.908484 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.908471    2560 scope.go:117] "RemoveContainer" containerID="bdb3d6d56a0e265baf219421e3650af37976d5100f9b40a0b565d9dbf43aa647"
Sep 08 15:41:31.908744 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:41:31.908728    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdb3d6d56a0e265baf219421e3650af37976d5100f9b40a0b565d9dbf43aa647\": container with ID starting with bdb3d6d56a0e265baf219421e3650af37976d5100f9b40a0b565d9dbf43aa647 not found: ID does not exist" containerID="bdb3d6d56a0e265baf219421e3650af37976d5100f9b40a0b565d9dbf43aa647"
Sep 08 15:41:31.908804 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.908746    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdb3d6d56a0e265baf219421e3650af37976d5100f9b40a0b565d9dbf43aa647"} err="failed to get container status \"bdb3d6d56a0e265baf219421e3650af37976d5100f9b40a0b565d9dbf43aa647\": rpc error: code = NotFound desc = could not find container \"bdb3d6d56a0e265baf219421e3650af37976d5100f9b40a0b565d9dbf43aa647\": container with ID starting with bdb3d6d56a0e265baf219421e3650af37976d5100f9b40a0b565d9dbf43aa647 not found: ID does not exist"
Sep 08 15:41:31.917358 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.917338    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7"]
Sep 08 15:41:31.920518 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.920497    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-630e4d-55c84cdff-bgwb7"]
Sep 08 15:41:31.935319 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.935295    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7"]
Sep 08 15:41:31.940164 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:31.940142    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-630e4d-7bcc585c8b-xt7g7"]
Sep 08 15:41:33.800421 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:33.800385    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31fcda52-b9ff-4ab7-820e-1055c4fc0baa" path="/var/lib/kubelet/pods/31fcda52-b9ff-4ab7-820e-1055c4fc0baa/volumes"
Sep 08 15:41:33.800781 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:33.800736    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f4e4551-9695-4108-a670-bf8e9d52f9bf" path="/var/lib/kubelet/pods/4f4e4551-9695-4108-a670-bf8e9d52f9bf/volumes"
Sep 08 15:41:33.801047 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:33.801033    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57ea2742-18d0-4ef6-836a-1c407684916a" path="/var/lib/kubelet/pods/57ea2742-18d0-4ef6-836a-1c407684916a/volumes"
Sep 08 15:41:35.998705 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:35.998672    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:37.670485 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:37.670434    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:37.770729 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:37.770691    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:37.932403 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:37.932374    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:38.540764 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:38.540723    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:40.107011 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.106974    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"]
Sep 08 15:41:40.107398 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.107376    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57ea2742-18d0-4ef6-836a-1c407684916a" containerName="payload-pre-processing"
Sep 08 15:41:40.107398 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.107387    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="57ea2742-18d0-4ef6-836a-1c407684916a" containerName="payload-pre-processing"
Sep 08 15:41:40.107476 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.107403    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31fcda52-b9ff-4ab7-820e-1055c4fc0baa" containerName="maas-api"
Sep 08 15:41:40.107476 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.107409    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="31fcda52-b9ff-4ab7-820e-1055c4fc0baa" containerName="maas-api"
Sep 08 15:41:40.107476 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.107415    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f4e4551-9695-4108-a670-bf8e9d52f9bf" containerName="payload-processing"
Sep 08 15:41:40.107476 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.107420    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f4e4551-9695-4108-a670-bf8e9d52f9bf" containerName="payload-processing"
Sep 08 15:41:40.107597 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.107491    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="31fcda52-b9ff-4ab7-820e-1055c4fc0baa" containerName="maas-api"
Sep 08 15:41:40.107597 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.107502    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f4e4551-9695-4108-a670-bf8e9d52f9bf" containerName="payload-processing"
Sep 08 15:41:40.107597 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.107508    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="57ea2742-18d0-4ef6-836a-1c407684916a" containerName="payload-pre-processing"
Sep 08 15:41:40.112305 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.112281    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"
Sep 08 15:41:40.114957 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.114934    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-9ca1fc\"/\"default-dockercfg-b8r6b\""
Sep 08 15:41:40.115105 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.114990    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-9ca1fc\"/\"openshift-service-ca.crt\""
Sep 08 15:41:40.115105 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.115042    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-9ca1fc\"/\"kube-root-ca.crt\""
Sep 08 15:41:40.115105 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.114937    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-9ca1fc\"/\"rate-test-model-9ca1fc-kserve-self-signed-certs\""
Sep 08 15:41:40.122527 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.122495    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"]
Sep 08 15:41:40.208316 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.208269    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:40.240572 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.240535    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8560aa2-26e2-465c-8b9b-66f84e2c9855-home\") pod \"rate-test-model-9ca1fc-kserve-565c9d6455-njzlw\" (UID: \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\") " pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"
Sep 08 15:41:40.240732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.240583    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8560aa2-26e2-465c-8b9b-66f84e2c9855-tls-certs\") pod \"rate-test-model-9ca1fc-kserve-565c9d6455-njzlw\" (UID: \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\") " pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"
Sep 08 15:41:40.240732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.240638    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwgq7\" (UniqueName: \"kubernetes.io/projected/a8560aa2-26e2-465c-8b9b-66f84e2c9855-kube-api-access-lwgq7\") pod \"rate-test-model-9ca1fc-kserve-565c9d6455-njzlw\" (UID: \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\") " pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"
Sep 08 15:41:40.240732 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.240690    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8560aa2-26e2-465c-8b9b-66f84e2c9855-dshm\") pod \"rate-test-model-9ca1fc-kserve-565c9d6455-njzlw\" (UID: \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\") " pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"
Sep 08 15:41:40.240859 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.240744    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8560aa2-26e2-465c-8b9b-66f84e2c9855-tmp-dir\") pod \"rate-test-model-9ca1fc-kserve-565c9d6455-njzlw\" (UID: \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\") " pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"
Sep 08 15:41:40.240859 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.240802    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8560aa2-26e2-465c-8b9b-66f84e2c9855-model-cache\") pod \"rate-test-model-9ca1fc-kserve-565c9d6455-njzlw\" (UID: \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\") " pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"
Sep 08 15:41:40.341366 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.341327    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwgq7\" (UniqueName: \"kubernetes.io/projected/a8560aa2-26e2-465c-8b9b-66f84e2c9855-kube-api-access-lwgq7\") pod \"rate-test-model-9ca1fc-kserve-565c9d6455-njzlw\" (UID: \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\") " pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"
Sep 08 15:41:40.341564 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.341396    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8560aa2-26e2-465c-8b9b-66f84e2c9855-dshm\") pod \"rate-test-model-9ca1fc-kserve-565c9d6455-njzlw\" (UID: \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\") " pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"
Sep 08 15:41:40.341564 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.341427    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8560aa2-26e2-465c-8b9b-66f84e2c9855-tmp-dir\") pod \"rate-test-model-9ca1fc-kserve-565c9d6455-njzlw\" (UID: \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\") " pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"
Sep 08 15:41:40.341564 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.341464    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8560aa2-26e2-465c-8b9b-66f84e2c9855-model-cache\") pod \"rate-test-model-9ca1fc-kserve-565c9d6455-njzlw\" (UID: \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\") " pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"
Sep 08 15:41:40.341564 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.341558    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8560aa2-26e2-465c-8b9b-66f84e2c9855-home\") pod \"rate-test-model-9ca1fc-kserve-565c9d6455-njzlw\" (UID: \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\") " pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"
Sep 08 15:41:40.341787 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.341589    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8560aa2-26e2-465c-8b9b-66f84e2c9855-tls-certs\") pod \"rate-test-model-9ca1fc-kserve-565c9d6455-njzlw\" (UID: \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\") " pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"
Sep 08 15:41:40.341934 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.341895    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8560aa2-26e2-465c-8b9b-66f84e2c9855-tmp-dir\") pod \"rate-test-model-9ca1fc-kserve-565c9d6455-njzlw\" (UID: \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\") " pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"
Sep 08 15:41:40.341934 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.341910    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8560aa2-26e2-465c-8b9b-66f84e2c9855-model-cache\") pod \"rate-test-model-9ca1fc-kserve-565c9d6455-njzlw\" (UID: \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\") " pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"
Sep 08 15:41:40.342062 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.341983    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8560aa2-26e2-465c-8b9b-66f84e2c9855-home\") pod \"rate-test-model-9ca1fc-kserve-565c9d6455-njzlw\" (UID: \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\") " pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"
Sep 08 15:41:40.344043 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.344012    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8560aa2-26e2-465c-8b9b-66f84e2c9855-tls-certs\") pod \"rate-test-model-9ca1fc-kserve-565c9d6455-njzlw\" (UID: \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\") " pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"
Sep 08 15:41:40.344257 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.344236    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8560aa2-26e2-465c-8b9b-66f84e2c9855-dshm\") pod \"rate-test-model-9ca1fc-kserve-565c9d6455-njzlw\" (UID: \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\") " pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"
Sep 08 15:41:40.352233 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.352206    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwgq7\" (UniqueName: \"kubernetes.io/projected/a8560aa2-26e2-465c-8b9b-66f84e2c9855-kube-api-access-lwgq7\") pod \"rate-test-model-9ca1fc-kserve-565c9d6455-njzlw\" (UID: \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\") " pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"
Sep 08 15:41:40.421725 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.421627    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"
Sep 08 15:41:40.694171 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.694148    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"]
Sep 08 15:41:40.696700 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:41:40.696671    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda8560aa2_26e2_465c_8b9b_66f84e2c9855.slice/crio-098dca7667e3f41acb619c71c335e61f4115efb4386b5a81c0e719374fe4ab0c WatchSource:0}: Error finding container 098dca7667e3f41acb619c71c335e61f4115efb4386b5a81c0e719374fe4ab0c: Status 404 returned error can't find the container with id 098dca7667e3f41acb619c71c335e61f4115efb4386b5a81c0e719374fe4ab0c
Sep 08 15:41:40.936863 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.936829    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw" event={"ID":"a8560aa2-26e2-465c-8b9b-66f84e2c9855","Type":"ContainerStarted","Data":"90ef5d8cf1943b1251bbdb99f3e117eda14e8ae17ca38bfa7d9349de8a87dfe3"}
Sep 08 15:41:40.936863 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.936870    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw" event={"ID":"a8560aa2-26e2-465c-8b9b-66f84e2c9855","Type":"ContainerStarted","Data":"098dca7667e3f41acb619c71c335e61f4115efb4386b5a81c0e719374fe4ab0c"}
Sep 08 15:41:40.958595 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:40.958487    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw" podStartSLOduration=0.958465763 podStartE2EDuration="958.465763ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:41:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:41:40.957222153 +0000 UTC m=+1453.669164644" watchObservedRunningTime="2026-09-08 15:41:40.958465763 +0000 UTC m=+1453.670408266"
Sep 08 15:41:50.422408 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:50.422359    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"
Sep 08 15:41:50.422834 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:50.422428    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"
Sep 08 15:41:50.435495 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:50.435467    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"
Sep 08 15:41:50.999570 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:50.999536    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"
Sep 08 15:41:53.817879 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:53.817845    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7"]
Sep 08 15:41:53.818651 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:53.818615    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7" podUID="ee3c39b6-f4c7-4d81-90a5-75526fecc6fa" containerName="maas-api" containerID="cri-o://bb4f98129d946148b200bce42ced80162e2e051b6204ba1cf7afea4a36d3ad26" gracePeriod=30
Sep 08 15:41:54.007167 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:54.007128    2560 generic.go:356] "Generic (PLEG): container finished" podID="ee3c39b6-f4c7-4d81-90a5-75526fecc6fa" containerID="bb4f98129d946148b200bce42ced80162e2e051b6204ba1cf7afea4a36d3ad26" exitCode=0
Sep 08 15:41:54.007329 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:54.007203    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7" event={"ID":"ee3c39b6-f4c7-4d81-90a5-75526fecc6fa","Type":"ContainerDied","Data":"bb4f98129d946148b200bce42ced80162e2e051b6204ba1cf7afea4a36d3ad26"}
Sep 08 15:41:54.077690 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:54.077631    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7"
Sep 08 15:41:54.172279 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:54.172239    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee3c39b6-f4c7-4d81-90a5-75526fecc6fa-maas-api-tls\") pod \"ee3c39b6-f4c7-4d81-90a5-75526fecc6fa\" (UID: \"ee3c39b6-f4c7-4d81-90a5-75526fecc6fa\") "
Sep 08 15:41:54.172511 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:54.172306    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l54zb\" (UniqueName: \"kubernetes.io/projected/ee3c39b6-f4c7-4d81-90a5-75526fecc6fa-kube-api-access-l54zb\") pod \"ee3c39b6-f4c7-4d81-90a5-75526fecc6fa\" (UID: \"ee3c39b6-f4c7-4d81-90a5-75526fecc6fa\") "
Sep 08 15:41:54.172511 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:54.172446    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee3c39b6-f4c7-4d81-90a5-75526fecc6fa-metrics-certs\") pod \"ee3c39b6-f4c7-4d81-90a5-75526fecc6fa\" (UID: \"ee3c39b6-f4c7-4d81-90a5-75526fecc6fa\") "
Sep 08 15:41:54.172511 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:54.172492    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee3c39b6-f4c7-4d81-90a5-75526fecc6fa-openshift-service-ca\") pod \"ee3c39b6-f4c7-4d81-90a5-75526fecc6fa\" (UID: \"ee3c39b6-f4c7-4d81-90a5-75526fecc6fa\") "
Sep 08 15:41:54.172977 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:54.172945    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee3c39b6-f4c7-4d81-90a5-75526fecc6fa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ee3c39b6-f4c7-4d81-90a5-75526fecc6fa" (UID: "ee3c39b6-f4c7-4d81-90a5-75526fecc6fa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:41:54.174606 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:54.174585    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee3c39b6-f4c7-4d81-90a5-75526fecc6fa-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ee3c39b6-f4c7-4d81-90a5-75526fecc6fa" (UID: "ee3c39b6-f4c7-4d81-90a5-75526fecc6fa"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:41:54.174834 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:54.174817    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee3c39b6-f4c7-4d81-90a5-75526fecc6fa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ee3c39b6-f4c7-4d81-90a5-75526fecc6fa" (UID: "ee3c39b6-f4c7-4d81-90a5-75526fecc6fa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:41:54.175259 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:54.175237    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee3c39b6-f4c7-4d81-90a5-75526fecc6fa-kube-api-access-l54zb" (OuterVolumeSpecName: "kube-api-access-l54zb") pod "ee3c39b6-f4c7-4d81-90a5-75526fecc6fa" (UID: "ee3c39b6-f4c7-4d81-90a5-75526fecc6fa"). InnerVolumeSpecName "kube-api-access-l54zb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:41:54.274402 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:54.274368    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee3c39b6-f4c7-4d81-90a5-75526fecc6fa-maas-api-tls\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:41:54.274402 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:54.274395    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l54zb\" (UniqueName: \"kubernetes.io/projected/ee3c39b6-f4c7-4d81-90a5-75526fecc6fa-kube-api-access-l54zb\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:41:54.274402 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:54.274405    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee3c39b6-f4c7-4d81-90a5-75526fecc6fa-metrics-certs\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:41:54.274402 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:54.274414    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee3c39b6-f4c7-4d81-90a5-75526fecc6fa-openshift-service-ca\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:41:54.844621 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:54.844582    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:54.946230 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:54.946196    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:55.013938 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:55.013911    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7"
Sep 08 15:41:55.014121 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:55.013953    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7" event={"ID":"ee3c39b6-f4c7-4d81-90a5-75526fecc6fa","Type":"ContainerDied","Data":"0363f129d8681298919bf087a16fb00d349eb7bb362203e96c843bdb4287d1c7"}
Sep 08 15:41:55.014121 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:55.014017    2560 scope.go:117] "RemoveContainer" containerID="bb4f98129d946148b200bce42ced80162e2e051b6204ba1cf7afea4a36d3ad26"
Sep 08 15:41:55.037528 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:55.037503    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7"]
Sep 08 15:41:55.044115 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:55.044082    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-bea11f-6697559944-ztsx7"]
Sep 08 15:41:55.799999 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:41:55.799965    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee3c39b6-f4c7-4d81-90a5-75526fecc6fa" path="/var/lib/kubelet/pods/ee3c39b6-f4c7-4d81-90a5-75526fecc6fa/volumes"
Sep 08 15:42:00.575721 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:00.575677    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:00.648178 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:00.648122    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:00.843004 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:00.842918    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:01.459926 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:01.459893    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:11.840000 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:11.839966    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6"]
Sep 08 15:42:11.840580 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:11.840251    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6" podUID="e606c7e2-90e3-45ef-932d-afc43ab58e12" containerName="maas-api" containerID="cri-o://16ed1cb6e212d81c212a31729962d87bbebde339eaf1ffe96bf4139d5ba99b0f" gracePeriod=30
Sep 08 15:42:11.875446 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:11.875405    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:11.891103 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:11.891055    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8"]
Sep 08 15:42:11.891571 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:11.891278    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8" podUID="47b63d67-2d05-4020-bc72-2592f6b21710" containerName="payload-pre-processing" containerID="cri-o://4c070c91c3d2739cf83f2c00e1108b169c467e6ff7502c6711e64dcba87b92c7" gracePeriod=30
Sep 08 15:42:11.975391 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:11.975361    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:12.078276 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:12.078242    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:12.108646 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:12.108617    2560 generic.go:356] "Generic (PLEG): container finished" podID="47b63d67-2d05-4020-bc72-2592f6b21710" containerID="4c070c91c3d2739cf83f2c00e1108b169c467e6ff7502c6711e64dcba87b92c7" exitCode=0
Sep 08 15:42:12.108768 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:12.108683    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8" event={"ID":"47b63d67-2d05-4020-bc72-2592f6b21710","Type":"ContainerDied","Data":"4c070c91c3d2739cf83f2c00e1108b169c467e6ff7502c6711e64dcba87b92c7"}
Sep 08 15:42:12.110741 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:12.110714    2560 generic.go:356] "Generic (PLEG): container finished" podID="e606c7e2-90e3-45ef-932d-afc43ab58e12" containerID="16ed1cb6e212d81c212a31729962d87bbebde339eaf1ffe96bf4139d5ba99b0f" exitCode=0
Sep 08 15:42:12.110856 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:12.110740    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6" event={"ID":"e606c7e2-90e3-45ef-932d-afc43ab58e12","Type":"ContainerDied","Data":"16ed1cb6e212d81c212a31729962d87bbebde339eaf1ffe96bf4139d5ba99b0f"}
Sep 08 15:42:12.148083 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:12.148035    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6"
Sep 08 15:42:12.171598 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:12.171553    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:12.200992 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:12.200962    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8"
Sep 08 15:42:12.249096 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:12.249057    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k66gv\" (UniqueName: \"kubernetes.io/projected/e606c7e2-90e3-45ef-932d-afc43ab58e12-kube-api-access-k66gv\") pod \"e606c7e2-90e3-45ef-932d-afc43ab58e12\" (UID: \"e606c7e2-90e3-45ef-932d-afc43ab58e12\") "
Sep 08 15:42:12.249248 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:12.249131    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e606c7e2-90e3-45ef-932d-afc43ab58e12-openshift-service-ca\") pod \"e606c7e2-90e3-45ef-932d-afc43ab58e12\" (UID: \"e606c7e2-90e3-45ef-932d-afc43ab58e12\") "
Sep 08 15:42:12.249248 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:12.249202    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e606c7e2-90e3-45ef-932d-afc43ab58e12-metrics-certs\") pod \"e606c7e2-90e3-45ef-932d-afc43ab58e12\" (UID: \"e606c7e2-90e3-45ef-932d-afc43ab58e12\") "
Sep 08 15:42:12.249322 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:12.249260    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e606c7e2-90e3-45ef-932d-afc43ab58e12-maas-api-tls\") pod \"e606c7e2-90e3-45ef-932d-afc43ab58e12\" (UID: \"e606c7e2-90e3-45ef-932d-afc43ab58e12\") "
Sep 08 15:42:12.249519 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:12.249492    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e606c7e2-90e3-45ef-932d-afc43ab58e12-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e606c7e2-90e3-45ef-932d-afc43ab58e12" (UID: "e606c7e2-90e3-45ef-932d-afc43ab58e12"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:42:12.251274 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:12.251247    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e606c7e2-90e3-45ef-932d-afc43ab58e12-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e606c7e2-90e3-45ef-932d-afc43ab58e12" (UID: "e606c7e2-90e3-45ef-932d-afc43ab58e12"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:42:12.251274 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:12.251263    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e606c7e2-90e3-45ef-932d-afc43ab58e12-kube-api-access-k66gv" (OuterVolumeSpecName: "kube-api-access-k66gv") pod "e606c7e2-90e3-45ef-932d-afc43ab58e12" (UID: "e606c7e2-90e3-45ef-932d-afc43ab58e12"). InnerVolumeSpecName "kube-api-access-k66gv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:42:12.251424 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:12.251277    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e606c7e2-90e3-45ef-932d-afc43ab58e12-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e606c7e2-90e3-45ef-932d-afc43ab58e12" (UID: "e606c7e2-90e3-45ef-932d-afc43ab58e12"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:42:12.349959 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:12.349924    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fgndk\" (UniqueName: \"kubernetes.io/projected/47b63d67-2d05-4020-bc72-2592f6b21710-kube-api-access-fgndk\") pod \"47b63d67-2d05-4020-bc72-2592f6b21710\" (UID: \"47b63d67-2d05-4020-bc72-2592f6b21710\") "
Sep 08 15:42:12.350156 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:12.349975    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47b63d67-2d05-4020-bc72-2592f6b21710-plugins-config-volume\") pod \"47b63d67-2d05-4020-bc72-2592f6b21710\" (UID: \"47b63d67-2d05-4020-bc72-2592f6b21710\") "
Sep 08 15:42:12.350322 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:12.350304    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e606c7e2-90e3-45ef-932d-afc43ab58e12-openshift-service-ca\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:42:12.350391 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:12.350327    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e606c7e2-90e3-45ef-932d-afc43ab58e12-metrics-certs\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:42:12.350391 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:12.350328    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47b63d67-2d05-4020-bc72-2592f6b21710-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "47b63d67-2d05-4020-bc72-2592f6b21710" (UID: "47b63d67-2d05-4020-bc72-2592f6b21710"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:42:12.350391 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:12.350341    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e606c7e2-90e3-45ef-932d-afc43ab58e12-maas-api-tls\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:42:12.350391 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:12.350381    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k66gv\" (UniqueName: \"kubernetes.io/projected/e606c7e2-90e3-45ef-932d-afc43ab58e12-kube-api-access-k66gv\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:42:12.351895 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:12.351876    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47b63d67-2d05-4020-bc72-2592f6b21710-kube-api-access-fgndk" (OuterVolumeSpecName: "kube-api-access-fgndk") pod "47b63d67-2d05-4020-bc72-2592f6b21710" (UID: "47b63d67-2d05-4020-bc72-2592f6b21710"). InnerVolumeSpecName "kube-api-access-fgndk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:42:12.430129 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:12.430083    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:12.450955 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:12.450926    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgndk\" (UniqueName: \"kubernetes.io/projected/47b63d67-2d05-4020-bc72-2592f6b21710-kube-api-access-fgndk\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:42:12.450955 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:12.450952    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47b63d67-2d05-4020-bc72-2592f6b21710-plugins-config-volume\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:42:13.116760 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:13.116679    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8"
Sep 08 15:42:13.117228 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:13.116678    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8" event={"ID":"47b63d67-2d05-4020-bc72-2592f6b21710","Type":"ContainerDied","Data":"8dc1b276fff8af0284de4f5fb2bb80a71992ce02ffb5b1901b83f60e4fe5cd5b"}
Sep 08 15:42:13.117228 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:13.116817    2560 scope.go:117] "RemoveContainer" containerID="4c070c91c3d2739cf83f2c00e1108b169c467e6ff7502c6711e64dcba87b92c7"
Sep 08 15:42:13.118398 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:13.118379    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6"
Sep 08 15:42:13.118464 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:13.118412    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6" event={"ID":"e606c7e2-90e3-45ef-932d-afc43ab58e12","Type":"ContainerDied","Data":"1320343276d526624b45206645609301c0c96068b6772828524bcf6abb66f3e3"}
Sep 08 15:42:13.125696 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:13.125679    2560 scope.go:117] "RemoveContainer" containerID="16ed1cb6e212d81c212a31729962d87bbebde339eaf1ffe96bf4139d5ba99b0f"
Sep 08 15:42:13.144690 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:13.144666    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8"]
Sep 08 15:42:13.149130 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:13.149110    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-03e8df-675bdf66c4-qnfd8"]
Sep 08 15:42:13.163534 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:13.163512    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6"]
Sep 08 15:42:13.169409 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:13.169388    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-03e8df-6456f86c46-k58z6"]
Sep 08 15:42:13.800333 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:13.800299    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47b63d67-2d05-4020-bc72-2592f6b21710" path="/var/lib/kubelet/pods/47b63d67-2d05-4020-bc72-2592f6b21710/volumes"
Sep 08 15:42:13.800700 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:13.800687    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e606c7e2-90e3-45ef-932d-afc43ab58e12" path="/var/lib/kubelet/pods/e606c7e2-90e3-45ef-932d-afc43ab58e12/volumes"
Sep 08 15:42:18.727534 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:18.727499    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:18.828398 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:18.828337    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:18.992797 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:18.992690    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:19.423997 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:19.423952    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:23.357192 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:23.357155    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:27.789841 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:27.789803    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vc5n9_17e30810-dfd0-4e6f-9204-731d1a6479a3/console-operator/2.log"
Sep 08 15:42:27.793261 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:27.793241    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vc5n9_17e30810-dfd0-4e6f-9204-731d1a6479a3/console-operator/2.log"
Sep 08 15:42:29.511609 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:29.511563    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:30.263830 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:30.263797    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:30.329894 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:30.329856    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8"]
Sep 08 15:42:30.330157 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:30.330098    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8" podUID="24bc2a8e-f67f-4289-81d4-0f582de6299d" containerName="payload-processing" containerID="cri-o://e1dfdc65a5d8193275c1f3655e9be31b6ad50d2328e9b23eb113d1a0ad91029d" gracePeriod=30
Sep 08 15:42:30.376744 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:30.376714    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:30.473021 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:30.472963    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:30.554369 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:30.554296    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:30.590062 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:30.590040    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8"
Sep 08 15:42:30.614271 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:30.614245    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24bc2a8e-f67f-4289-81d4-0f582de6299d-plugins-config-volume\") pod \"24bc2a8e-f67f-4289-81d4-0f582de6299d\" (UID: \"24bc2a8e-f67f-4289-81d4-0f582de6299d\") "
Sep 08 15:42:30.614436 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:30.614330    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lj6rg\" (UniqueName: \"kubernetes.io/projected/24bc2a8e-f67f-4289-81d4-0f582de6299d-kube-api-access-lj6rg\") pod \"24bc2a8e-f67f-4289-81d4-0f582de6299d\" (UID: \"24bc2a8e-f67f-4289-81d4-0f582de6299d\") "
Sep 08 15:42:30.614891 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:30.614862    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24bc2a8e-f67f-4289-81d4-0f582de6299d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "24bc2a8e-f67f-4289-81d4-0f582de6299d" (UID: "24bc2a8e-f67f-4289-81d4-0f582de6299d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:42:30.617173 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:30.617146    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24bc2a8e-f67f-4289-81d4-0f582de6299d-kube-api-access-lj6rg" (OuterVolumeSpecName: "kube-api-access-lj6rg") pod "24bc2a8e-f67f-4289-81d4-0f582de6299d" (UID: "24bc2a8e-f67f-4289-81d4-0f582de6299d"). InnerVolumeSpecName "kube-api-access-lj6rg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:42:30.626869 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:30.626848    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:30.715509 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:30.715475    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24bc2a8e-f67f-4289-81d4-0f582de6299d-plugins-config-volume\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:42:30.715509 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:30.715505    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lj6rg\" (UniqueName: \"kubernetes.io/projected/24bc2a8e-f67f-4289-81d4-0f582de6299d-kube-api-access-lj6rg\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:42:30.900468 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:30.900365    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:31.210123 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:31.210091    2560 generic.go:356] "Generic (PLEG): container finished" podID="24bc2a8e-f67f-4289-81d4-0f582de6299d" containerID="e1dfdc65a5d8193275c1f3655e9be31b6ad50d2328e9b23eb113d1a0ad91029d" exitCode=0
Sep 08 15:42:31.210329 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:31.210129    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8"
Sep 08 15:42:31.210329 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:31.210177    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8" event={"ID":"24bc2a8e-f67f-4289-81d4-0f582de6299d","Type":"ContainerDied","Data":"e1dfdc65a5d8193275c1f3655e9be31b6ad50d2328e9b23eb113d1a0ad91029d"}
Sep 08 15:42:31.210329 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:31.210233    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8" event={"ID":"24bc2a8e-f67f-4289-81d4-0f582de6299d","Type":"ContainerDied","Data":"a95c3f30b395dea6b9ac8baf48cb39fa29b2d55ab6b51dfcc0a02c808ecd9f7b"}
Sep 08 15:42:31.210329 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:31.210265    2560 scope.go:117] "RemoveContainer" containerID="e1dfdc65a5d8193275c1f3655e9be31b6ad50d2328e9b23eb113d1a0ad91029d"
Sep 08 15:42:31.219562 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:31.219523    2560 scope.go:117] "RemoveContainer" containerID="e1dfdc65a5d8193275c1f3655e9be31b6ad50d2328e9b23eb113d1a0ad91029d"
Sep 08 15:42:31.219878 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:42:31.219859    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1dfdc65a5d8193275c1f3655e9be31b6ad50d2328e9b23eb113d1a0ad91029d\": container with ID starting with e1dfdc65a5d8193275c1f3655e9be31b6ad50d2328e9b23eb113d1a0ad91029d not found: ID does not exist" containerID="e1dfdc65a5d8193275c1f3655e9be31b6ad50d2328e9b23eb113d1a0ad91029d"
Sep 08 15:42:31.219923 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:31.219887    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1dfdc65a5d8193275c1f3655e9be31b6ad50d2328e9b23eb113d1a0ad91029d"} err="failed to get container status \"e1dfdc65a5d8193275c1f3655e9be31b6ad50d2328e9b23eb113d1a0ad91029d\": rpc error: code = NotFound desc = could not find container \"e1dfdc65a5d8193275c1f3655e9be31b6ad50d2328e9b23eb113d1a0ad91029d\": container with ID starting with e1dfdc65a5d8193275c1f3655e9be31b6ad50d2328e9b23eb113d1a0ad91029d not found: ID does not exist"
Sep 08 15:42:31.231900 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:31.231874    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8"]
Sep 08 15:42:31.236634 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:31.236610    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a7ae4e-6c6475c99c-xswv8"]
Sep 08 15:42:31.302543 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:31.302513    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:31.803255 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:31.803226    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24bc2a8e-f67f-4289-81d4-0f582de6299d" path="/var/lib/kubelet/pods/24bc2a8e-f67f-4289-81d4-0f582de6299d/volumes"
Sep 08 15:42:37.141595 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:37.141546    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:37.242246 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:37.242212    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:37.419064 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:37.418964    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:49.337865 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:49.337827    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:49.421700 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:42:49.421666    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:43:08.145953 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:08.145920    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"]
Sep 08 15:43:08.168494 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:08.168464    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:43:09.310952 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:09.310922    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:43:09.717606 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:09.717576    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:43:10.104934 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:10.104843    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:43:10.479445 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:10.479401    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:43:13.105658 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:13.105620    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:43:13.243445 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:13.243414    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:43:14.241239 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.241199    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l"]
Sep 08 15:43:14.241606 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.241573    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24bc2a8e-f67f-4289-81d4-0f582de6299d" containerName="payload-processing"
Sep 08 15:43:14.241606 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.241585    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="24bc2a8e-f67f-4289-81d4-0f582de6299d" containerName="payload-processing"
Sep 08 15:43:14.241606 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.241594    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee3c39b6-f4c7-4d81-90a5-75526fecc6fa" containerName="maas-api"
Sep 08 15:43:14.241606 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.241600    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee3c39b6-f4c7-4d81-90a5-75526fecc6fa" containerName="maas-api"
Sep 08 15:43:14.241811 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.241613    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e606c7e2-90e3-45ef-932d-afc43ab58e12" containerName="maas-api"
Sep 08 15:43:14.241811 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.241619    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e606c7e2-90e3-45ef-932d-afc43ab58e12" containerName="maas-api"
Sep 08 15:43:14.241811 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.241633    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47b63d67-2d05-4020-bc72-2592f6b21710" containerName="payload-pre-processing"
Sep 08 15:43:14.241811 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.241639    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="47b63d67-2d05-4020-bc72-2592f6b21710" containerName="payload-pre-processing"
Sep 08 15:43:14.241811 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.241722    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="47b63d67-2d05-4020-bc72-2592f6b21710" containerName="payload-pre-processing"
Sep 08 15:43:14.241811 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.241735    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e606c7e2-90e3-45ef-932d-afc43ab58e12" containerName="maas-api"
Sep 08 15:43:14.241811 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.241749    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee3c39b6-f4c7-4d81-90a5-75526fecc6fa" containerName="maas-api"
Sep 08 15:43:14.241811 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.241756    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="24bc2a8e-f67f-4289-81d4-0f582de6299d" containerName="payload-processing"
Sep 08 15:43:14.245053 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.245036    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l"
Sep 08 15:43:14.247230 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.247207    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-41de51-serving-cert\""
Sep 08 15:43:14.258086 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.255809    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l"]
Sep 08 15:43:14.267054 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.267028    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:43:14.314910 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.314878    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55d135e7-bf30-450e-89a9-58f53bb65514-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l\" (UID: \"55d135e7-bf30-450e-89a9-58f53bb65514\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l"
Sep 08 15:43:14.314910 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.314913    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55d135e7-bf30-450e-89a9-58f53bb65514-metrics-certs\") pod \"maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l\" (UID: \"55d135e7-bf30-450e-89a9-58f53bb65514\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l"
Sep 08 15:43:14.315102 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.314957    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55d135e7-bf30-450e-89a9-58f53bb65514-maas-api-tls\") pod \"maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l\" (UID: \"55d135e7-bf30-450e-89a9-58f53bb65514\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l"
Sep 08 15:43:14.315102 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.315005    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-858b5\" (UniqueName: \"kubernetes.io/projected/55d135e7-bf30-450e-89a9-58f53bb65514-kube-api-access-858b5\") pod \"maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l\" (UID: \"55d135e7-bf30-450e-89a9-58f53bb65514\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l"
Sep 08 15:43:14.366907 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.366875    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:43:14.416403 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.416374    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55d135e7-bf30-450e-89a9-58f53bb65514-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l\" (UID: \"55d135e7-bf30-450e-89a9-58f53bb65514\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l"
Sep 08 15:43:14.416553 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.416411    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55d135e7-bf30-450e-89a9-58f53bb65514-metrics-certs\") pod \"maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l\" (UID: \"55d135e7-bf30-450e-89a9-58f53bb65514\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l"
Sep 08 15:43:14.416553 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.416466    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55d135e7-bf30-450e-89a9-58f53bb65514-maas-api-tls\") pod \"maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l\" (UID: \"55d135e7-bf30-450e-89a9-58f53bb65514\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l"
Sep 08 15:43:14.416553 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.416494    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-858b5\" (UniqueName: \"kubernetes.io/projected/55d135e7-bf30-450e-89a9-58f53bb65514-kube-api-access-858b5\") pod \"maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l\" (UID: \"55d135e7-bf30-450e-89a9-58f53bb65514\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l"
Sep 08 15:43:14.416678 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:43:14.416597    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-serving-cert: secret "maas-api-e2e-ipp-a-41de51-serving-cert" not found
Sep 08 15:43:14.416716 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:43:14.416678    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55d135e7-bf30-450e-89a9-58f53bb65514-maas-api-tls podName:55d135e7-bf30-450e-89a9-58f53bb65514 nodeName:}" failed. No retries permitted until 2026-09-08 15:43:14.916659848 +0000 UTC m=+1547.628602317 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/55d135e7-bf30-450e-89a9-58f53bb65514-maas-api-tls") pod "maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l" (UID: "55d135e7-bf30-450e-89a9-58f53bb65514") : secret "maas-api-e2e-ipp-a-41de51-serving-cert" not found
Sep 08 15:43:14.417187 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.417169    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55d135e7-bf30-450e-89a9-58f53bb65514-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l\" (UID: \"55d135e7-bf30-450e-89a9-58f53bb65514\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l"
Sep 08 15:43:14.418927 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.418909    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55d135e7-bf30-450e-89a9-58f53bb65514-metrics-certs\") pod \"maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l\" (UID: \"55d135e7-bf30-450e-89a9-58f53bb65514\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l"
Sep 08 15:43:14.426193 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.426168    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-858b5\" (UniqueName: \"kubernetes.io/projected/55d135e7-bf30-450e-89a9-58f53bb65514-kube-api-access-858b5\") pod \"maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l\" (UID: \"55d135e7-bf30-450e-89a9-58f53bb65514\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l"
Sep 08 15:43:14.509923 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.509833    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:43:14.906839 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.906747    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:43:14.922006 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.921967    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55d135e7-bf30-450e-89a9-58f53bb65514-maas-api-tls\") pod \"maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l\" (UID: \"55d135e7-bf30-450e-89a9-58f53bb65514\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l"
Sep 08 15:43:14.924359 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:14.924336    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55d135e7-bf30-450e-89a9-58f53bb65514-maas-api-tls\") pod \"maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l\" (UID: \"55d135e7-bf30-450e-89a9-58f53bb65514\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l"
Sep 08 15:43:15.003189 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:15.003147    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:43:15.036520 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:15.036484    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs"]
Sep 08 15:43:15.040395 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:15.040366    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs"
Sep 08 15:43:15.043244 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:15.043220    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-41de51-dockercfg-bm75n\""
Sep 08 15:43:15.043585 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:15.043567    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-41de51\""
Sep 08 15:43:15.052422 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:15.052395    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs"]
Sep 08 15:43:15.124489 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:15.124456    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72e29e11-a7fb-4daf-b867-9f43ab5074ff-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs\" (UID: \"72e29e11-a7fb-4daf-b867-9f43ab5074ff\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs"
Sep 08 15:43:15.124655 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:15.124600    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrgkk\" (UniqueName: \"kubernetes.io/projected/72e29e11-a7fb-4daf-b867-9f43ab5074ff-kube-api-access-mrgkk\") pod \"payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs\" (UID: \"72e29e11-a7fb-4daf-b867-9f43ab5074ff\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs"
Sep 08 15:43:15.159529 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:15.159444    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l"
Sep 08 15:43:15.225901 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:15.225863    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72e29e11-a7fb-4daf-b867-9f43ab5074ff-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs\" (UID: \"72e29e11-a7fb-4daf-b867-9f43ab5074ff\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs"
Sep 08 15:43:15.226107 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:15.226058    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrgkk\" (UniqueName: \"kubernetes.io/projected/72e29e11-a7fb-4daf-b867-9f43ab5074ff-kube-api-access-mrgkk\") pod \"payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs\" (UID: \"72e29e11-a7fb-4daf-b867-9f43ab5074ff\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs"
Sep 08 15:43:15.226821 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:15.226801    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72e29e11-a7fb-4daf-b867-9f43ab5074ff-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs\" (UID: \"72e29e11-a7fb-4daf-b867-9f43ab5074ff\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs"
Sep 08 15:43:15.235261 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:15.235241    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrgkk\" (UniqueName: \"kubernetes.io/projected/72e29e11-a7fb-4daf-b867-9f43ab5074ff-kube-api-access-mrgkk\") pod \"payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs\" (UID: \"72e29e11-a7fb-4daf-b867-9f43ab5074ff\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs"
Sep 08 15:43:15.355405 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:15.355370    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs"
Sep 08 15:43:15.436007 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:15.435976    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l"]
Sep 08 15:43:15.438233 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:43:15.438181    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55d135e7_bf30_450e_89a9_58f53bb65514.slice/crio-0c64c9739c2cec22df2bc32d8cdf8a8e1e2754fffd390e9dbf8d050427f1cbe9 WatchSource:0}: Error finding container 0c64c9739c2cec22df2bc32d8cdf8a8e1e2754fffd390e9dbf8d050427f1cbe9: Status 404 returned error can't find the container with id 0c64c9739c2cec22df2bc32d8cdf8a8e1e2754fffd390e9dbf8d050427f1cbe9
Sep 08 15:43:15.628255 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:15.628229    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs"]
Sep 08 15:43:15.630436 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:43:15.630406    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72e29e11_a7fb_4daf_b867_9f43ab5074ff.slice/crio-b495ea8371ad0782b51b28b7a31fb7e6dfc16ffb1328eea0874d674a163f7baf WatchSource:0}: Error finding container b495ea8371ad0782b51b28b7a31fb7e6dfc16ffb1328eea0874d674a163f7baf: Status 404 returned error can't find the container with id b495ea8371ad0782b51b28b7a31fb7e6dfc16ffb1328eea0874d674a163f7baf
Sep 08 15:43:16.438453 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:16.438417    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l" event={"ID":"55d135e7-bf30-450e-89a9-58f53bb65514","Type":"ContainerStarted","Data":"0c64c9739c2cec22df2bc32d8cdf8a8e1e2754fffd390e9dbf8d050427f1cbe9"}
Sep 08 15:43:16.440304 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:16.440272    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs" event={"ID":"72e29e11-a7fb-4daf-b867-9f43ab5074ff","Type":"ContainerStarted","Data":"5455422e041db207621e33e32af49e92e16f52d29131d515501f980fd5fa8d65"}
Sep 08 15:43:16.440449 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:16.440322    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs" event={"ID":"72e29e11-a7fb-4daf-b867-9f43ab5074ff","Type":"ContainerStarted","Data":"b495ea8371ad0782b51b28b7a31fb7e6dfc16ffb1328eea0874d674a163f7baf"}
Sep 08 15:43:16.440449 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:16.440383    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs"
Sep 08 15:43:16.460336 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:16.460280    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs" podStartSLOduration=1.46026166 podStartE2EDuration="1.46026166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:43:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:43:16.457374991 +0000 UTC m=+1549.169317481" watchObservedRunningTime="2026-09-08 15:43:16.46026166 +0000 UTC m=+1549.172204156"
Sep 08 15:43:19.446684 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:19.446647    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:43:19.458520 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:19.458460    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l" event={"ID":"55d135e7-bf30-450e-89a9-58f53bb65514","Type":"ContainerStarted","Data":"90ebeb8c1640c6e793a55f4514ff3e6aa0e9190748adc16f898705db2977ce9f"}
Sep 08 15:43:19.458688 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:19.458628    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l"
Sep 08 15:43:19.479120 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:19.479053    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l" podStartSLOduration=1.660473542 podStartE2EDuration="5.479032803s" totalImagesPullingTime="3.818559261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:43:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:43:19.478182297 +0000 UTC m=+1552.190124787" watchObservedRunningTime="2026-09-08 15:43:19.479032803 +0000 UTC m=+1552.190975306"
Sep 08 15:43:20.779908 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:20.779869    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:43:23.166217 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.166178    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw" podUID="a8560aa2-26e2-465c-8b9b-66f84e2c9855" containerName="main" containerID="cri-o://90ef5d8cf1943b1251bbdb99f3e117eda14e8ae17ca38bfa7d9349de8a87dfe3" gracePeriod=45
Sep 08 15:43:23.420610 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.420554    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"
Sep 08 15:43:23.478897 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.478866    2560 generic.go:356] "Generic (PLEG): container finished" podID="a8560aa2-26e2-465c-8b9b-66f84e2c9855" containerID="90ef5d8cf1943b1251bbdb99f3e117eda14e8ae17ca38bfa7d9349de8a87dfe3" exitCode=0
Sep 08 15:43:23.479080 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.478903    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw" event={"ID":"a8560aa2-26e2-465c-8b9b-66f84e2c9855","Type":"ContainerDied","Data":"90ef5d8cf1943b1251bbdb99f3e117eda14e8ae17ca38bfa7d9349de8a87dfe3"}
Sep 08 15:43:23.479080 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.478906    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"
Sep 08 15:43:23.479080 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.478948    2560 scope.go:117] "RemoveContainer" containerID="90ef5d8cf1943b1251bbdb99f3e117eda14e8ae17ca38bfa7d9349de8a87dfe3"
Sep 08 15:43:23.479080 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.478935    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw" event={"ID":"a8560aa2-26e2-465c-8b9b-66f84e2c9855","Type":"ContainerDied","Data":"098dca7667e3f41acb619c71c335e61f4115efb4386b5a81c0e719374fe4ab0c"}
Sep 08 15:43:23.487269 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.487247    2560 scope.go:117] "RemoveContainer" containerID="90ef5d8cf1943b1251bbdb99f3e117eda14e8ae17ca38bfa7d9349de8a87dfe3"
Sep 08 15:43:23.487560 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:43:23.487543    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90ef5d8cf1943b1251bbdb99f3e117eda14e8ae17ca38bfa7d9349de8a87dfe3\": container with ID starting with 90ef5d8cf1943b1251bbdb99f3e117eda14e8ae17ca38bfa7d9349de8a87dfe3 not found: ID does not exist" containerID="90ef5d8cf1943b1251bbdb99f3e117eda14e8ae17ca38bfa7d9349de8a87dfe3"
Sep 08 15:43:23.487603 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.487568    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90ef5d8cf1943b1251bbdb99f3e117eda14e8ae17ca38bfa7d9349de8a87dfe3"} err="failed to get container status \"90ef5d8cf1943b1251bbdb99f3e117eda14e8ae17ca38bfa7d9349de8a87dfe3\": rpc error: code = NotFound desc = could not find container \"90ef5d8cf1943b1251bbdb99f3e117eda14e8ae17ca38bfa7d9349de8a87dfe3\": container with ID starting with 90ef5d8cf1943b1251bbdb99f3e117eda14e8ae17ca38bfa7d9349de8a87dfe3 not found: ID does not exist"
Sep 08 15:43:23.500881 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.500859    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8560aa2-26e2-465c-8b9b-66f84e2c9855-tls-certs\") pod \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\" (UID: \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\") "
Sep 08 15:43:23.500971 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.500888    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8560aa2-26e2-465c-8b9b-66f84e2c9855-model-cache\") pod \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\" (UID: \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\") "
Sep 08 15:43:23.500971 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.500914    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8560aa2-26e2-465c-8b9b-66f84e2c9855-tmp-dir\") pod \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\" (UID: \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\") "
Sep 08 15:43:23.501046 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.500993    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8560aa2-26e2-465c-8b9b-66f84e2c9855-dshm\") pod \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\" (UID: \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\") "
Sep 08 15:43:23.501105 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.501044    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwgq7\" (UniqueName: \"kubernetes.io/projected/a8560aa2-26e2-465c-8b9b-66f84e2c9855-kube-api-access-lwgq7\") pod \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\" (UID: \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\") "
Sep 08 15:43:23.501156 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.501140    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8560aa2-26e2-465c-8b9b-66f84e2c9855-home\") pod \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\" (UID: \"a8560aa2-26e2-465c-8b9b-66f84e2c9855\") "
Sep 08 15:43:23.501201 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.501175    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8560aa2-26e2-465c-8b9b-66f84e2c9855-model-cache" (OuterVolumeSpecName: "model-cache") pod "a8560aa2-26e2-465c-8b9b-66f84e2c9855" (UID: "a8560aa2-26e2-465c-8b9b-66f84e2c9855"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:43:23.501250 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.501218    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8560aa2-26e2-465c-8b9b-66f84e2c9855-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a8560aa2-26e2-465c-8b9b-66f84e2c9855" (UID: "a8560aa2-26e2-465c-8b9b-66f84e2c9855"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:43:23.501462 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.501445    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8560aa2-26e2-465c-8b9b-66f84e2c9855-model-cache\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:43:23.501553 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.501468    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8560aa2-26e2-465c-8b9b-66f84e2c9855-tmp-dir\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:43:23.501553 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.501445    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8560aa2-26e2-465c-8b9b-66f84e2c9855-home" (OuterVolumeSpecName: "home") pod "a8560aa2-26e2-465c-8b9b-66f84e2c9855" (UID: "a8560aa2-26e2-465c-8b9b-66f84e2c9855"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:43:23.503126 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.503106    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8560aa2-26e2-465c-8b9b-66f84e2c9855-dshm" (OuterVolumeSpecName: "dshm") pod "a8560aa2-26e2-465c-8b9b-66f84e2c9855" (UID: "a8560aa2-26e2-465c-8b9b-66f84e2c9855"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:43:23.503234 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.503148    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8560aa2-26e2-465c-8b9b-66f84e2c9855-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a8560aa2-26e2-465c-8b9b-66f84e2c9855" (UID: "a8560aa2-26e2-465c-8b9b-66f84e2c9855"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:43:23.503298 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.503260    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8560aa2-26e2-465c-8b9b-66f84e2c9855-kube-api-access-lwgq7" (OuterVolumeSpecName: "kube-api-access-lwgq7") pod "a8560aa2-26e2-465c-8b9b-66f84e2c9855" (UID: "a8560aa2-26e2-465c-8b9b-66f84e2c9855"). InnerVolumeSpecName "kube-api-access-lwgq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:43:23.602051 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.602016    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8560aa2-26e2-465c-8b9b-66f84e2c9855-home\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:43:23.602051 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.602045    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8560aa2-26e2-465c-8b9b-66f84e2c9855-tls-certs\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:43:23.602051 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.602055    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8560aa2-26e2-465c-8b9b-66f84e2c9855-dshm\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:43:23.602304 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.602064    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwgq7\" (UniqueName: \"kubernetes.io/projected/a8560aa2-26e2-465c-8b9b-66f84e2c9855-kube-api-access-lwgq7\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:43:23.801305 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.801278    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"]
Sep 08 15:43:23.805577 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:23.805552    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-9ca1fc/rate-test-model-9ca1fc-kserve-565c9d6455-njzlw"]
Sep 08 15:43:25.469188 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:25.469153    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l"
Sep 08 15:43:25.801202 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:25.801121    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8560aa2-26e2-465c-8b9b-66f84e2c9855" path="/var/lib/kubelet/pods/a8560aa2-26e2-465c-8b9b-66f84e2c9855/volumes"
Sep 08 15:43:27.448150 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:27.448113    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs"
Sep 08 15:43:41.418789 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:41.418755    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:43:41.519098 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:41.519048    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:43:42.398634 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:42.398598    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:43:42.493527 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:42.493494    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:43:43.562548 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:43.562513    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:43:43.660884 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:43.660848    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:43:43.810860 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:43.810826    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:43:44.070853 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:44.070818    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:43:44.162372 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:44.162341    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:43:44.168025 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:44.167995    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd"]
Sep 08 15:43:44.168467 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:44.168452    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8560aa2-26e2-465c-8b9b-66f84e2c9855" containerName="main"
Sep 08 15:43:44.168511 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:44.168469    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8560aa2-26e2-465c-8b9b-66f84e2c9855" containerName="main"
Sep 08 15:43:44.168557 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:44.168548    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8560aa2-26e2-465c-8b9b-66f84e2c9855" containerName="main"
Sep 08 15:43:44.171668 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:44.171653    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd"
Sep 08 15:43:44.173624 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:44.173607    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-0fea02\""
Sep 08 15:43:44.182599 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:44.182580    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd"]
Sep 08 15:43:44.300377 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:44.300342    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj57j\" (UniqueName: \"kubernetes.io/projected/ea7ba1d4-4058-41e9-9863-aa9874b9295a-kube-api-access-kj57j\") pod \"payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd\" (UID: \"ea7ba1d4-4058-41e9-9863-aa9874b9295a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd"
Sep 08 15:43:44.300548 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:44.300494    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea7ba1d4-4058-41e9-9863-aa9874b9295a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd\" (UID: \"ea7ba1d4-4058-41e9-9863-aa9874b9295a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd"
Sep 08 15:43:44.401421 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:44.401326    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea7ba1d4-4058-41e9-9863-aa9874b9295a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd\" (UID: \"ea7ba1d4-4058-41e9-9863-aa9874b9295a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd"
Sep 08 15:43:44.401421 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:44.401383    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kj57j\" (UniqueName: \"kubernetes.io/projected/ea7ba1d4-4058-41e9-9863-aa9874b9295a-kube-api-access-kj57j\") pod \"payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd\" (UID: \"ea7ba1d4-4058-41e9-9863-aa9874b9295a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd"
Sep 08 15:43:44.402014 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:44.401993    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea7ba1d4-4058-41e9-9863-aa9874b9295a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd\" (UID: \"ea7ba1d4-4058-41e9-9863-aa9874b9295a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd"
Sep 08 15:43:44.410694 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:44.410661    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj57j\" (UniqueName: \"kubernetes.io/projected/ea7ba1d4-4058-41e9-9863-aa9874b9295a-kube-api-access-kj57j\") pod \"payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd\" (UID: \"ea7ba1d4-4058-41e9-9863-aa9874b9295a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd"
Sep 08 15:43:44.482343 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:44.482310    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd"
Sep 08 15:43:44.755463 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:44.755434    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd"]
Sep 08 15:43:44.757280 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:43:44.757248    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea7ba1d4_4058_41e9_9863_aa9874b9295a.slice/crio-3b02c7ed8bb6aabe0aacd7e836dab4853e89c5575b9618c9118e6746e26099ef WatchSource:0}: Error finding container 3b02c7ed8bb6aabe0aacd7e836dab4853e89c5575b9618c9118e6746e26099ef: Status 404 returned error can't find the container with id 3b02c7ed8bb6aabe0aacd7e836dab4853e89c5575b9618c9118e6746e26099ef
Sep 08 15:43:45.592263 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:45.592228    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd" event={"ID":"ea7ba1d4-4058-41e9-9863-aa9874b9295a","Type":"ContainerStarted","Data":"1a37837c0af92a07ae7050d617091fd817870d1454dffc7ae52f2115e92d97b8"}
Sep 08 15:43:45.592469 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:45.592287    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd" event={"ID":"ea7ba1d4-4058-41e9-9863-aa9874b9295a","Type":"ContainerStarted","Data":"3b02c7ed8bb6aabe0aacd7e836dab4853e89c5575b9618c9118e6746e26099ef"}
Sep 08 15:43:45.592469 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:45.592361    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd"
Sep 08 15:43:45.611880 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:45.611828    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd" podStartSLOduration=1.611812096 podStartE2EDuration="1.611812096s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:43:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:43:45.608578472 +0000 UTC m=+1578.320520963" watchObservedRunningTime="2026-09-08 15:43:45.611812096 +0000 UTC m=+1578.323754585"
Sep 08 15:43:56.070838 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:56.070803    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:43:56.599614 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:43:56.599584    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd"
Sep 08 15:44:07.351099 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:07.351042    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:44:12.395686 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:12.395654    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-zdsqr_77f92f27-262b-498d-a1fa-94ece7554033/payload-processing/0.log"
Sep 08 15:44:17.398347 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:17.398310    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:44:41.656267 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:41.656185    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:44:50.391968 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:50.391906    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs_72e29e11-a7fb-4daf-b867-9f43ab5074ff/payload-processing/0.log"
Sep 08 15:44:50.530057 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:50.530025    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-zdsqr_77f92f27-262b-498d-a1fa-94ece7554033/payload-processing/0.log"
Sep 08 15:44:50.964313 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:50.964272    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:44:51.332500 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:51.332425    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:44:51.427677 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:51.427645    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:44:54.858157 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:54.858123    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:44:54.951362 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:54.951333    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:44:56.026834 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.026794    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds"]
Sep 08 15:44:56.031302 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.031279    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds"
Sep 08 15:44:56.033651 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.033631    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-21c704-serving-cert\""
Sep 08 15:44:56.051104 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.051064    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds"]
Sep 08 15:44:56.085694 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.085660    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:44:56.151923 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.151887    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca5908a4-9f4c-403b-beb2-ea5aba552480-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds\" (UID: \"ca5908a4-9f4c-403b-beb2-ea5aba552480\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds"
Sep 08 15:44:56.152130 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.151943    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llhnw\" (UniqueName: \"kubernetes.io/projected/ca5908a4-9f4c-403b-beb2-ea5aba552480-kube-api-access-llhnw\") pod \"maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds\" (UID: \"ca5908a4-9f4c-403b-beb2-ea5aba552480\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds"
Sep 08 15:44:56.152130 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.152054    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca5908a4-9f4c-403b-beb2-ea5aba552480-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds\" (UID: \"ca5908a4-9f4c-403b-beb2-ea5aba552480\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds"
Sep 08 15:44:56.152267 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.152138    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca5908a4-9f4c-403b-beb2-ea5aba552480-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds\" (UID: \"ca5908a4-9f4c-403b-beb2-ea5aba552480\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds"
Sep 08 15:44:56.199751 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.199717    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:44:56.253116 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.253077    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llhnw\" (UniqueName: \"kubernetes.io/projected/ca5908a4-9f4c-403b-beb2-ea5aba552480-kube-api-access-llhnw\") pod \"maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds\" (UID: \"ca5908a4-9f4c-403b-beb2-ea5aba552480\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds"
Sep 08 15:44:56.253257 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.253192    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca5908a4-9f4c-403b-beb2-ea5aba552480-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds\" (UID: \"ca5908a4-9f4c-403b-beb2-ea5aba552480\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds"
Sep 08 15:44:56.253257 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.253217    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca5908a4-9f4c-403b-beb2-ea5aba552480-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds\" (UID: \"ca5908a4-9f4c-403b-beb2-ea5aba552480\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds"
Sep 08 15:44:56.253346 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.253263    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca5908a4-9f4c-403b-beb2-ea5aba552480-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds\" (UID: \"ca5908a4-9f4c-403b-beb2-ea5aba552480\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds"
Sep 08 15:44:56.253887 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.253869    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca5908a4-9f4c-403b-beb2-ea5aba552480-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds\" (UID: \"ca5908a4-9f4c-403b-beb2-ea5aba552480\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds"
Sep 08 15:44:56.255757 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.255730    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca5908a4-9f4c-403b-beb2-ea5aba552480-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds\" (UID: \"ca5908a4-9f4c-403b-beb2-ea5aba552480\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds"
Sep 08 15:44:56.255862 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.255772    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca5908a4-9f4c-403b-beb2-ea5aba552480-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds\" (UID: \"ca5908a4-9f4c-403b-beb2-ea5aba552480\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds"
Sep 08 15:44:56.261202 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.261179    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llhnw\" (UniqueName: \"kubernetes.io/projected/ca5908a4-9f4c-403b-beb2-ea5aba552480-kube-api-access-llhnw\") pod \"maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds\" (UID: \"ca5908a4-9f4c-403b-beb2-ea5aba552480\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds"
Sep 08 15:44:56.290809 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.290738    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:44:56.343507 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.343474    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds"
Sep 08 15:44:56.632783 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.632183    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds"]
Sep 08 15:44:56.636963 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:44:56.636937    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca5908a4_9f4c_403b_beb2_ea5aba552480.slice/crio-5798ac2f89f92ff0235c2596919059471fc670ebac921c53f74273eaf2124e45 WatchSource:0}: Error finding container 5798ac2f89f92ff0235c2596919059471fc670ebac921c53f74273eaf2124e45: Status 404 returned error can't find the container with id 5798ac2f89f92ff0235c2596919059471fc670ebac921c53f74273eaf2124e45
Sep 08 15:44:56.703229 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.703197    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:44:56.811017 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.810963    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:44:56.951757 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.951720    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds" event={"ID":"ca5908a4-9f4c-403b-beb2-ea5aba552480","Type":"ContainerStarted","Data":"5798ac2f89f92ff0235c2596919059471fc670ebac921c53f74273eaf2124e45"}
Sep 08 15:44:56.955188 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.955167    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x"]
Sep 08 15:44:56.959707 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.959691    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x"
Sep 08 15:44:56.962395 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.962376    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-21c704-dockercfg-ldj7z\""
Sep 08 15:44:56.964516 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.964500    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-21c704\""
Sep 08 15:44:56.974273 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:56.974252    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x"]
Sep 08 15:44:57.054707 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.054675    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7"]
Sep 08 15:44:57.058283 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.058266    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7"
Sep 08 15:44:57.059516 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.059492    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e1c47ad-bb88-4e17-9998-fa6c378945cc-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x\" (UID: \"8e1c47ad-bb88-4e17-9998-fa6c378945cc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x"
Sep 08 15:44:57.059620 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.059527    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zjvt\" (UniqueName: \"kubernetes.io/projected/8e1c47ad-bb88-4e17-9998-fa6c378945cc-kube-api-access-5zjvt\") pod \"payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x\" (UID: \"8e1c47ad-bb88-4e17-9998-fa6c378945cc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x"
Sep 08 15:44:57.073313 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.073292    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7"]
Sep 08 15:44:57.160902 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.160867    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e1c47ad-bb88-4e17-9998-fa6c378945cc-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x\" (UID: \"8e1c47ad-bb88-4e17-9998-fa6c378945cc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x"
Sep 08 15:44:57.161105 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.160910    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zjvt\" (UniqueName: \"kubernetes.io/projected/8e1c47ad-bb88-4e17-9998-fa6c378945cc-kube-api-access-5zjvt\") pod \"payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x\" (UID: \"8e1c47ad-bb88-4e17-9998-fa6c378945cc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x"
Sep 08 15:44:57.161105 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.160976    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glsm6\" (UniqueName: \"kubernetes.io/projected/8cf09db8-08dd-43d6-bd30-64583aa1d05e-kube-api-access-glsm6\") pod \"payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7\" (UID: \"8cf09db8-08dd-43d6-bd30-64583aa1d05e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7"
Sep 08 15:44:57.161105 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.161003    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cf09db8-08dd-43d6-bd30-64583aa1d05e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7\" (UID: \"8cf09db8-08dd-43d6-bd30-64583aa1d05e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7"
Sep 08 15:44:57.161625 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.161602    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e1c47ad-bb88-4e17-9998-fa6c378945cc-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x\" (UID: \"8e1c47ad-bb88-4e17-9998-fa6c378945cc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x"
Sep 08 15:44:57.169270 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.169250    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zjvt\" (UniqueName: \"kubernetes.io/projected/8e1c47ad-bb88-4e17-9998-fa6c378945cc-kube-api-access-5zjvt\") pod \"payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x\" (UID: \"8e1c47ad-bb88-4e17-9998-fa6c378945cc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x"
Sep 08 15:44:57.262606 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.262506    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glsm6\" (UniqueName: \"kubernetes.io/projected/8cf09db8-08dd-43d6-bd30-64583aa1d05e-kube-api-access-glsm6\") pod \"payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7\" (UID: \"8cf09db8-08dd-43d6-bd30-64583aa1d05e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7"
Sep 08 15:44:57.262606 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.262579    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cf09db8-08dd-43d6-bd30-64583aa1d05e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7\" (UID: \"8cf09db8-08dd-43d6-bd30-64583aa1d05e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7"
Sep 08 15:44:57.263435 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.263378    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cf09db8-08dd-43d6-bd30-64583aa1d05e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7\" (UID: \"8cf09db8-08dd-43d6-bd30-64583aa1d05e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7"
Sep 08 15:44:57.269895 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.269874    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x"
Sep 08 15:44:57.270284 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.270265    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glsm6\" (UniqueName: \"kubernetes.io/projected/8cf09db8-08dd-43d6-bd30-64583aa1d05e-kube-api-access-glsm6\") pod \"payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7\" (UID: \"8cf09db8-08dd-43d6-bd30-64583aa1d05e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7"
Sep 08 15:44:57.368169 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.368145    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7"
Sep 08 15:44:57.547324 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.547295    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x"]
Sep 08 15:44:57.549699 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:44:57.549671    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e1c47ad_bb88_4e17_9998_fa6c378945cc.slice/crio-342f7f50fa5f48b3ab3decbb8feb53aba54f0b9bf76a515898d13e66e0145492 WatchSource:0}: Error finding container 342f7f50fa5f48b3ab3decbb8feb53aba54f0b9bf76a515898d13e66e0145492: Status 404 returned error can't find the container with id 342f7f50fa5f48b3ab3decbb8feb53aba54f0b9bf76a515898d13e66e0145492
Sep 08 15:44:57.643982 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.643951    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7"]
Sep 08 15:44:57.648089 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:44:57.648041    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cf09db8_08dd_43d6_bd30_64583aa1d05e.slice/crio-719431646a563cf71452739962d7b01db6dab40aec8e5a2f85d224d5aed154db WatchSource:0}: Error finding container 719431646a563cf71452739962d7b01db6dab40aec8e5a2f85d224d5aed154db: Status 404 returned error can't find the container with id 719431646a563cf71452739962d7b01db6dab40aec8e5a2f85d224d5aed154db
Sep 08 15:44:57.960351 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.960310    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7" event={"ID":"8cf09db8-08dd-43d6-bd30-64583aa1d05e","Type":"ContainerStarted","Data":"4d106b458c379d162a4210912849ac5b48f3825f3d460132762e8ba2ba0b963f"}
Sep 08 15:44:57.960557 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.960375    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7" event={"ID":"8cf09db8-08dd-43d6-bd30-64583aa1d05e","Type":"ContainerStarted","Data":"719431646a563cf71452739962d7b01db6dab40aec8e5a2f85d224d5aed154db"}
Sep 08 15:44:57.960557 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.960449    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7"
Sep 08 15:44:57.962190 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.962162    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds" event={"ID":"ca5908a4-9f4c-403b-beb2-ea5aba552480","Type":"ContainerStarted","Data":"3356a7b9cbded818d3de17f955c6c73169e349fda4bbc282415985bc33e875a4"}
Sep 08 15:44:57.962349 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.962215    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds"
Sep 08 15:44:57.963915 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.963889    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x" event={"ID":"8e1c47ad-bb88-4e17-9998-fa6c378945cc","Type":"ContainerStarted","Data":"b300bfbe065d4ea1140aef822c91c9912fcc54ad77d60d6fef3d0eba1f1e0420"}
Sep 08 15:44:57.964058 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.963937    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x" event={"ID":"8e1c47ad-bb88-4e17-9998-fa6c378945cc","Type":"ContainerStarted","Data":"342f7f50fa5f48b3ab3decbb8feb53aba54f0b9bf76a515898d13e66e0145492"}
Sep 08 15:44:57.964160 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.964126    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x"
Sep 08 15:44:57.981123 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.981044    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7" podStartSLOduration=0.981023605 podStartE2EDuration="981.023605ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:44:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:44:57.979865528 +0000 UTC m=+1650.691808021" watchObservedRunningTime="2026-09-08 15:44:57.981023605 +0000 UTC m=+1650.692966097"
Sep 08 15:44:57.999131 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:57.999065    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds" podStartSLOduration=1.28027595 podStartE2EDuration="1.999048279s" totalImagesPullingTime="718.772329ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:44:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:44:57.997909503 +0000 UTC m=+1650.709851995" watchObservedRunningTime="2026-09-08 15:44:57.999048279 +0000 UTC m=+1650.710990772"
Sep 08 15:44:58.014871 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:44:58.014817    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x" podStartSLOduration=2.014799993 podStartE2EDuration="2.014799993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:44:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:44:58.013716205 +0000 UTC m=+1650.725658695" watchObservedRunningTime="2026-09-08 15:44:58.014799993 +0000 UTC m=+1650.726742483"
Sep 08 15:45:03.975523 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:03.975490    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds"
Sep 08 15:45:08.971282 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:08.971250    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x"
Sep 08 15:45:08.971700 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:08.971329    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7"
Sep 08 15:45:19.877713 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:19.877676    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:45:26.658990 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:26.657818    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds"]
Sep 08 15:45:26.658990 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:26.658098    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds" podUID="ca5908a4-9f4c-403b-beb2-ea5aba552480" containerName="maas-api" containerID="cri-o://3356a7b9cbded818d3de17f955c6c73169e349fda4bbc282415985bc33e875a4" gracePeriod=30
Sep 08 15:45:26.687190 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:26.687159    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:45:26.715469 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:26.715435    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7"]
Sep 08 15:45:26.715719 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:26.715691    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7" podUID="8cf09db8-08dd-43d6-bd30-64583aa1d05e" containerName="payload-processing" containerID="cri-o://4d106b458c379d162a4210912849ac5b48f3825f3d460132762e8ba2ba0b963f" gracePeriod=30
Sep 08 15:45:26.724134 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:26.724102    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x"]
Sep 08 15:45:26.724547 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:26.724518    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x" podUID="8e1c47ad-bb88-4e17-9998-fa6c378945cc" containerName="payload-pre-processing" containerID="cri-o://b300bfbe065d4ea1140aef822c91c9912fcc54ad77d60d6fef3d0eba1f1e0420" gracePeriod=30
Sep 08 15:45:26.766357 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:26.766327    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:45:26.846408 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:26.843783    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:45:26.925002 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:26.924978    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds"
Sep 08 15:45:26.953844 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:26.953392    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca5908a4-9f4c-403b-beb2-ea5aba552480-maas-api-tls\") pod \"ca5908a4-9f4c-403b-beb2-ea5aba552480\" (UID: \"ca5908a4-9f4c-403b-beb2-ea5aba552480\") "
Sep 08 15:45:26.953844 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:26.953456    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca5908a4-9f4c-403b-beb2-ea5aba552480-openshift-service-ca\") pod \"ca5908a4-9f4c-403b-beb2-ea5aba552480\" (UID: \"ca5908a4-9f4c-403b-beb2-ea5aba552480\") "
Sep 08 15:45:26.954611 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:26.954207    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-llhnw\" (UniqueName: \"kubernetes.io/projected/ca5908a4-9f4c-403b-beb2-ea5aba552480-kube-api-access-llhnw\") pod \"ca5908a4-9f4c-403b-beb2-ea5aba552480\" (UID: \"ca5908a4-9f4c-403b-beb2-ea5aba552480\") "
Sep 08 15:45:26.954611 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:26.954300    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca5908a4-9f4c-403b-beb2-ea5aba552480-metrics-certs\") pod \"ca5908a4-9f4c-403b-beb2-ea5aba552480\" (UID: \"ca5908a4-9f4c-403b-beb2-ea5aba552480\") "
Sep 08 15:45:26.954781 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:26.954635    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca5908a4-9f4c-403b-beb2-ea5aba552480-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ca5908a4-9f4c-403b-beb2-ea5aba552480" (UID: "ca5908a4-9f4c-403b-beb2-ea5aba552480"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:45:26.955323 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:26.955253    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:45:26.957306 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:26.957273    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca5908a4-9f4c-403b-beb2-ea5aba552480-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ca5908a4-9f4c-403b-beb2-ea5aba552480" (UID: "ca5908a4-9f4c-403b-beb2-ea5aba552480"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:45:26.957836 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:26.957798    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca5908a4-9f4c-403b-beb2-ea5aba552480-kube-api-access-llhnw" (OuterVolumeSpecName: "kube-api-access-llhnw") pod "ca5908a4-9f4c-403b-beb2-ea5aba552480" (UID: "ca5908a4-9f4c-403b-beb2-ea5aba552480"). InnerVolumeSpecName "kube-api-access-llhnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:45:26.958863 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:26.958839    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca5908a4-9f4c-403b-beb2-ea5aba552480-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ca5908a4-9f4c-403b-beb2-ea5aba552480" (UID: "ca5908a4-9f4c-403b-beb2-ea5aba552480"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:45:27.005742 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.005715    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7"
Sep 08 15:45:27.012272 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.012250    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x"
Sep 08 15:45:27.055790 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.055761    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e1c47ad-bb88-4e17-9998-fa6c378945cc-plugins-config-volume\") pod \"8e1c47ad-bb88-4e17-9998-fa6c378945cc\" (UID: \"8e1c47ad-bb88-4e17-9998-fa6c378945cc\") "
Sep 08 15:45:27.055966 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.055878    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cf09db8-08dd-43d6-bd30-64583aa1d05e-plugins-config-volume\") pod \"8cf09db8-08dd-43d6-bd30-64583aa1d05e\" (UID: \"8cf09db8-08dd-43d6-bd30-64583aa1d05e\") "
Sep 08 15:45:27.055966 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.055909    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5zjvt\" (UniqueName: \"kubernetes.io/projected/8e1c47ad-bb88-4e17-9998-fa6c378945cc-kube-api-access-5zjvt\") pod \"8e1c47ad-bb88-4e17-9998-fa6c378945cc\" (UID: \"8e1c47ad-bb88-4e17-9998-fa6c378945cc\") "
Sep 08 15:45:27.055966 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.055954    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-glsm6\" (UniqueName: \"kubernetes.io/projected/8cf09db8-08dd-43d6-bd30-64583aa1d05e-kube-api-access-glsm6\") pod \"8cf09db8-08dd-43d6-bd30-64583aa1d05e\" (UID: \"8cf09db8-08dd-43d6-bd30-64583aa1d05e\") "
Sep 08 15:45:27.056343 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.056215    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e1c47ad-bb88-4e17-9998-fa6c378945cc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8e1c47ad-bb88-4e17-9998-fa6c378945cc" (UID: "8e1c47ad-bb88-4e17-9998-fa6c378945cc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:45:27.056343 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.056245    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca5908a4-9f4c-403b-beb2-ea5aba552480-maas-api-tls\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:45:27.056343 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.056268    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca5908a4-9f4c-403b-beb2-ea5aba552480-openshift-service-ca\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:45:27.056343 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.056282    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-llhnw\" (UniqueName: \"kubernetes.io/projected/ca5908a4-9f4c-403b-beb2-ea5aba552480-kube-api-access-llhnw\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:45:27.056343 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.056295    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca5908a4-9f4c-403b-beb2-ea5aba552480-metrics-certs\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:45:27.056343 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.056243    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cf09db8-08dd-43d6-bd30-64583aa1d05e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8cf09db8-08dd-43d6-bd30-64583aa1d05e" (UID: "8cf09db8-08dd-43d6-bd30-64583aa1d05e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:45:27.058081 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.058049    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e1c47ad-bb88-4e17-9998-fa6c378945cc-kube-api-access-5zjvt" (OuterVolumeSpecName: "kube-api-access-5zjvt") pod "8e1c47ad-bb88-4e17-9998-fa6c378945cc" (UID: "8e1c47ad-bb88-4e17-9998-fa6c378945cc"). InnerVolumeSpecName "kube-api-access-5zjvt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:45:27.058708 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.058687    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cf09db8-08dd-43d6-bd30-64583aa1d05e-kube-api-access-glsm6" (OuterVolumeSpecName: "kube-api-access-glsm6") pod "8cf09db8-08dd-43d6-bd30-64583aa1d05e" (UID: "8cf09db8-08dd-43d6-bd30-64583aa1d05e"). InnerVolumeSpecName "kube-api-access-glsm6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:45:27.119522 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.119481    2560 generic.go:356] "Generic (PLEG): container finished" podID="8cf09db8-08dd-43d6-bd30-64583aa1d05e" containerID="4d106b458c379d162a4210912849ac5b48f3825f3d460132762e8ba2ba0b963f" exitCode=0
Sep 08 15:45:27.119717 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.119536    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7"
Sep 08 15:45:27.119717 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.119578    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7" event={"ID":"8cf09db8-08dd-43d6-bd30-64583aa1d05e","Type":"ContainerDied","Data":"4d106b458c379d162a4210912849ac5b48f3825f3d460132762e8ba2ba0b963f"}
Sep 08 15:45:27.119717 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.119640    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7" event={"ID":"8cf09db8-08dd-43d6-bd30-64583aa1d05e","Type":"ContainerDied","Data":"719431646a563cf71452739962d7b01db6dab40aec8e5a2f85d224d5aed154db"}
Sep 08 15:45:27.119717 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.119663    2560 scope.go:117] "RemoveContainer" containerID="4d106b458c379d162a4210912849ac5b48f3825f3d460132762e8ba2ba0b963f"
Sep 08 15:45:27.121123 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.121099    2560 generic.go:356] "Generic (PLEG): container finished" podID="ca5908a4-9f4c-403b-beb2-ea5aba552480" containerID="3356a7b9cbded818d3de17f955c6c73169e349fda4bbc282415985bc33e875a4" exitCode=0
Sep 08 15:45:27.121384 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.121145    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds"
Sep 08 15:45:27.121384 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.121232    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds" event={"ID":"ca5908a4-9f4c-403b-beb2-ea5aba552480","Type":"ContainerDied","Data":"3356a7b9cbded818d3de17f955c6c73169e349fda4bbc282415985bc33e875a4"}
Sep 08 15:45:27.121384 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.121286    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds" event={"ID":"ca5908a4-9f4c-403b-beb2-ea5aba552480","Type":"ContainerDied","Data":"5798ac2f89f92ff0235c2596919059471fc670ebac921c53f74273eaf2124e45"}
Sep 08 15:45:27.122859 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.122832    2560 generic.go:356] "Generic (PLEG): container finished" podID="8e1c47ad-bb88-4e17-9998-fa6c378945cc" containerID="b300bfbe065d4ea1140aef822c91c9912fcc54ad77d60d6fef3d0eba1f1e0420" exitCode=0
Sep 08 15:45:27.122988 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.122879    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x"
Sep 08 15:45:27.122988 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.122910    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x" event={"ID":"8e1c47ad-bb88-4e17-9998-fa6c378945cc","Type":"ContainerDied","Data":"b300bfbe065d4ea1140aef822c91c9912fcc54ad77d60d6fef3d0eba1f1e0420"}
Sep 08 15:45:27.122988 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.122947    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x" event={"ID":"8e1c47ad-bb88-4e17-9998-fa6c378945cc","Type":"ContainerDied","Data":"342f7f50fa5f48b3ab3decbb8feb53aba54f0b9bf76a515898d13e66e0145492"}
Sep 08 15:45:27.131493 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.131414    2560 scope.go:117] "RemoveContainer" containerID="4d106b458c379d162a4210912849ac5b48f3825f3d460132762e8ba2ba0b963f"
Sep 08 15:45:27.131835 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:45:27.131805    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d106b458c379d162a4210912849ac5b48f3825f3d460132762e8ba2ba0b963f\": container with ID starting with 4d106b458c379d162a4210912849ac5b48f3825f3d460132762e8ba2ba0b963f not found: ID does not exist" containerID="4d106b458c379d162a4210912849ac5b48f3825f3d460132762e8ba2ba0b963f"
Sep 08 15:45:27.131938 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.131847    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d106b458c379d162a4210912849ac5b48f3825f3d460132762e8ba2ba0b963f"} err="failed to get container status \"4d106b458c379d162a4210912849ac5b48f3825f3d460132762e8ba2ba0b963f\": rpc error: code = NotFound desc = could not find container \"4d106b458c379d162a4210912849ac5b48f3825f3d460132762e8ba2ba0b963f\": container with ID starting with 4d106b458c379d162a4210912849ac5b48f3825f3d460132762e8ba2ba0b963f not found: ID does not exist"
Sep 08 15:45:27.131938 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.131868    2560 scope.go:117] "RemoveContainer" containerID="3356a7b9cbded818d3de17f955c6c73169e349fda4bbc282415985bc33e875a4"
Sep 08 15:45:27.140736 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.140718    2560 scope.go:117] "RemoveContainer" containerID="3356a7b9cbded818d3de17f955c6c73169e349fda4bbc282415985bc33e875a4"
Sep 08 15:45:27.141043 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:45:27.141020    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3356a7b9cbded818d3de17f955c6c73169e349fda4bbc282415985bc33e875a4\": container with ID starting with 3356a7b9cbded818d3de17f955c6c73169e349fda4bbc282415985bc33e875a4 not found: ID does not exist" containerID="3356a7b9cbded818d3de17f955c6c73169e349fda4bbc282415985bc33e875a4"
Sep 08 15:45:27.141140 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.141052    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3356a7b9cbded818d3de17f955c6c73169e349fda4bbc282415985bc33e875a4"} err="failed to get container status \"3356a7b9cbded818d3de17f955c6c73169e349fda4bbc282415985bc33e875a4\": rpc error: code = NotFound desc = could not find container \"3356a7b9cbded818d3de17f955c6c73169e349fda4bbc282415985bc33e875a4\": container with ID starting with 3356a7b9cbded818d3de17f955c6c73169e349fda4bbc282415985bc33e875a4 not found: ID does not exist"
Sep 08 15:45:27.141140 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.141088    2560 scope.go:117] "RemoveContainer" containerID="b300bfbe065d4ea1140aef822c91c9912fcc54ad77d60d6fef3d0eba1f1e0420"
Sep 08 15:45:27.149576 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.149558    2560 scope.go:117] "RemoveContainer" containerID="b300bfbe065d4ea1140aef822c91c9912fcc54ad77d60d6fef3d0eba1f1e0420"
Sep 08 15:45:27.149866 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:45:27.149842    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b300bfbe065d4ea1140aef822c91c9912fcc54ad77d60d6fef3d0eba1f1e0420\": container with ID starting with b300bfbe065d4ea1140aef822c91c9912fcc54ad77d60d6fef3d0eba1f1e0420 not found: ID does not exist" containerID="b300bfbe065d4ea1140aef822c91c9912fcc54ad77d60d6fef3d0eba1f1e0420"
Sep 08 15:45:27.149940 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.149873    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b300bfbe065d4ea1140aef822c91c9912fcc54ad77d60d6fef3d0eba1f1e0420"} err="failed to get container status \"b300bfbe065d4ea1140aef822c91c9912fcc54ad77d60d6fef3d0eba1f1e0420\": rpc error: code = NotFound desc = could not find container \"b300bfbe065d4ea1140aef822c91c9912fcc54ad77d60d6fef3d0eba1f1e0420\": container with ID starting with b300bfbe065d4ea1140aef822c91c9912fcc54ad77d60d6fef3d0eba1f1e0420 not found: ID does not exist"
Sep 08 15:45:27.156937 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.156913    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e1c47ad-bb88-4e17-9998-fa6c378945cc-plugins-config-volume\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:45:27.156937 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.156937    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cf09db8-08dd-43d6-bd30-64583aa1d05e-plugins-config-volume\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:45:27.157054 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.156946    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zjvt\" (UniqueName: \"kubernetes.io/projected/8e1c47ad-bb88-4e17-9998-fa6c378945cc-kube-api-access-5zjvt\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:45:27.157054 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.156956    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glsm6\" (UniqueName: \"kubernetes.io/projected/8cf09db8-08dd-43d6-bd30-64583aa1d05e-kube-api-access-glsm6\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:45:27.165543 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.165517    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x"]
Sep 08 15:45:27.167954 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.167934    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-21c704-84f75dcf58-pgf2x"]
Sep 08 15:45:27.181488 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.181439    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7"]
Sep 08 15:45:27.192644 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.192626    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-21c704-555755764f-jckq7"]
Sep 08 15:45:27.204115 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.204089    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds"]
Sep 08 15:45:27.206122 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.206099    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-21c704-5cf5ddd49b-7dtds"]
Sep 08 15:45:27.271743 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.271712    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:45:27.806858 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.806821    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cf09db8-08dd-43d6-bd30-64583aa1d05e" path="/var/lib/kubelet/pods/8cf09db8-08dd-43d6-bd30-64583aa1d05e/volumes"
Sep 08 15:45:27.807293 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.807264    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e1c47ad-bb88-4e17-9998-fa6c378945cc" path="/var/lib/kubelet/pods/8e1c47ad-bb88-4e17-9998-fa6c378945cc/volumes"
Sep 08 15:45:27.807622 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:27.807608    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca5908a4-9f4c-403b-beb2-ea5aba552480" path="/var/lib/kubelet/pods/ca5908a4-9f4c-403b-beb2-ea5aba552480/volumes"
Sep 08 15:45:33.486693 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:33.486661    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:45:33.587910 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:33.587868    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:45:33.757840 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:33.757760    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:45:37.541977 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:37.541938    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:45:44.420839 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:44.420803    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:45:47.568902 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:47.568867    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:45:53.208180 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.208056    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l"]
Sep 08 15:45:53.208647 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.208341    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l" podUID="55d135e7-bf30-450e-89a9-58f53bb65514" containerName="maas-api" containerID="cri-o://90ebeb8c1640c6e793a55f4514ff3e6aa0e9190748adc16f898705db2977ce9f" gracePeriod=30
Sep 08 15:45:53.239453 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.239413    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:45:53.263367 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.263336    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs"]
Sep 08 15:45:53.263695 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.263534    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs" podUID="72e29e11-a7fb-4daf-b867-9f43ab5074ff" containerName="payload-processing" containerID="cri-o://5455422e041db207621e33e32af49e92e16f52d29131d515501f980fd5fa8d65" gracePeriod=30
Sep 08 15:45:53.340238 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.340179    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:45:53.442884 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.442800    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:45:53.511971 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.511945    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l"
Sep 08 15:45:53.535845 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.535818    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:45:53.555711 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.555688    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs"
Sep 08 15:45:53.597281 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.597251    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55d135e7-bf30-450e-89a9-58f53bb65514-openshift-service-ca\") pod \"55d135e7-bf30-450e-89a9-58f53bb65514\" (UID: \"55d135e7-bf30-450e-89a9-58f53bb65514\") "
Sep 08 15:45:53.597466 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.597306    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55d135e7-bf30-450e-89a9-58f53bb65514-metrics-certs\") pod \"55d135e7-bf30-450e-89a9-58f53bb65514\" (UID: \"55d135e7-bf30-450e-89a9-58f53bb65514\") "
Sep 08 15:45:53.597466 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.597379    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55d135e7-bf30-450e-89a9-58f53bb65514-maas-api-tls\") pod \"55d135e7-bf30-450e-89a9-58f53bb65514\" (UID: \"55d135e7-bf30-450e-89a9-58f53bb65514\") "
Sep 08 15:45:53.597466 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.597436    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-858b5\" (UniqueName: \"kubernetes.io/projected/55d135e7-bf30-450e-89a9-58f53bb65514-kube-api-access-858b5\") pod \"55d135e7-bf30-450e-89a9-58f53bb65514\" (UID: \"55d135e7-bf30-450e-89a9-58f53bb65514\") "
Sep 08 15:45:53.597702 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.597677    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55d135e7-bf30-450e-89a9-58f53bb65514-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "55d135e7-bf30-450e-89a9-58f53bb65514" (UID: "55d135e7-bf30-450e-89a9-58f53bb65514"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:45:53.599525 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.599501    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55d135e7-bf30-450e-89a9-58f53bb65514-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "55d135e7-bf30-450e-89a9-58f53bb65514" (UID: "55d135e7-bf30-450e-89a9-58f53bb65514"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:45:53.599634 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.599547    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55d135e7-bf30-450e-89a9-58f53bb65514-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "55d135e7-bf30-450e-89a9-58f53bb65514" (UID: "55d135e7-bf30-450e-89a9-58f53bb65514"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:45:53.599634 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.599616    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55d135e7-bf30-450e-89a9-58f53bb65514-kube-api-access-858b5" (OuterVolumeSpecName: "kube-api-access-858b5") pod "55d135e7-bf30-450e-89a9-58f53bb65514" (UID: "55d135e7-bf30-450e-89a9-58f53bb65514"). InnerVolumeSpecName "kube-api-access-858b5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:45:53.698871 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.698838    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72e29e11-a7fb-4daf-b867-9f43ab5074ff-plugins-config-volume\") pod \"72e29e11-a7fb-4daf-b867-9f43ab5074ff\" (UID: \"72e29e11-a7fb-4daf-b867-9f43ab5074ff\") "
Sep 08 15:45:53.699021 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.698881    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrgkk\" (UniqueName: \"kubernetes.io/projected/72e29e11-a7fb-4daf-b867-9f43ab5074ff-kube-api-access-mrgkk\") pod \"72e29e11-a7fb-4daf-b867-9f43ab5074ff\" (UID: \"72e29e11-a7fb-4daf-b867-9f43ab5074ff\") "
Sep 08 15:45:53.699106 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.699044    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55d135e7-bf30-450e-89a9-58f53bb65514-maas-api-tls\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:45:53.699106 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.699055    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-858b5\" (UniqueName: \"kubernetes.io/projected/55d135e7-bf30-450e-89a9-58f53bb65514-kube-api-access-858b5\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:45:53.699106 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.699065    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55d135e7-bf30-450e-89a9-58f53bb65514-openshift-service-ca\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:45:53.699106 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.699096    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55d135e7-bf30-450e-89a9-58f53bb65514-metrics-certs\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:45:53.699264 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.699222    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72e29e11-a7fb-4daf-b867-9f43ab5074ff-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "72e29e11-a7fb-4daf-b867-9f43ab5074ff" (UID: "72e29e11-a7fb-4daf-b867-9f43ab5074ff"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:45:53.700916 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.700895    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72e29e11-a7fb-4daf-b867-9f43ab5074ff-kube-api-access-mrgkk" (OuterVolumeSpecName: "kube-api-access-mrgkk") pod "72e29e11-a7fb-4daf-b867-9f43ab5074ff" (UID: "72e29e11-a7fb-4daf-b867-9f43ab5074ff"). InnerVolumeSpecName "kube-api-access-mrgkk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:45:53.800608 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.800525    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72e29e11-a7fb-4daf-b867-9f43ab5074ff-plugins-config-volume\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:45:53.800608 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.800556    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrgkk\" (UniqueName: \"kubernetes.io/projected/72e29e11-a7fb-4daf-b867-9f43ab5074ff-kube-api-access-mrgkk\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:45:53.828541 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:53.828512    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:45:54.257468 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:54.257436    2560 generic.go:356] "Generic (PLEG): container finished" podID="55d135e7-bf30-450e-89a9-58f53bb65514" containerID="90ebeb8c1640c6e793a55f4514ff3e6aa0e9190748adc16f898705db2977ce9f" exitCode=0
Sep 08 15:45:54.257897 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:54.257480    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l"
Sep 08 15:45:54.257897 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:54.257535    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l" event={"ID":"55d135e7-bf30-450e-89a9-58f53bb65514","Type":"ContainerDied","Data":"90ebeb8c1640c6e793a55f4514ff3e6aa0e9190748adc16f898705db2977ce9f"}
Sep 08 15:45:54.257897 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:54.257587    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l" event={"ID":"55d135e7-bf30-450e-89a9-58f53bb65514","Type":"ContainerDied","Data":"0c64c9739c2cec22df2bc32d8cdf8a8e1e2754fffd390e9dbf8d050427f1cbe9"}
Sep 08 15:45:54.257897 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:54.257616    2560 scope.go:117] "RemoveContainer" containerID="90ebeb8c1640c6e793a55f4514ff3e6aa0e9190748adc16f898705db2977ce9f"
Sep 08 15:45:54.259180 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:54.258886    2560 generic.go:356] "Generic (PLEG): container finished" podID="72e29e11-a7fb-4daf-b867-9f43ab5074ff" containerID="5455422e041db207621e33e32af49e92e16f52d29131d515501f980fd5fa8d65" exitCode=0
Sep 08 15:45:54.259180 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:54.258928    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs"
Sep 08 15:45:54.259180 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:54.258940    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs" event={"ID":"72e29e11-a7fb-4daf-b867-9f43ab5074ff","Type":"ContainerDied","Data":"5455422e041db207621e33e32af49e92e16f52d29131d515501f980fd5fa8d65"}
Sep 08 15:45:54.259180 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:54.258977    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs" event={"ID":"72e29e11-a7fb-4daf-b867-9f43ab5074ff","Type":"ContainerDied","Data":"b495ea8371ad0782b51b28b7a31fb7e6dfc16ffb1328eea0874d674a163f7baf"}
Sep 08 15:45:54.268693 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:54.268679    2560 scope.go:117] "RemoveContainer" containerID="90ebeb8c1640c6e793a55f4514ff3e6aa0e9190748adc16f898705db2977ce9f"
Sep 08 15:45:54.268990 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:45:54.268965    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90ebeb8c1640c6e793a55f4514ff3e6aa0e9190748adc16f898705db2977ce9f\": container with ID starting with 90ebeb8c1640c6e793a55f4514ff3e6aa0e9190748adc16f898705db2977ce9f not found: ID does not exist" containerID="90ebeb8c1640c6e793a55f4514ff3e6aa0e9190748adc16f898705db2977ce9f"
Sep 08 15:45:54.269110 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:54.269003    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90ebeb8c1640c6e793a55f4514ff3e6aa0e9190748adc16f898705db2977ce9f"} err="failed to get container status \"90ebeb8c1640c6e793a55f4514ff3e6aa0e9190748adc16f898705db2977ce9f\": rpc error: code = NotFound desc = could not find container \"90ebeb8c1640c6e793a55f4514ff3e6aa0e9190748adc16f898705db2977ce9f\": container with ID starting with 90ebeb8c1640c6e793a55f4514ff3e6aa0e9190748adc16f898705db2977ce9f not found: ID does not exist"
Sep 08 15:45:54.269110 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:54.269027    2560 scope.go:117] "RemoveContainer" containerID="5455422e041db207621e33e32af49e92e16f52d29131d515501f980fd5fa8d65"
Sep 08 15:45:54.276973 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:54.276954    2560 scope.go:117] "RemoveContainer" containerID="5455422e041db207621e33e32af49e92e16f52d29131d515501f980fd5fa8d65"
Sep 08 15:45:54.277294 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:45:54.277273    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5455422e041db207621e33e32af49e92e16f52d29131d515501f980fd5fa8d65\": container with ID starting with 5455422e041db207621e33e32af49e92e16f52d29131d515501f980fd5fa8d65 not found: ID does not exist" containerID="5455422e041db207621e33e32af49e92e16f52d29131d515501f980fd5fa8d65"
Sep 08 15:45:54.277375 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:54.277300    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5455422e041db207621e33e32af49e92e16f52d29131d515501f980fd5fa8d65"} err="failed to get container status \"5455422e041db207621e33e32af49e92e16f52d29131d515501f980fd5fa8d65\": rpc error: code = NotFound desc = could not find container \"5455422e041db207621e33e32af49e92e16f52d29131d515501f980fd5fa8d65\": container with ID starting with 5455422e041db207621e33e32af49e92e16f52d29131d515501f980fd5fa8d65 not found: ID does not exist"
Sep 08 15:45:54.279693 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:54.279671    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs"]
Sep 08 15:45:54.282096 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:54.282062    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-41de51-5d7bb4d857-lb9cs"]
Sep 08 15:45:54.291286 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:54.291265    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l"]
Sep 08 15:45:54.297146 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:54.297127    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-41de51-7d84d5fccb-sr99l"]
Sep 08 15:45:55.800764 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:55.800729    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55d135e7-bf30-450e-89a9-58f53bb65514" path="/var/lib/kubelet/pods/55d135e7-bf30-450e-89a9-58f53bb65514/volumes"
Sep 08 15:45:55.801132 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:55.801118    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72e29e11-a7fb-4daf-b867-9f43ab5074ff" path="/var/lib/kubelet/pods/72e29e11-a7fb-4daf-b867-9f43ab5074ff/volumes"
Sep 08 15:45:58.850011 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:58.849968    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:45:59.220353 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:59.220314    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:45:59.321300 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:45:59.321267    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:46:13.831903 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:46:13.831848    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:46:35.560601 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:46:35.560559    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:46:36.223626 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:46:36.223579    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:46:42.091707 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:46:42.091669    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:46:42.194145 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:46:42.194102    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:46:43.846727 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:46:43.846692    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:46:43.909086 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:46:43.909044    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:46:53.298997 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:46:53.298960    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd"]
Sep 08 15:46:53.299477 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:46:53.299255    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd" podUID="ea7ba1d4-4058-41e9-9863-aa9874b9295a" containerName="payload-pre-processing" containerID="cri-o://1a37837c0af92a07ae7050d617091fd817870d1454dffc7ae52f2115e92d97b8" gracePeriod=30
Sep 08 15:46:53.563106 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:46:53.563014    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd"
Sep 08 15:46:53.567695 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:46:53.567665    2560 generic.go:356] "Generic (PLEG): container finished" podID="ea7ba1d4-4058-41e9-9863-aa9874b9295a" containerID="1a37837c0af92a07ae7050d617091fd817870d1454dffc7ae52f2115e92d97b8" exitCode=0
Sep 08 15:46:53.567858 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:46:53.567735    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd"
Sep 08 15:46:53.567858 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:46:53.567753    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd" event={"ID":"ea7ba1d4-4058-41e9-9863-aa9874b9295a","Type":"ContainerDied","Data":"1a37837c0af92a07ae7050d617091fd817870d1454dffc7ae52f2115e92d97b8"}
Sep 08 15:46:53.567858 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:46:53.567831    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd" event={"ID":"ea7ba1d4-4058-41e9-9863-aa9874b9295a","Type":"ContainerDied","Data":"3b02c7ed8bb6aabe0aacd7e836dab4853e89c5575b9618c9118e6746e26099ef"}
Sep 08 15:46:53.568042 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:46:53.567862    2560 scope.go:117] "RemoveContainer" containerID="1a37837c0af92a07ae7050d617091fd817870d1454dffc7ae52f2115e92d97b8"
Sep 08 15:46:53.576906 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:46:53.576890    2560 scope.go:117] "RemoveContainer" containerID="1a37837c0af92a07ae7050d617091fd817870d1454dffc7ae52f2115e92d97b8"
Sep 08 15:46:53.577217 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:46:53.577189    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a37837c0af92a07ae7050d617091fd817870d1454dffc7ae52f2115e92d97b8\": container with ID starting with 1a37837c0af92a07ae7050d617091fd817870d1454dffc7ae52f2115e92d97b8 not found: ID does not exist" containerID="1a37837c0af92a07ae7050d617091fd817870d1454dffc7ae52f2115e92d97b8"
Sep 08 15:46:53.577331 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:46:53.577228    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a37837c0af92a07ae7050d617091fd817870d1454dffc7ae52f2115e92d97b8"} err="failed to get container status \"1a37837c0af92a07ae7050d617091fd817870d1454dffc7ae52f2115e92d97b8\": rpc error: code = NotFound desc = could not find container \"1a37837c0af92a07ae7050d617091fd817870d1454dffc7ae52f2115e92d97b8\": container with ID starting with 1a37837c0af92a07ae7050d617091fd817870d1454dffc7ae52f2115e92d97b8 not found: ID does not exist"
Sep 08 15:46:53.747970 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:46:53.747931    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kj57j\" (UniqueName: \"kubernetes.io/projected/ea7ba1d4-4058-41e9-9863-aa9874b9295a-kube-api-access-kj57j\") pod \"ea7ba1d4-4058-41e9-9863-aa9874b9295a\" (UID: \"ea7ba1d4-4058-41e9-9863-aa9874b9295a\") "
Sep 08 15:46:53.748184 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:46:53.747998    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea7ba1d4-4058-41e9-9863-aa9874b9295a-plugins-config-volume\") pod \"ea7ba1d4-4058-41e9-9863-aa9874b9295a\" (UID: \"ea7ba1d4-4058-41e9-9863-aa9874b9295a\") "
Sep 08 15:46:53.748398 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:46:53.748368    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea7ba1d4-4058-41e9-9863-aa9874b9295a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ea7ba1d4-4058-41e9-9863-aa9874b9295a" (UID: "ea7ba1d4-4058-41e9-9863-aa9874b9295a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:46:53.750054 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:46:53.750027    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea7ba1d4-4058-41e9-9863-aa9874b9295a-kube-api-access-kj57j" (OuterVolumeSpecName: "kube-api-access-kj57j") pod "ea7ba1d4-4058-41e9-9863-aa9874b9295a" (UID: "ea7ba1d4-4058-41e9-9863-aa9874b9295a"). InnerVolumeSpecName "kube-api-access-kj57j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:46:53.849290 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:46:53.849205    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kj57j\" (UniqueName: \"kubernetes.io/projected/ea7ba1d4-4058-41e9-9863-aa9874b9295a-kube-api-access-kj57j\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:46:53.849290 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:46:53.849238    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea7ba1d4-4058-41e9-9863-aa9874b9295a-plugins-config-volume\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:46:53.883420 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:46:53.883392    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd"]
Sep 08 15:46:53.887512 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:46:53.887488    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0fea02-6d7b9b49f8-cfbtd"]
Sep 08 15:46:54.730279 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:46:54.730218    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:46:54.795111 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:46:54.795064    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:46:55.799976 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:46:55.799941    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea7ba1d4-4058-41e9-9863-aa9874b9295a" path="/var/lib/kubelet/pods/ea7ba1d4-4058-41e9-9863-aa9874b9295a/volumes"
Sep 08 15:47:00.328834 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:00.328794    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:47:00.391616 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:00.391580    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:47:02.098220 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:02.098182    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:47:02.195519 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:02.195484    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:47:11.713664 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:11.713628    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw"]
Sep 08 15:47:11.714206 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:11.713814    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw" podUID="265fd771-aa9f-4b17-a164-b05ae1f0faa1" containerName="maas-api" containerID="cri-o://1b28102dcb0e8f0add9890674387f29cecf70a8d8d16558b034080430d914b47" gracePeriod=30
Sep 08 15:47:11.726683 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:11.726649    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:47:11.786440 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:11.786403    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm"]
Sep 08 15:47:11.786716 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:11.786681    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm" podUID="9ffccf45-ca2a-435b-9464-3f4aae9f6678" containerName="payload-pre-processing" containerID="cri-o://7dd6aa53722cdcf053b2d49d3b6d5b3c7342d6907c962168d21adf0430a6032d" gracePeriod=30
Sep 08 15:47:11.815693 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:11.815664    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:47:11.876574 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:11.876541    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:47:11.979431 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:11.978305    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:47:12.048579 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.048553    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw"
Sep 08 15:47:12.072715 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.072689    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:47:12.082380 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.082359    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm"
Sep 08 15:47:12.098621 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.098592    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/265fd771-aa9f-4b17-a164-b05ae1f0faa1-metrics-certs\") pod \"265fd771-aa9f-4b17-a164-b05ae1f0faa1\" (UID: \"265fd771-aa9f-4b17-a164-b05ae1f0faa1\") "
Sep 08 15:47:12.098742 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.098659    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/265fd771-aa9f-4b17-a164-b05ae1f0faa1-maas-api-tls\") pod \"265fd771-aa9f-4b17-a164-b05ae1f0faa1\" (UID: \"265fd771-aa9f-4b17-a164-b05ae1f0faa1\") "
Sep 08 15:47:12.098742 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.098684    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbcfs\" (UniqueName: \"kubernetes.io/projected/265fd771-aa9f-4b17-a164-b05ae1f0faa1-kube-api-access-hbcfs\") pod \"265fd771-aa9f-4b17-a164-b05ae1f0faa1\" (UID: \"265fd771-aa9f-4b17-a164-b05ae1f0faa1\") "
Sep 08 15:47:12.098742 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.098703    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/265fd771-aa9f-4b17-a164-b05ae1f0faa1-openshift-service-ca\") pod \"265fd771-aa9f-4b17-a164-b05ae1f0faa1\" (UID: \"265fd771-aa9f-4b17-a164-b05ae1f0faa1\") "
Sep 08 15:47:12.099207 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.099175    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/265fd771-aa9f-4b17-a164-b05ae1f0faa1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "265fd771-aa9f-4b17-a164-b05ae1f0faa1" (UID: "265fd771-aa9f-4b17-a164-b05ae1f0faa1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:47:12.100698 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.100672    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/265fd771-aa9f-4b17-a164-b05ae1f0faa1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "265fd771-aa9f-4b17-a164-b05ae1f0faa1" (UID: "265fd771-aa9f-4b17-a164-b05ae1f0faa1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:47:12.100776 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.100745    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/265fd771-aa9f-4b17-a164-b05ae1f0faa1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "265fd771-aa9f-4b17-a164-b05ae1f0faa1" (UID: "265fd771-aa9f-4b17-a164-b05ae1f0faa1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:47:12.100925 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.100898    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/265fd771-aa9f-4b17-a164-b05ae1f0faa1-kube-api-access-hbcfs" (OuterVolumeSpecName: "kube-api-access-hbcfs") pod "265fd771-aa9f-4b17-a164-b05ae1f0faa1" (UID: "265fd771-aa9f-4b17-a164-b05ae1f0faa1"). InnerVolumeSpecName "kube-api-access-hbcfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:47:12.199876 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.199844    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ffccf45-ca2a-435b-9464-3f4aae9f6678-plugins-config-volume\") pod \"9ffccf45-ca2a-435b-9464-3f4aae9f6678\" (UID: \"9ffccf45-ca2a-435b-9464-3f4aae9f6678\") "
Sep 08 15:47:12.200083 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.199904    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdlfd\" (UniqueName: \"kubernetes.io/projected/9ffccf45-ca2a-435b-9464-3f4aae9f6678-kube-api-access-gdlfd\") pod \"9ffccf45-ca2a-435b-9464-3f4aae9f6678\" (UID: \"9ffccf45-ca2a-435b-9464-3f4aae9f6678\") "
Sep 08 15:47:12.200168 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.200105    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/265fd771-aa9f-4b17-a164-b05ae1f0faa1-metrics-certs\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:47:12.200168 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.200118    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/265fd771-aa9f-4b17-a164-b05ae1f0faa1-maas-api-tls\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:47:12.200168 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.200128    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbcfs\" (UniqueName: \"kubernetes.io/projected/265fd771-aa9f-4b17-a164-b05ae1f0faa1-kube-api-access-hbcfs\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:47:12.200168 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.200139    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/265fd771-aa9f-4b17-a164-b05ae1f0faa1-openshift-service-ca\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:47:12.200422 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.200397    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ffccf45-ca2a-435b-9464-3f4aae9f6678-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9ffccf45-ca2a-435b-9464-3f4aae9f6678" (UID: "9ffccf45-ca2a-435b-9464-3f4aae9f6678"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:47:12.201920 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.201898    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ffccf45-ca2a-435b-9464-3f4aae9f6678-kube-api-access-gdlfd" (OuterVolumeSpecName: "kube-api-access-gdlfd") pod "9ffccf45-ca2a-435b-9464-3f4aae9f6678" (UID: "9ffccf45-ca2a-435b-9464-3f4aae9f6678"). InnerVolumeSpecName "kube-api-access-gdlfd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:47:12.301135 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.301033    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ffccf45-ca2a-435b-9464-3f4aae9f6678-plugins-config-volume\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:47:12.301135 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.301063    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdlfd\" (UniqueName: \"kubernetes.io/projected/9ffccf45-ca2a-435b-9464-3f4aae9f6678-kube-api-access-gdlfd\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:47:12.398674 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.398624    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:47:12.666260 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.666176    2560 generic.go:356] "Generic (PLEG): container finished" podID="265fd771-aa9f-4b17-a164-b05ae1f0faa1" containerID="1b28102dcb0e8f0add9890674387f29cecf70a8d8d16558b034080430d914b47" exitCode=0
Sep 08 15:47:12.666260 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.666233    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw"
Sep 08 15:47:12.666499 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.666270    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw" event={"ID":"265fd771-aa9f-4b17-a164-b05ae1f0faa1","Type":"ContainerDied","Data":"1b28102dcb0e8f0add9890674387f29cecf70a8d8d16558b034080430d914b47"}
Sep 08 15:47:12.666499 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.666363    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw" event={"ID":"265fd771-aa9f-4b17-a164-b05ae1f0faa1","Type":"ContainerDied","Data":"0f79cdb65d24ad999f3dc99e05f4ff4e0e46c0e62e3fe19103a15d9cdf05623e"}
Sep 08 15:47:12.666499 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.666394    2560 scope.go:117] "RemoveContainer" containerID="1b28102dcb0e8f0add9890674387f29cecf70a8d8d16558b034080430d914b47"
Sep 08 15:47:12.667877 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.667852    2560 generic.go:356] "Generic (PLEG): container finished" podID="9ffccf45-ca2a-435b-9464-3f4aae9f6678" containerID="7dd6aa53722cdcf053b2d49d3b6d5b3c7342d6907c962168d21adf0430a6032d" exitCode=0
Sep 08 15:47:12.668028 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.667901    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm" event={"ID":"9ffccf45-ca2a-435b-9464-3f4aae9f6678","Type":"ContainerDied","Data":"7dd6aa53722cdcf053b2d49d3b6d5b3c7342d6907c962168d21adf0430a6032d"}
Sep 08 15:47:12.668028 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.667917    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm"
Sep 08 15:47:12.668028 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.667938    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm" event={"ID":"9ffccf45-ca2a-435b-9464-3f4aae9f6678","Type":"ContainerDied","Data":"3308c9debaef89d031b6f254a3d2e7f6c4b1a9c1ea0df4579930a841670bd780"}
Sep 08 15:47:12.675770 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.675749    2560 scope.go:117] "RemoveContainer" containerID="1b28102dcb0e8f0add9890674387f29cecf70a8d8d16558b034080430d914b47"
Sep 08 15:47:12.676099 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:47:12.676045    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b28102dcb0e8f0add9890674387f29cecf70a8d8d16558b034080430d914b47\": container with ID starting with 1b28102dcb0e8f0add9890674387f29cecf70a8d8d16558b034080430d914b47 not found: ID does not exist" containerID="1b28102dcb0e8f0add9890674387f29cecf70a8d8d16558b034080430d914b47"
Sep 08 15:47:12.676184 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.676099    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b28102dcb0e8f0add9890674387f29cecf70a8d8d16558b034080430d914b47"} err="failed to get container status \"1b28102dcb0e8f0add9890674387f29cecf70a8d8d16558b034080430d914b47\": rpc error: code = NotFound desc = could not find container \"1b28102dcb0e8f0add9890674387f29cecf70a8d8d16558b034080430d914b47\": container with ID starting with 1b28102dcb0e8f0add9890674387f29cecf70a8d8d16558b034080430d914b47 not found: ID does not exist"
Sep 08 15:47:12.676184 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.676117    2560 scope.go:117] "RemoveContainer" containerID="7dd6aa53722cdcf053b2d49d3b6d5b3c7342d6907c962168d21adf0430a6032d"
Sep 08 15:47:12.684991 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.684972    2560 scope.go:117] "RemoveContainer" containerID="7dd6aa53722cdcf053b2d49d3b6d5b3c7342d6907c962168d21adf0430a6032d"
Sep 08 15:47:12.685247 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:47:12.685232    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dd6aa53722cdcf053b2d49d3b6d5b3c7342d6907c962168d21adf0430a6032d\": container with ID starting with 7dd6aa53722cdcf053b2d49d3b6d5b3c7342d6907c962168d21adf0430a6032d not found: ID does not exist" containerID="7dd6aa53722cdcf053b2d49d3b6d5b3c7342d6907c962168d21adf0430a6032d"
Sep 08 15:47:12.685304 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.685253    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dd6aa53722cdcf053b2d49d3b6d5b3c7342d6907c962168d21adf0430a6032d"} err="failed to get container status \"7dd6aa53722cdcf053b2d49d3b6d5b3c7342d6907c962168d21adf0430a6032d\": rpc error: code = NotFound desc = could not find container \"7dd6aa53722cdcf053b2d49d3b6d5b3c7342d6907c962168d21adf0430a6032d\": container with ID starting with 7dd6aa53722cdcf053b2d49d3b6d5b3c7342d6907c962168d21adf0430a6032d not found: ID does not exist"
Sep 08 15:47:12.689738 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.689718    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw"]
Sep 08 15:47:12.691475 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.691450    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ca1fc-7fc4bc4556-crxdw"]
Sep 08 15:47:12.701545 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.701526    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm"]
Sep 08 15:47:12.707082 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:12.707052    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9ca1fc-967f9c89d-xwfvm"]
Sep 08 15:47:13.799751 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:13.799718    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="265fd771-aa9f-4b17-a164-b05ae1f0faa1" path="/var/lib/kubelet/pods/265fd771-aa9f-4b17-a164-b05ae1f0faa1/volumes"
Sep 08 15:47:13.800144 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:13.800136    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ffccf45-ca2a-435b-9464-3f4aae9f6678" path="/var/lib/kubelet/pods/9ffccf45-ca2a-435b-9464-3f4aae9f6678/volumes"
Sep 08 15:47:19.143728 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:19.143689    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:47:19.244824 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:19.244785    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:47:19.460937 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:19.460907    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:47:20.052117 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:20.052040    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:47:24.325741 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:24.325704    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:47:27.818373 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:27.818344    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vc5n9_17e30810-dfd0-4e6f-9204-731d1a6479a3/console-operator/2.log"
Sep 08 15:47:27.822752 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:27.822729    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vc5n9_17e30810-dfd0-4e6f-9204-731d1a6479a3/console-operator/2.log"
Sep 08 15:47:30.324814 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:30.324777    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:47:30.526421 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:30.526389    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:47:30.577587 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:30.577497    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4"]
Sep 08 15:47:30.577790 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:30.577751    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4" podUID="cdca0a4b-6927-48dc-90b6-bb712a50d2cf" containerName="payload-processing" containerID="cri-o://2b0ef22f5e7ae017e13451cd143931ba8fa3e4e4c479a5c0f252755184ff36f7" gracePeriod=30
Sep 08 15:47:30.615589 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:30.615557    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:47:30.678257 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:30.678228    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:47:30.760410 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:30.760378    2560 generic.go:356] "Generic (PLEG): container finished" podID="cdca0a4b-6927-48dc-90b6-bb712a50d2cf" containerID="2b0ef22f5e7ae017e13451cd143931ba8fa3e4e4c479a5c0f252755184ff36f7" exitCode=0
Sep 08 15:47:30.760566 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:30.760461    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4" event={"ID":"cdca0a4b-6927-48dc-90b6-bb712a50d2cf","Type":"ContainerDied","Data":"2b0ef22f5e7ae017e13451cd143931ba8fa3e4e4c479a5c0f252755184ff36f7"}
Sep 08 15:47:30.779468 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:30.779424    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:47:30.849879 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:30.849845    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4"
Sep 08 15:47:30.872673 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:30.872643    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:47:30.975630 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:30.975596    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wl2kp\" (UniqueName: \"kubernetes.io/projected/cdca0a4b-6927-48dc-90b6-bb712a50d2cf-kube-api-access-wl2kp\") pod \"cdca0a4b-6927-48dc-90b6-bb712a50d2cf\" (UID: \"cdca0a4b-6927-48dc-90b6-bb712a50d2cf\") "
Sep 08 15:47:30.975804 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:30.975660    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdca0a4b-6927-48dc-90b6-bb712a50d2cf-plugins-config-volume\") pod \"cdca0a4b-6927-48dc-90b6-bb712a50d2cf\" (UID: \"cdca0a4b-6927-48dc-90b6-bb712a50d2cf\") "
Sep 08 15:47:30.976054 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:30.976028    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdca0a4b-6927-48dc-90b6-bb712a50d2cf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cdca0a4b-6927-48dc-90b6-bb712a50d2cf" (UID: "cdca0a4b-6927-48dc-90b6-bb712a50d2cf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:47:30.977711 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:30.977692    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdca0a4b-6927-48dc-90b6-bb712a50d2cf-kube-api-access-wl2kp" (OuterVolumeSpecName: "kube-api-access-wl2kp") pod "cdca0a4b-6927-48dc-90b6-bb712a50d2cf" (UID: "cdca0a4b-6927-48dc-90b6-bb712a50d2cf"). InnerVolumeSpecName "kube-api-access-wl2kp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:47:31.076995 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:31.076961    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wl2kp\" (UniqueName: \"kubernetes.io/projected/cdca0a4b-6927-48dc-90b6-bb712a50d2cf-kube-api-access-wl2kp\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:47:31.076995 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:31.076993    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdca0a4b-6927-48dc-90b6-bb712a50d2cf-plugins-config-volume\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:47:31.180007 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:31.179978    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:47:31.766844 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:31.766818    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4"
Sep 08 15:47:31.767222 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:31.766817    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4" event={"ID":"cdca0a4b-6927-48dc-90b6-bb712a50d2cf","Type":"ContainerDied","Data":"63b824f7f93b72557fc3fb9194343c7fd37fecd51911b590543a328c3979beed"}
Sep 08 15:47:31.767222 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:31.766950    2560 scope.go:117] "RemoveContainer" containerID="2b0ef22f5e7ae017e13451cd143931ba8fa3e4e4c479a5c0f252755184ff36f7"
Sep 08 15:47:31.789192 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:31.789155    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4"]
Sep 08 15:47:31.801774 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:31.801752    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-9fb31f-67794c679f-9jsr4"]
Sep 08 15:47:33.799491 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:33.799457    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdca0a4b-6927-48dc-90b6-bb712a50d2cf" path="/var/lib/kubelet/pods/cdca0a4b-6927-48dc-90b6-bb712a50d2cf/volumes"
Sep 08 15:47:37.271628 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:37.271596    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:47:37.371712 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:37.371673    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:47:37.549051 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:37.548953    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:47:40.016185 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:40.016146    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b994b57d8-tgtp7"]
Sep 08 15:47:40.016648 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:40.016373    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7b994b57d8-tgtp7" podUID="25db5961-2fd9-4e78-9397-ea9f5f2a9626" containerName="manager" containerID="cri-o://26a0d89a9e044617a4741d06e1e26d63d0105d7a590fadb9235dc05ba70a89db" gracePeriod=30
Sep 08 15:47:40.261699 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:40.261670    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b994b57d8-tgtp7"
Sep 08 15:47:40.362922 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:40.362841    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/25db5961-2fd9-4e78-9397-ea9f5f2a9626-webhook-cert\") pod \"25db5961-2fd9-4e78-9397-ea9f5f2a9626\" (UID: \"25db5961-2fd9-4e78-9397-ea9f5f2a9626\") "
Sep 08 15:47:40.362922 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:40.362891    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25db5961-2fd9-4e78-9397-ea9f5f2a9626-metrics-certs\") pod \"25db5961-2fd9-4e78-9397-ea9f5f2a9626\" (UID: \"25db5961-2fd9-4e78-9397-ea9f5f2a9626\") "
Sep 08 15:47:40.363106 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:40.362925    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfrbv\" (UniqueName: \"kubernetes.io/projected/25db5961-2fd9-4e78-9397-ea9f5f2a9626-kube-api-access-nfrbv\") pod \"25db5961-2fd9-4e78-9397-ea9f5f2a9626\" (UID: \"25db5961-2fd9-4e78-9397-ea9f5f2a9626\") "
Sep 08 15:47:40.364852 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:40.364817    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25db5961-2fd9-4e78-9397-ea9f5f2a9626-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "25db5961-2fd9-4e78-9397-ea9f5f2a9626" (UID: "25db5961-2fd9-4e78-9397-ea9f5f2a9626"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:47:40.364998 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:40.364930    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25db5961-2fd9-4e78-9397-ea9f5f2a9626-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "25db5961-2fd9-4e78-9397-ea9f5f2a9626" (UID: "25db5961-2fd9-4e78-9397-ea9f5f2a9626"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:47:40.364998 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:40.364944    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25db5961-2fd9-4e78-9397-ea9f5f2a9626-kube-api-access-nfrbv" (OuterVolumeSpecName: "kube-api-access-nfrbv") pod "25db5961-2fd9-4e78-9397-ea9f5f2a9626" (UID: "25db5961-2fd9-4e78-9397-ea9f5f2a9626"). InnerVolumeSpecName "kube-api-access-nfrbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:47:40.463623 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:40.463587    2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/25db5961-2fd9-4e78-9397-ea9f5f2a9626-webhook-cert\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:47:40.463623 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:40.463618    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25db5961-2fd9-4e78-9397-ea9f5f2a9626-metrics-certs\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:47:40.463623 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:40.463627    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfrbv\" (UniqueName: \"kubernetes.io/projected/25db5961-2fd9-4e78-9397-ea9f5f2a9626-kube-api-access-nfrbv\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:47:40.814993 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:40.814957    2560 generic.go:356] "Generic (PLEG): container finished" podID="25db5961-2fd9-4e78-9397-ea9f5f2a9626" containerID="26a0d89a9e044617a4741d06e1e26d63d0105d7a590fadb9235dc05ba70a89db" exitCode=0
Sep 08 15:47:40.815195 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:40.815004    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b994b57d8-tgtp7"
Sep 08 15:47:40.815195 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:40.815032    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b994b57d8-tgtp7" event={"ID":"25db5961-2fd9-4e78-9397-ea9f5f2a9626","Type":"ContainerDied","Data":"26a0d89a9e044617a4741d06e1e26d63d0105d7a590fadb9235dc05ba70a89db"}
Sep 08 15:47:40.815195 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:40.815109    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b994b57d8-tgtp7" event={"ID":"25db5961-2fd9-4e78-9397-ea9f5f2a9626","Type":"ContainerDied","Data":"70575cfb706a810f47cc3f98374f08e4812962e9681a58a795e67f38e4c238fe"}
Sep 08 15:47:40.815195 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:40.815135    2560 scope.go:117] "RemoveContainer" containerID="26a0d89a9e044617a4741d06e1e26d63d0105d7a590fadb9235dc05ba70a89db"
Sep 08 15:47:40.824933 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:40.824912    2560 scope.go:117] "RemoveContainer" containerID="26a0d89a9e044617a4741d06e1e26d63d0105d7a590fadb9235dc05ba70a89db"
Sep 08 15:47:40.825253 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:47:40.825235    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26a0d89a9e044617a4741d06e1e26d63d0105d7a590fadb9235dc05ba70a89db\": container with ID starting with 26a0d89a9e044617a4741d06e1e26d63d0105d7a590fadb9235dc05ba70a89db not found: ID does not exist" containerID="26a0d89a9e044617a4741d06e1e26d63d0105d7a590fadb9235dc05ba70a89db"
Sep 08 15:47:40.825325 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:40.825265    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26a0d89a9e044617a4741d06e1e26d63d0105d7a590fadb9235dc05ba70a89db"} err="failed to get container status \"26a0d89a9e044617a4741d06e1e26d63d0105d7a590fadb9235dc05ba70a89db\": rpc error: code = NotFound desc = could not find container \"26a0d89a9e044617a4741d06e1e26d63d0105d7a590fadb9235dc05ba70a89db\": container with ID starting with 26a0d89a9e044617a4741d06e1e26d63d0105d7a590fadb9235dc05ba70a89db not found: ID does not exist"
Sep 08 15:47:40.835661 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:40.835638    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b994b57d8-tgtp7"]
Sep 08 15:47:40.840643 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:40.840617    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7b994b57d8-tgtp7"]
Sep 08 15:47:41.801022 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:47:41.800981    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25db5961-2fd9-4e78-9397-ea9f5f2a9626" path="/var/lib/kubelet/pods/25db5961-2fd9-4e78-9397-ea9f5f2a9626/volumes"
Sep 08 15:48:41.626765 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.626731    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7b994b57d8-bkptb"]
Sep 08 15:48:41.627202 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627098    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea7ba1d4-4058-41e9-9863-aa9874b9295a" containerName="payload-pre-processing"
Sep 08 15:48:41.627202 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627111    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea7ba1d4-4058-41e9-9863-aa9874b9295a" containerName="payload-pre-processing"
Sep 08 15:48:41.627202 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627123    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ffccf45-ca2a-435b-9464-3f4aae9f6678" containerName="payload-pre-processing"
Sep 08 15:48:41.627202 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627130    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ffccf45-ca2a-435b-9464-3f4aae9f6678" containerName="payload-pre-processing"
Sep 08 15:48:41.627202 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627145    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25db5961-2fd9-4e78-9397-ea9f5f2a9626" containerName="manager"
Sep 08 15:48:41.627202 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627150    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="25db5961-2fd9-4e78-9397-ea9f5f2a9626" containerName="manager"
Sep 08 15:48:41.627202 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627157    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdca0a4b-6927-48dc-90b6-bb712a50d2cf" containerName="payload-processing"
Sep 08 15:48:41.627202 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627162    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdca0a4b-6927-48dc-90b6-bb712a50d2cf" containerName="payload-processing"
Sep 08 15:48:41.627202 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627168    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55d135e7-bf30-450e-89a9-58f53bb65514" containerName="maas-api"
Sep 08 15:48:41.627202 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627173    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="55d135e7-bf30-450e-89a9-58f53bb65514" containerName="maas-api"
Sep 08 15:48:41.627202 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627179    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca5908a4-9f4c-403b-beb2-ea5aba552480" containerName="maas-api"
Sep 08 15:48:41.627202 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627184    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca5908a4-9f4c-403b-beb2-ea5aba552480" containerName="maas-api"
Sep 08 15:48:41.627202 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627193    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72e29e11-a7fb-4daf-b867-9f43ab5074ff" containerName="payload-processing"
Sep 08 15:48:41.627202 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627198    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="72e29e11-a7fb-4daf-b867-9f43ab5074ff" containerName="payload-processing"
Sep 08 15:48:41.627202 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627205    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cf09db8-08dd-43d6-bd30-64583aa1d05e" containerName="payload-processing"
Sep 08 15:48:41.627202 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627210    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cf09db8-08dd-43d6-bd30-64583aa1d05e" containerName="payload-processing"
Sep 08 15:48:41.627848 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627217    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e1c47ad-bb88-4e17-9998-fa6c378945cc" containerName="payload-pre-processing"
Sep 08 15:48:41.627848 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627222    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e1c47ad-bb88-4e17-9998-fa6c378945cc" containerName="payload-pre-processing"
Sep 08 15:48:41.627848 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627234    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="265fd771-aa9f-4b17-a164-b05ae1f0faa1" containerName="maas-api"
Sep 08 15:48:41.627848 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627238    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="265fd771-aa9f-4b17-a164-b05ae1f0faa1" containerName="maas-api"
Sep 08 15:48:41.627848 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627759    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ffccf45-ca2a-435b-9464-3f4aae9f6678" containerName="payload-pre-processing"
Sep 08 15:48:41.627848 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627840    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea7ba1d4-4058-41e9-9863-aa9874b9295a" containerName="payload-pre-processing"
Sep 08 15:48:41.628160 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627854    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="72e29e11-a7fb-4daf-b867-9f43ab5074ff" containerName="payload-processing"
Sep 08 15:48:41.628160 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627867    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cf09db8-08dd-43d6-bd30-64583aa1d05e" containerName="payload-processing"
Sep 08 15:48:41.628160 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627890    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="55d135e7-bf30-450e-89a9-58f53bb65514" containerName="maas-api"
Sep 08 15:48:41.628160 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627907    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e1c47ad-bb88-4e17-9998-fa6c378945cc" containerName="payload-pre-processing"
Sep 08 15:48:41.628160 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627917    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="25db5961-2fd9-4e78-9397-ea9f5f2a9626" containerName="manager"
Sep 08 15:48:41.628160 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627928    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="265fd771-aa9f-4b17-a164-b05ae1f0faa1" containerName="maas-api"
Sep 08 15:48:41.628160 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627946    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca5908a4-9f4c-403b-beb2-ea5aba552480" containerName="maas-api"
Sep 08 15:48:41.628160 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.627957    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdca0a4b-6927-48dc-90b6-bb712a50d2cf" containerName="payload-processing"
Sep 08 15:48:41.631909 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.631884    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b994b57d8-bkptb"
Sep 08 15:48:41.635003 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.634977    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 08 15:48:41.635151 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.634977    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wpgkm\""
Sep 08 15:48:41.635151 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.634976    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 08 15:48:41.635151 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.635095    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 08 15:48:41.641900 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.641880    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b994b57d8-bkptb"]
Sep 08 15:48:41.688450 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.688419    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f932b9d5-e294-411a-b6b5-57f1ed77c0dc-metrics-certs\") pod \"maas-controller-7b994b57d8-bkptb\" (UID: \"f932b9d5-e294-411a-b6b5-57f1ed77c0dc\") " pod="opendatahub/maas-controller-7b994b57d8-bkptb"
Sep 08 15:48:41.688606 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.688537    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f932b9d5-e294-411a-b6b5-57f1ed77c0dc-webhook-cert\") pod \"maas-controller-7b994b57d8-bkptb\" (UID: \"f932b9d5-e294-411a-b6b5-57f1ed77c0dc\") " pod="opendatahub/maas-controller-7b994b57d8-bkptb"
Sep 08 15:48:41.688606 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.688571    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8gzs\" (UniqueName: \"kubernetes.io/projected/f932b9d5-e294-411a-b6b5-57f1ed77c0dc-kube-api-access-v8gzs\") pod \"maas-controller-7b994b57d8-bkptb\" (UID: \"f932b9d5-e294-411a-b6b5-57f1ed77c0dc\") " pod="opendatahub/maas-controller-7b994b57d8-bkptb"
Sep 08 15:48:41.789343 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.789306    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f932b9d5-e294-411a-b6b5-57f1ed77c0dc-webhook-cert\") pod \"maas-controller-7b994b57d8-bkptb\" (UID: \"f932b9d5-e294-411a-b6b5-57f1ed77c0dc\") " pod="opendatahub/maas-controller-7b994b57d8-bkptb"
Sep 08 15:48:41.789343 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.789344    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8gzs\" (UniqueName: \"kubernetes.io/projected/f932b9d5-e294-411a-b6b5-57f1ed77c0dc-kube-api-access-v8gzs\") pod \"maas-controller-7b994b57d8-bkptb\" (UID: \"f932b9d5-e294-411a-b6b5-57f1ed77c0dc\") " pod="opendatahub/maas-controller-7b994b57d8-bkptb"
Sep 08 15:48:41.789563 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.789416    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f932b9d5-e294-411a-b6b5-57f1ed77c0dc-metrics-certs\") pod \"maas-controller-7b994b57d8-bkptb\" (UID: \"f932b9d5-e294-411a-b6b5-57f1ed77c0dc\") " pod="opendatahub/maas-controller-7b994b57d8-bkptb"
Sep 08 15:48:41.791862 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.791828    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f932b9d5-e294-411a-b6b5-57f1ed77c0dc-webhook-cert\") pod \"maas-controller-7b994b57d8-bkptb\" (UID: \"f932b9d5-e294-411a-b6b5-57f1ed77c0dc\") " pod="opendatahub/maas-controller-7b994b57d8-bkptb"
Sep 08 15:48:41.791975 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.791867    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f932b9d5-e294-411a-b6b5-57f1ed77c0dc-metrics-certs\") pod \"maas-controller-7b994b57d8-bkptb\" (UID: \"f932b9d5-e294-411a-b6b5-57f1ed77c0dc\") " pod="opendatahub/maas-controller-7b994b57d8-bkptb"
Sep 08 15:48:41.800506 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.800481    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8gzs\" (UniqueName: \"kubernetes.io/projected/f932b9d5-e294-411a-b6b5-57f1ed77c0dc-kube-api-access-v8gzs\") pod \"maas-controller-7b994b57d8-bkptb\" (UID: \"f932b9d5-e294-411a-b6b5-57f1ed77c0dc\") " pod="opendatahub/maas-controller-7b994b57d8-bkptb"
Sep 08 15:48:41.944921 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:41.944883    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b994b57d8-bkptb"
Sep 08 15:48:42.217470 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:42.217445    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b994b57d8-bkptb"]
Sep 08 15:48:42.219635 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:48:42.219606    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf932b9d5_e294_411a_b6b5_57f1ed77c0dc.slice/crio-3c622cc4203e41aed83c3b86c430970aeddb72691892aae639103fcac595c601 WatchSource:0}: Error finding container 3c622cc4203e41aed83c3b86c430970aeddb72691892aae639103fcac595c601: Status 404 returned error can't find the container with id 3c622cc4203e41aed83c3b86c430970aeddb72691892aae639103fcac595c601
Sep 08 15:48:42.220972 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:42.220956    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 15:48:43.107823 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:43.107731    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b994b57d8-bkptb" event={"ID":"f932b9d5-e294-411a-b6b5-57f1ed77c0dc","Type":"ContainerStarted","Data":"cd96c9c0d8c8d932147ad247884d6746eacb9e27677e8a21c978b036a60b3e76"}
Sep 08 15:48:43.107823 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:43.107795    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b994b57d8-bkptb" event={"ID":"f932b9d5-e294-411a-b6b5-57f1ed77c0dc","Type":"ContainerStarted","Data":"3c622cc4203e41aed83c3b86c430970aeddb72691892aae639103fcac595c601"}
Sep 08 15:48:43.107823 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:43.107824    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7b994b57d8-bkptb"
Sep 08 15:48:43.128507 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:43.128430    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7b994b57d8-bkptb" podStartSLOduration=1.665803447 podStartE2EDuration="2.128412297s" totalImagesPullingTime="462.60885ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:48:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:48:43.12564288 +0000 UTC m=+1875.837585371" watchObservedRunningTime="2026-09-08 15:48:43.128412297 +0000 UTC m=+1875.840354803"
Sep 08 15:48:54.114767 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:48:54.114683    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7b994b57d8-bkptb"
Sep 08 15:49:01.934544 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:01.934510    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:49:03.376455 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:03.376419    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:49:03.442647 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:03.442601    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b994b57d8-bkptb"]
Sep 08 15:49:03.442981 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:03.442914    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7b994b57d8-bkptb" podUID="f932b9d5-e294-411a-b6b5-57f1ed77c0dc" containerName="manager" containerID="cri-o://cd96c9c0d8c8d932147ad247884d6746eacb9e27677e8a21c978b036a60b3e76" gracePeriod=30
Sep 08 15:49:03.692025 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:03.691997    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b994b57d8-bkptb"
Sep 08 15:49:03.772887 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:03.772858    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f932b9d5-e294-411a-b6b5-57f1ed77c0dc-webhook-cert\") pod \"f932b9d5-e294-411a-b6b5-57f1ed77c0dc\" (UID: \"f932b9d5-e294-411a-b6b5-57f1ed77c0dc\") "
Sep 08 15:49:03.773044 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:03.772902    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f932b9d5-e294-411a-b6b5-57f1ed77c0dc-metrics-certs\") pod \"f932b9d5-e294-411a-b6b5-57f1ed77c0dc\" (UID: \"f932b9d5-e294-411a-b6b5-57f1ed77c0dc\") "
Sep 08 15:49:03.773044 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:03.773013    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8gzs\" (UniqueName: \"kubernetes.io/projected/f932b9d5-e294-411a-b6b5-57f1ed77c0dc-kube-api-access-v8gzs\") pod \"f932b9d5-e294-411a-b6b5-57f1ed77c0dc\" (UID: \"f932b9d5-e294-411a-b6b5-57f1ed77c0dc\") "
Sep 08 15:49:03.775088 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:03.775044    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f932b9d5-e294-411a-b6b5-57f1ed77c0dc-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f932b9d5-e294-411a-b6b5-57f1ed77c0dc" (UID: "f932b9d5-e294-411a-b6b5-57f1ed77c0dc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:49:03.775214 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:03.775099    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f932b9d5-e294-411a-b6b5-57f1ed77c0dc-kube-api-access-v8gzs" (OuterVolumeSpecName: "kube-api-access-v8gzs") pod "f932b9d5-e294-411a-b6b5-57f1ed77c0dc" (UID: "f932b9d5-e294-411a-b6b5-57f1ed77c0dc"). InnerVolumeSpecName "kube-api-access-v8gzs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:49:03.775276 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:03.775216    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f932b9d5-e294-411a-b6b5-57f1ed77c0dc-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f932b9d5-e294-411a-b6b5-57f1ed77c0dc" (UID: "f932b9d5-e294-411a-b6b5-57f1ed77c0dc"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:49:03.874719 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:03.874683    2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f932b9d5-e294-411a-b6b5-57f1ed77c0dc-webhook-cert\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:49:03.874719 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:03.874717    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f932b9d5-e294-411a-b6b5-57f1ed77c0dc-metrics-certs\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:49:03.874928 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:03.874730    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8gzs\" (UniqueName: \"kubernetes.io/projected/f932b9d5-e294-411a-b6b5-57f1ed77c0dc-kube-api-access-v8gzs\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:49:04.206118 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:04.206057    2560 generic.go:356] "Generic (PLEG): container finished" podID="f932b9d5-e294-411a-b6b5-57f1ed77c0dc" containerID="cd96c9c0d8c8d932147ad247884d6746eacb9e27677e8a21c978b036a60b3e76" exitCode=0
Sep 08 15:49:04.206313 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:04.206143    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b994b57d8-bkptb"
Sep 08 15:49:04.206313 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:04.206149    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b994b57d8-bkptb" event={"ID":"f932b9d5-e294-411a-b6b5-57f1ed77c0dc","Type":"ContainerDied","Data":"cd96c9c0d8c8d932147ad247884d6746eacb9e27677e8a21c978b036a60b3e76"}
Sep 08 15:49:04.206313 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:04.206242    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b994b57d8-bkptb" event={"ID":"f932b9d5-e294-411a-b6b5-57f1ed77c0dc","Type":"ContainerDied","Data":"3c622cc4203e41aed83c3b86c430970aeddb72691892aae639103fcac595c601"}
Sep 08 15:49:04.206313 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:04.206272    2560 scope.go:117] "RemoveContainer" containerID="cd96c9c0d8c8d932147ad247884d6746eacb9e27677e8a21c978b036a60b3e76"
Sep 08 15:49:04.214796 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:04.214775    2560 scope.go:117] "RemoveContainer" containerID="cd96c9c0d8c8d932147ad247884d6746eacb9e27677e8a21c978b036a60b3e76"
Sep 08 15:49:04.215172 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:49:04.215047    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd96c9c0d8c8d932147ad247884d6746eacb9e27677e8a21c978b036a60b3e76\": container with ID starting with cd96c9c0d8c8d932147ad247884d6746eacb9e27677e8a21c978b036a60b3e76 not found: ID does not exist" containerID="cd96c9c0d8c8d932147ad247884d6746eacb9e27677e8a21c978b036a60b3e76"
Sep 08 15:49:04.215264 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:04.215182    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd96c9c0d8c8d932147ad247884d6746eacb9e27677e8a21c978b036a60b3e76"} err="failed to get container status \"cd96c9c0d8c8d932147ad247884d6746eacb9e27677e8a21c978b036a60b3e76\": rpc error: code = NotFound desc = could not find container \"cd96c9c0d8c8d932147ad247884d6746eacb9e27677e8a21c978b036a60b3e76\": container with ID starting with cd96c9c0d8c8d932147ad247884d6746eacb9e27677e8a21c978b036a60b3e76 not found: ID does not exist"
Sep 08 15:49:04.220908 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:04.220880    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b994b57d8-bkptb"]
Sep 08 15:49:04.223961 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:04.223942    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7b994b57d8-bkptb"]
Sep 08 15:49:04.885783 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:04.885735    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:49:04.991622 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:04.991589    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7b994b57d8-fqs8r"]
Sep 08 15:49:04.991933 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:04.991917    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f932b9d5-e294-411a-b6b5-57f1ed77c0dc" containerName="manager"
Sep 08 15:49:04.991933 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:04.991933    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f932b9d5-e294-411a-b6b5-57f1ed77c0dc" containerName="manager"
Sep 08 15:49:04.992025 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:04.992015    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f932b9d5-e294-411a-b6b5-57f1ed77c0dc" containerName="manager"
Sep 08 15:49:04.996333 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:04.996315    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b994b57d8-fqs8r"
Sep 08 15:49:04.998460 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:04.998434    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 08 15:49:04.998579 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:04.998456    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 08 15:49:04.998579 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:04.998437    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wpgkm\""
Sep 08 15:49:04.998579 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:04.998437    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 08 15:49:05.005327 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:05.005303    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b994b57d8-fqs8r"]
Sep 08 15:49:05.085678 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:05.085639    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/caf1d3fd-2130-4426-b4b1-0d566242f364-webhook-cert\") pod \"maas-controller-7b994b57d8-fqs8r\" (UID: \"caf1d3fd-2130-4426-b4b1-0d566242f364\") " pod="opendatahub/maas-controller-7b994b57d8-fqs8r"
Sep 08 15:49:05.085854 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:05.085700    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/caf1d3fd-2130-4426-b4b1-0d566242f364-metrics-certs\") pod \"maas-controller-7b994b57d8-fqs8r\" (UID: \"caf1d3fd-2130-4426-b4b1-0d566242f364\") " pod="opendatahub/maas-controller-7b994b57d8-fqs8r"
Sep 08 15:49:05.085854 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:05.085775    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jmcf\" (UniqueName: \"kubernetes.io/projected/caf1d3fd-2130-4426-b4b1-0d566242f364-kube-api-access-9jmcf\") pod \"maas-controller-7b994b57d8-fqs8r\" (UID: \"caf1d3fd-2130-4426-b4b1-0d566242f364\") " pod="opendatahub/maas-controller-7b994b57d8-fqs8r"
Sep 08 15:49:05.187129 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:05.187094    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/caf1d3fd-2130-4426-b4b1-0d566242f364-metrics-certs\") pod \"maas-controller-7b994b57d8-fqs8r\" (UID: \"caf1d3fd-2130-4426-b4b1-0d566242f364\") " pod="opendatahub/maas-controller-7b994b57d8-fqs8r"
Sep 08 15:49:05.187326 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:05.187159    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jmcf\" (UniqueName: \"kubernetes.io/projected/caf1d3fd-2130-4426-b4b1-0d566242f364-kube-api-access-9jmcf\") pod \"maas-controller-7b994b57d8-fqs8r\" (UID: \"caf1d3fd-2130-4426-b4b1-0d566242f364\") " pod="opendatahub/maas-controller-7b994b57d8-fqs8r"
Sep 08 15:49:05.187326 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:05.187213    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/caf1d3fd-2130-4426-b4b1-0d566242f364-webhook-cert\") pod \"maas-controller-7b994b57d8-fqs8r\" (UID: \"caf1d3fd-2130-4426-b4b1-0d566242f364\") " pod="opendatahub/maas-controller-7b994b57d8-fqs8r"
Sep 08 15:49:05.189544 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:05.189514    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/caf1d3fd-2130-4426-b4b1-0d566242f364-webhook-cert\") pod \"maas-controller-7b994b57d8-fqs8r\" (UID: \"caf1d3fd-2130-4426-b4b1-0d566242f364\") " pod="opendatahub/maas-controller-7b994b57d8-fqs8r"
Sep 08 15:49:05.189689 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:05.189558    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/caf1d3fd-2130-4426-b4b1-0d566242f364-metrics-certs\") pod \"maas-controller-7b994b57d8-fqs8r\" (UID: \"caf1d3fd-2130-4426-b4b1-0d566242f364\") " pod="opendatahub/maas-controller-7b994b57d8-fqs8r"
Sep 08 15:49:05.195213 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:05.195188    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jmcf\" (UniqueName: \"kubernetes.io/projected/caf1d3fd-2130-4426-b4b1-0d566242f364-kube-api-access-9jmcf\") pod \"maas-controller-7b994b57d8-fqs8r\" (UID: \"caf1d3fd-2130-4426-b4b1-0d566242f364\") " pod="opendatahub/maas-controller-7b994b57d8-fqs8r"
Sep 08 15:49:05.308051 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:05.308019    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b994b57d8-fqs8r"
Sep 08 15:49:05.475844 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:05.475757    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:49:05.580639 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:05.580547    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b994b57d8-fqs8r"]
Sep 08 15:49:05.582823 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:49:05.582796    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaf1d3fd_2130_4426_b4b1_0d566242f364.slice/crio-fd3ac20745b4f3f0615ac663f50b10474d0bab159fafa19caf7123e6b34f9960 WatchSource:0}: Error finding container fd3ac20745b4f3f0615ac663f50b10474d0bab159fafa19caf7123e6b34f9960: Status 404 returned error can't find the container with id fd3ac20745b4f3f0615ac663f50b10474d0bab159fafa19caf7123e6b34f9960
Sep 08 15:49:05.799893 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:05.799814    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f932b9d5-e294-411a-b6b5-57f1ed77c0dc" path="/var/lib/kubelet/pods/f932b9d5-e294-411a-b6b5-57f1ed77c0dc/volumes"
Sep 08 15:49:06.218523 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:06.218492    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b994b57d8-fqs8r" event={"ID":"caf1d3fd-2130-4426-b4b1-0d566242f364","Type":"ContainerStarted","Data":"e1252461228f22c70261c61fe33bd689322f2d8fcb4c2f40488883a9d12351a7"}
Sep 08 15:49:06.218942 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:06.218538    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b994b57d8-fqs8r" event={"ID":"caf1d3fd-2130-4426-b4b1-0d566242f364","Type":"ContainerStarted","Data":"fd3ac20745b4f3f0615ac663f50b10474d0bab159fafa19caf7123e6b34f9960"}
Sep 08 15:49:06.218942 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:06.218684    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7b994b57d8-fqs8r"
Sep 08 15:49:06.235776 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:06.235712    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7b994b57d8-fqs8r" podStartSLOduration=1.92846556 podStartE2EDuration="2.23569206s" totalImagesPullingTime="307.2265ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:49:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:49:06.234940856 +0000 UTC m=+1898.946883562" watchObservedRunningTime="2026-09-08 15:49:06.23569206 +0000 UTC m=+1898.947634553"
Sep 08 15:49:17.226444 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:17.226406    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7b994b57d8-fqs8r"
Sep 08 15:49:20.055713 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:49:20.055673    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:50:20.661739 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:50:20.661657    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6l8mj"]
Sep 08 15:50:20.662168 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:50:20.661900    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6l8mj" podUID="8693ed62-2396-4039-b639-8e8063f8dc1f" containerName="manager" containerID="cri-o://89418823ee960d460e0f0c4787e19d2690f0bdd967ffa5f5e97140a0a2dd1551" gracePeriod=10
Sep 08 15:50:21.003121 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:50:21.003093    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6l8mj"
Sep 08 15:50:21.147350 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:50:21.147315    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hrsp\" (UniqueName: \"kubernetes.io/projected/8693ed62-2396-4039-b639-8e8063f8dc1f-kube-api-access-6hrsp\") pod \"8693ed62-2396-4039-b639-8e8063f8dc1f\" (UID: \"8693ed62-2396-4039-b639-8e8063f8dc1f\") "
Sep 08 15:50:21.147537 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:50:21.147400    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8693ed62-2396-4039-b639-8e8063f8dc1f-extensions-socket-volume\") pod \"8693ed62-2396-4039-b639-8e8063f8dc1f\" (UID: \"8693ed62-2396-4039-b639-8e8063f8dc1f\") "
Sep 08 15:50:21.147793 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:50:21.147766    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8693ed62-2396-4039-b639-8e8063f8dc1f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8693ed62-2396-4039-b639-8e8063f8dc1f" (UID: "8693ed62-2396-4039-b639-8e8063f8dc1f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:50:21.149443 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:50:21.149414    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8693ed62-2396-4039-b639-8e8063f8dc1f-kube-api-access-6hrsp" (OuterVolumeSpecName: "kube-api-access-6hrsp") pod "8693ed62-2396-4039-b639-8e8063f8dc1f" (UID: "8693ed62-2396-4039-b639-8e8063f8dc1f"). InnerVolumeSpecName "kube-api-access-6hrsp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:50:21.248292 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:50:21.248258    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hrsp\" (UniqueName: \"kubernetes.io/projected/8693ed62-2396-4039-b639-8e8063f8dc1f-kube-api-access-6hrsp\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:50:21.248292 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:50:21.248288    2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8693ed62-2396-4039-b639-8e8063f8dc1f-extensions-socket-volume\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:50:21.578791 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:50:21.578694    2560 generic.go:356] "Generic (PLEG): container finished" podID="8693ed62-2396-4039-b639-8e8063f8dc1f" containerID="89418823ee960d460e0f0c4787e19d2690f0bdd967ffa5f5e97140a0a2dd1551" exitCode=0
Sep 08 15:50:21.578791 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:50:21.578738    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6l8mj"
Sep 08 15:50:21.579035 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:50:21.578781    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6l8mj" event={"ID":"8693ed62-2396-4039-b639-8e8063f8dc1f","Type":"ContainerDied","Data":"89418823ee960d460e0f0c4787e19d2690f0bdd967ffa5f5e97140a0a2dd1551"}
Sep 08 15:50:21.579035 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:50:21.578839    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6l8mj" event={"ID":"8693ed62-2396-4039-b639-8e8063f8dc1f","Type":"ContainerDied","Data":"45b1ea5c60f0492523b02c2c1a52e2f5048c20343cc2825ba605a0040a31abfb"}
Sep 08 15:50:21.579035 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:50:21.578869    2560 scope.go:117] "RemoveContainer" containerID="89418823ee960d460e0f0c4787e19d2690f0bdd967ffa5f5e97140a0a2dd1551"
Sep 08 15:50:21.588786 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:50:21.588765    2560 scope.go:117] "RemoveContainer" containerID="89418823ee960d460e0f0c4787e19d2690f0bdd967ffa5f5e97140a0a2dd1551"
Sep 08 15:50:21.589059 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:50:21.589038    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89418823ee960d460e0f0c4787e19d2690f0bdd967ffa5f5e97140a0a2dd1551\": container with ID starting with 89418823ee960d460e0f0c4787e19d2690f0bdd967ffa5f5e97140a0a2dd1551 not found: ID does not exist" containerID="89418823ee960d460e0f0c4787e19d2690f0bdd967ffa5f5e97140a0a2dd1551"
Sep 08 15:50:21.589149 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:50:21.589087    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89418823ee960d460e0f0c4787e19d2690f0bdd967ffa5f5e97140a0a2dd1551"} err="failed to get container status \"89418823ee960d460e0f0c4787e19d2690f0bdd967ffa5f5e97140a0a2dd1551\": rpc error: code = NotFound desc = could not find container \"89418823ee960d460e0f0c4787e19d2690f0bdd967ffa5f5e97140a0a2dd1551\": container with ID starting with 89418823ee960d460e0f0c4787e19d2690f0bdd967ffa5f5e97140a0a2dd1551 not found: ID does not exist"
Sep 08 15:50:21.611213 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:50:21.611185    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6l8mj"]
Sep 08 15:50:21.618668 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:50:21.618646    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-6l8mj"]
Sep 08 15:50:21.800220 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:50:21.800185    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8693ed62-2396-4039-b639-8e8063f8dc1f" path="/var/lib/kubelet/pods/8693ed62-2396-4039-b639-8e8063f8dc1f/volumes"
Sep 08 15:50:29.611764 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:50:29.611728    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:51:21.392628 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:51:21.392591    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4768n"]
Sep 08 15:51:21.393051 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:51:21.392980    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8693ed62-2396-4039-b639-8e8063f8dc1f" containerName="manager"
Sep 08 15:51:21.393051 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:51:21.392992    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8693ed62-2396-4039-b639-8e8063f8dc1f" containerName="manager"
Sep 08 15:51:21.393051 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:51:21.393048    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8693ed62-2396-4039-b639-8e8063f8dc1f" containerName="manager"
Sep 08 15:51:21.395978 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:51:21.395960    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4768n"
Sep 08 15:51:21.398319 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:51:21.398297    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 08 15:51:21.399260 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:51:21.399033    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 08 15:51:21.399260 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:51:21.399129    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-z9pcn\""
Sep 08 15:51:21.412033 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:51:21.412007    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4768n"]
Sep 08 15:51:21.474768 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:51:21.474733    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh7ct\" (UniqueName: \"kubernetes.io/projected/f58af686-b28e-4106-a2ae-3a78b0d7cc50-kube-api-access-qh7ct\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-4768n\" (UID: \"f58af686-b28e-4106-a2ae-3a78b0d7cc50\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4768n"
Sep 08 15:51:21.474768 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:51:21.474769    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f58af686-b28e-4106-a2ae-3a78b0d7cc50-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-4768n\" (UID: \"f58af686-b28e-4106-a2ae-3a78b0d7cc50\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4768n"
Sep 08 15:51:21.575407 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:51:21.575371    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh7ct\" (UniqueName: \"kubernetes.io/projected/f58af686-b28e-4106-a2ae-3a78b0d7cc50-kube-api-access-qh7ct\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-4768n\" (UID: \"f58af686-b28e-4106-a2ae-3a78b0d7cc50\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4768n"
Sep 08 15:51:21.575407 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:51:21.575407    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f58af686-b28e-4106-a2ae-3a78b0d7cc50-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-4768n\" (UID: \"f58af686-b28e-4106-a2ae-3a78b0d7cc50\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4768n"
Sep 08 15:51:21.575757 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:51:21.575740    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f58af686-b28e-4106-a2ae-3a78b0d7cc50-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-4768n\" (UID: \"f58af686-b28e-4106-a2ae-3a78b0d7cc50\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4768n"
Sep 08 15:51:21.584604 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:51:21.584582    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh7ct\" (UniqueName: \"kubernetes.io/projected/f58af686-b28e-4106-a2ae-3a78b0d7cc50-kube-api-access-qh7ct\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-4768n\" (UID: \"f58af686-b28e-4106-a2ae-3a78b0d7cc50\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4768n"
Sep 08 15:51:21.706668 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:51:21.706632    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4768n"
Sep 08 15:51:22.011775 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:51:22.011750    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4768n"]
Sep 08 15:51:22.014182 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:51:22.014151    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf58af686_b28e_4106_a2ae_3a78b0d7cc50.slice/crio-3b5e480d6b9b806b12db70e93b7e873e774de8f8ab0eeb59d76ed29416334708 WatchSource:0}: Error finding container 3b5e480d6b9b806b12db70e93b7e873e774de8f8ab0eeb59d76ed29416334708: Status 404 returned error can't find the container with id 3b5e480d6b9b806b12db70e93b7e873e774de8f8ab0eeb59d76ed29416334708
Sep 08 15:51:22.856652 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:51:22.856618    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4768n" event={"ID":"f58af686-b28e-4106-a2ae-3a78b0d7cc50","Type":"ContainerStarted","Data":"5d65ad41a256c3a9bbc03088f41535a8d95d8ea5bd9fcf03acf4fcfe257b127a"}
Sep 08 15:51:22.857020 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:51:22.856671    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4768n" event={"ID":"f58af686-b28e-4106-a2ae-3a78b0d7cc50","Type":"ContainerStarted","Data":"3b5e480d6b9b806b12db70e93b7e873e774de8f8ab0eeb59d76ed29416334708"}
Sep 08 15:51:22.857020 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:51:22.856747    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4768n"
Sep 08 15:51:22.882116 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:51:22.882052    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4768n" podStartSLOduration=1.882036501 podStartE2EDuration="1.882036501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:51:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:51:22.88051903 +0000 UTC m=+2035.592461532" watchObservedRunningTime="2026-09-08 15:51:22.882036501 +0000 UTC m=+2035.593978992"
Sep 08 15:51:33.863093 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:51:33.863048    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-4768n"
Sep 08 15:51:58.234165 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:51:58.234088    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:52:25.436939 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:52:25.436904    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:52:27.846934 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:52:27.846902    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vc5n9_17e30810-dfd0-4e6f-9204-731d1a6479a3/console-operator/2.log"
Sep 08 15:52:27.853595 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:52:27.853569    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vc5n9_17e30810-dfd0-4e6f-9204-731d1a6479a3/console-operator/2.log"
Sep 08 15:52:56.289479 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:52:56.289440    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:52:58.082908 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:52:58.082872    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:53:21.649610 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:21.649518    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-c5bcdfbdd-pwcjj"]
Sep 08 15:53:21.653421 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:21.653399    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-c5bcdfbdd-pwcjj"
Sep 08 15:53:21.663349 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:21.663327    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-c5bcdfbdd-pwcjj"]
Sep 08 15:53:21.713533 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:21.713501    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b170e5d5-6bf2-497d-ad04-002bf7d6d66f-maas-api-tls\") pod \"maas-api-c5bcdfbdd-pwcjj\" (UID: \"b170e5d5-6bf2-497d-ad04-002bf7d6d66f\") " pod="odh-ai-gateway-infra/maas-api-c5bcdfbdd-pwcjj"
Sep 08 15:53:21.713708 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:21.713559    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b170e5d5-6bf2-497d-ad04-002bf7d6d66f-openshift-service-ca\") pod \"maas-api-c5bcdfbdd-pwcjj\" (UID: \"b170e5d5-6bf2-497d-ad04-002bf7d6d66f\") " pod="odh-ai-gateway-infra/maas-api-c5bcdfbdd-pwcjj"
Sep 08 15:53:21.713708 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:21.713580    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48r4q\" (UniqueName: \"kubernetes.io/projected/b170e5d5-6bf2-497d-ad04-002bf7d6d66f-kube-api-access-48r4q\") pod \"maas-api-c5bcdfbdd-pwcjj\" (UID: \"b170e5d5-6bf2-497d-ad04-002bf7d6d66f\") " pod="odh-ai-gateway-infra/maas-api-c5bcdfbdd-pwcjj"
Sep 08 15:53:21.713708 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:21.713623    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b170e5d5-6bf2-497d-ad04-002bf7d6d66f-metrics-certs\") pod \"maas-api-c5bcdfbdd-pwcjj\" (UID: \"b170e5d5-6bf2-497d-ad04-002bf7d6d66f\") " pod="odh-ai-gateway-infra/maas-api-c5bcdfbdd-pwcjj"
Sep 08 15:53:21.814792 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:21.814751    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b170e5d5-6bf2-497d-ad04-002bf7d6d66f-maas-api-tls\") pod \"maas-api-c5bcdfbdd-pwcjj\" (UID: \"b170e5d5-6bf2-497d-ad04-002bf7d6d66f\") " pod="odh-ai-gateway-infra/maas-api-c5bcdfbdd-pwcjj"
Sep 08 15:53:21.815006 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:21.814829    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b170e5d5-6bf2-497d-ad04-002bf7d6d66f-openshift-service-ca\") pod \"maas-api-c5bcdfbdd-pwcjj\" (UID: \"b170e5d5-6bf2-497d-ad04-002bf7d6d66f\") " pod="odh-ai-gateway-infra/maas-api-c5bcdfbdd-pwcjj"
Sep 08 15:53:21.815006 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:21.814866    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48r4q\" (UniqueName: \"kubernetes.io/projected/b170e5d5-6bf2-497d-ad04-002bf7d6d66f-kube-api-access-48r4q\") pod \"maas-api-c5bcdfbdd-pwcjj\" (UID: \"b170e5d5-6bf2-497d-ad04-002bf7d6d66f\") " pod="odh-ai-gateway-infra/maas-api-c5bcdfbdd-pwcjj"
Sep 08 15:53:21.815006 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:21.814905    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b170e5d5-6bf2-497d-ad04-002bf7d6d66f-metrics-certs\") pod \"maas-api-c5bcdfbdd-pwcjj\" (UID: \"b170e5d5-6bf2-497d-ad04-002bf7d6d66f\") " pod="odh-ai-gateway-infra/maas-api-c5bcdfbdd-pwcjj"
Sep 08 15:53:21.815802 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:21.815605    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b170e5d5-6bf2-497d-ad04-002bf7d6d66f-openshift-service-ca\") pod \"maas-api-c5bcdfbdd-pwcjj\" (UID: \"b170e5d5-6bf2-497d-ad04-002bf7d6d66f\") " pod="odh-ai-gateway-infra/maas-api-c5bcdfbdd-pwcjj"
Sep 08 15:53:21.817496 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:21.817474    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b170e5d5-6bf2-497d-ad04-002bf7d6d66f-maas-api-tls\") pod \"maas-api-c5bcdfbdd-pwcjj\" (UID: \"b170e5d5-6bf2-497d-ad04-002bf7d6d66f\") " pod="odh-ai-gateway-infra/maas-api-c5bcdfbdd-pwcjj"
Sep 08 15:53:21.817608 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:21.817531    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b170e5d5-6bf2-497d-ad04-002bf7d6d66f-metrics-certs\") pod \"maas-api-c5bcdfbdd-pwcjj\" (UID: \"b170e5d5-6bf2-497d-ad04-002bf7d6d66f\") " pod="odh-ai-gateway-infra/maas-api-c5bcdfbdd-pwcjj"
Sep 08 15:53:21.822631 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:21.822606    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48r4q\" (UniqueName: \"kubernetes.io/projected/b170e5d5-6bf2-497d-ad04-002bf7d6d66f-kube-api-access-48r4q\") pod \"maas-api-c5bcdfbdd-pwcjj\" (UID: \"b170e5d5-6bf2-497d-ad04-002bf7d6d66f\") " pod="odh-ai-gateway-infra/maas-api-c5bcdfbdd-pwcjj"
Sep 08 15:53:21.964680 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:21.964644    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-c5bcdfbdd-pwcjj"
Sep 08 15:53:22.237614 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:22.237587    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-c5bcdfbdd-pwcjj"]
Sep 08 15:53:22.242403 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:53:22.242371    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb170e5d5_6bf2_497d_ad04_002bf7d6d66f.slice/crio-7691f9ac24c911831d49a4d72c1b1ef2e87c7b2162c01391fbfecc71ba4df9f4 WatchSource:0}: Error finding container 7691f9ac24c911831d49a4d72c1b1ef2e87c7b2162c01391fbfecc71ba4df9f4: Status 404 returned error can't find the container with id 7691f9ac24c911831d49a4d72c1b1ef2e87c7b2162c01391fbfecc71ba4df9f4
Sep 08 15:53:22.419675 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:22.419634    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-c5bcdfbdd-pwcjj" event={"ID":"b170e5d5-6bf2-497d-ad04-002bf7d6d66f","Type":"ContainerStarted","Data":"7691f9ac24c911831d49a4d72c1b1ef2e87c7b2162c01391fbfecc71ba4df9f4"}
Sep 08 15:53:23.426678 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:23.426640    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-c5bcdfbdd-pwcjj" event={"ID":"b170e5d5-6bf2-497d-ad04-002bf7d6d66f","Type":"ContainerStarted","Data":"bf0d245680c9e5f90b1561c697869d64b5af4eacea4545d6640c6801a9b5aa41"}
Sep 08 15:53:23.427055 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:23.426757    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-c5bcdfbdd-pwcjj"
Sep 08 15:53:23.445035 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:23.444985    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-c5bcdfbdd-pwcjj" podStartSLOduration=1.923556773 podStartE2EDuration="2.444970298s" totalImagesPullingTime="521.413525ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:53:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:53:23.443866235 +0000 UTC m=+2156.155808718" watchObservedRunningTime="2026-09-08 15:53:23.444970298 +0000 UTC m=+2156.156912788"
Sep 08 15:53:29.436165 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:29.436135    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-c5bcdfbdd-pwcjj"
Sep 08 15:53:36.846301 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:36.846264    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn"]
Sep 08 15:53:36.846799 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:36.846497    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn" podUID="a8fce02c-f8ae-43eb-a5d6-14c107eab70e" containerName="maas-api" containerID="cri-o://e86b3b530776813665f9cb133efbe6d7ca34e0dbeeb6e1022fb078e62d4c7a0e" gracePeriod=30
Sep 08 15:53:38.100405 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:38.100379    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn"
Sep 08 15:53:38.168574 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:38.168538    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8fce02c-f8ae-43eb-a5d6-14c107eab70e-metrics-certs\") pod \"a8fce02c-f8ae-43eb-a5d6-14c107eab70e\" (UID: \"a8fce02c-f8ae-43eb-a5d6-14c107eab70e\") "
Sep 08 15:53:38.168764 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:38.168592    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8fce02c-f8ae-43eb-a5d6-14c107eab70e-openshift-service-ca\") pod \"a8fce02c-f8ae-43eb-a5d6-14c107eab70e\" (UID: \"a8fce02c-f8ae-43eb-a5d6-14c107eab70e\") "
Sep 08 15:53:38.168764 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:38.168631    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8fce02c-f8ae-43eb-a5d6-14c107eab70e-maas-api-tls\") pod \"a8fce02c-f8ae-43eb-a5d6-14c107eab70e\" (UID: \"a8fce02c-f8ae-43eb-a5d6-14c107eab70e\") "
Sep 08 15:53:38.168764 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:38.168664    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qcmgd\" (UniqueName: \"kubernetes.io/projected/a8fce02c-f8ae-43eb-a5d6-14c107eab70e-kube-api-access-qcmgd\") pod \"a8fce02c-f8ae-43eb-a5d6-14c107eab70e\" (UID: \"a8fce02c-f8ae-43eb-a5d6-14c107eab70e\") "
Sep 08 15:53:38.169064 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:38.169024    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8fce02c-f8ae-43eb-a5d6-14c107eab70e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a8fce02c-f8ae-43eb-a5d6-14c107eab70e" (UID: "a8fce02c-f8ae-43eb-a5d6-14c107eab70e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:53:38.170958 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:38.170926    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8fce02c-f8ae-43eb-a5d6-14c107eab70e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a8fce02c-f8ae-43eb-a5d6-14c107eab70e" (UID: "a8fce02c-f8ae-43eb-a5d6-14c107eab70e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:53:38.171268 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:38.171244    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8fce02c-f8ae-43eb-a5d6-14c107eab70e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a8fce02c-f8ae-43eb-a5d6-14c107eab70e" (UID: "a8fce02c-f8ae-43eb-a5d6-14c107eab70e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:53:38.171365 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:38.171251    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8fce02c-f8ae-43eb-a5d6-14c107eab70e-kube-api-access-qcmgd" (OuterVolumeSpecName: "kube-api-access-qcmgd") pod "a8fce02c-f8ae-43eb-a5d6-14c107eab70e" (UID: "a8fce02c-f8ae-43eb-a5d6-14c107eab70e"). InnerVolumeSpecName "kube-api-access-qcmgd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:53:38.269940 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:38.269859    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8fce02c-f8ae-43eb-a5d6-14c107eab70e-metrics-certs\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:53:38.269940 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:38.269887    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8fce02c-f8ae-43eb-a5d6-14c107eab70e-openshift-service-ca\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:53:38.269940 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:38.269899    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8fce02c-f8ae-43eb-a5d6-14c107eab70e-maas-api-tls\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:53:38.269940 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:38.269912    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcmgd\" (UniqueName: \"kubernetes.io/projected/a8fce02c-f8ae-43eb-a5d6-14c107eab70e-kube-api-access-qcmgd\") on node \"ip-10-0-137-43.ec2.internal\" DevicePath \"\""
Sep 08 15:53:38.500700 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:38.500657    2560 generic.go:356] "Generic (PLEG): container finished" podID="a8fce02c-f8ae-43eb-a5d6-14c107eab70e" containerID="e86b3b530776813665f9cb133efbe6d7ca34e0dbeeb6e1022fb078e62d4c7a0e" exitCode=0
Sep 08 15:53:38.500863 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:38.500723    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn"
Sep 08 15:53:38.500863 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:38.500717    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn" event={"ID":"a8fce02c-f8ae-43eb-a5d6-14c107eab70e","Type":"ContainerDied","Data":"e86b3b530776813665f9cb133efbe6d7ca34e0dbeeb6e1022fb078e62d4c7a0e"}
Sep 08 15:53:38.500952 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:38.500878    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn" event={"ID":"a8fce02c-f8ae-43eb-a5d6-14c107eab70e","Type":"ContainerDied","Data":"24521d833d5911840bbb4c61fc44160b9ce0557ae362877bb19989cf18fb1ed9"}
Sep 08 15:53:38.500952 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:38.500901    2560 scope.go:117] "RemoveContainer" containerID="e86b3b530776813665f9cb133efbe6d7ca34e0dbeeb6e1022fb078e62d4c7a0e"
Sep 08 15:53:38.511099 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:38.511010    2560 scope.go:117] "RemoveContainer" containerID="e86b3b530776813665f9cb133efbe6d7ca34e0dbeeb6e1022fb078e62d4c7a0e"
Sep 08 15:53:38.511389 ip-10-0-137-43 kubenswrapper[2560]: E0908 15:53:38.511366    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e86b3b530776813665f9cb133efbe6d7ca34e0dbeeb6e1022fb078e62d4c7a0e\": container with ID starting with e86b3b530776813665f9cb133efbe6d7ca34e0dbeeb6e1022fb078e62d4c7a0e not found: ID does not exist" containerID="e86b3b530776813665f9cb133efbe6d7ca34e0dbeeb6e1022fb078e62d4c7a0e"
Sep 08 15:53:38.511469 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:38.511400    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e86b3b530776813665f9cb133efbe6d7ca34e0dbeeb6e1022fb078e62d4c7a0e"} err="failed to get container status \"e86b3b530776813665f9cb133efbe6d7ca34e0dbeeb6e1022fb078e62d4c7a0e\": rpc error: code = NotFound desc = could not find container \"e86b3b530776813665f9cb133efbe6d7ca34e0dbeeb6e1022fb078e62d4c7a0e\": container with ID starting with e86b3b530776813665f9cb133efbe6d7ca34e0dbeeb6e1022fb078e62d4c7a0e not found: ID does not exist"
Sep 08 15:53:38.524406 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:38.524337    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn"]
Sep 08 15:53:38.528053 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:38.528028    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f8554757b-fgpwn"]
Sep 08 15:53:39.800636 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:53:39.800604    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8fce02c-f8ae-43eb-a5d6-14c107eab70e" path="/var/lib/kubelet/pods/a8fce02c-f8ae-43eb-a5d6-14c107eab70e/volumes"
Sep 08 15:54:01.130210 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:01.130176    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-qlzp2_2c3b388f-7424-47df-8af6-a1c1dc717334/manager/0.log"
Sep 08 15:54:01.248174 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:01.248140    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7b994b57d8-fqs8r_caf1d3fd-2130-4426-b4b1-0d566242f364/manager/0.log"
Sep 08 15:54:01.476941 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:01.476914    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-kn4vl_5f1e5b0c-dddb-4859-8041-57996d5ec812/manager/0.log"
Sep 08 15:54:01.599038 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:01.599008    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-847d47f6c8-c7hnl_a72080d0-7eab-402c-a477-9d487435aab9/copy-manifests/0.log"
Sep 08 15:54:01.606342 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:01.606319    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-847d47f6c8-c7hnl_a72080d0-7eab-402c-a477-9d487435aab9/manager/0.log"
Sep 08 15:54:02.325243 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:02.325214    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-c5bcdfbdd-pwcjj_b170e5d5-6bf2-497d-ad04-002bf7d6d66f/maas-api/0.log"
Sep 08 15:54:02.671011 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:02.670919    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-97xj8_9d29db75-3ab3-496e-8d91-ee03ec995deb/postgres/0.log"
Sep 08 15:54:03.935364 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:03.935336    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5b65f94d4c-4768n_f58af686-b28e-4106-a2ae-3a78b0d7cc50/manager/0.log"
Sep 08 15:54:04.684897 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:04.684859    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-kt4pd_a1a62f91-49cb-4c70-a3e0-92989a5ceb43/discovery/0.log"
Sep 08 15:54:04.919823 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:04.919799    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d44f46d57-r2s76_5bf49137-ceba-4cbc-8f05-25b821fcab99/kube-auth-proxy/0.log"
Sep 08 15:54:05.165412 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:05.165326    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-tscfp_231e053b-1392-4597-bc81-2437855e9687/payload-pre-processing/0.log"
Sep 08 15:54:05.289647 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:05.289611    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-zdsqr_77f92f27-262b-498d-a1fa-94ece7554033/payload-processing/0.log"
Sep 08 15:54:05.400429 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:05.400393    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-88497859b-8nkdj_5551e120-e4db-4d93-8d2a-9e3626db076e/router/0.log"
Sep 08 15:54:05.971957 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:05.971922    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-gq4l4_ea31698f-b405-4f28-9378-2affc68d27be/main/0.log"
Sep 08 15:54:06.085373 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:06.085349    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-ltcb8_e9ff10e1-57d3-4eba-bac5-2802834deacc/main/0.log"
Sep 08 15:54:11.872290 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:11.872252    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:54:15.047241 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:15.047203    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4wzj5_4b05229d-340e-424a-8533-87c77e6b97d5/global-pull-secret-syncer/0.log"
Sep 08 15:54:15.188539 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:15.188472    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log"
Sep 08 15:54:15.319223 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:15.319115    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-43.ec2.internal_f7c1882d19ab40f24851c6d6b5b879ef/haproxy/0.log"
Sep 08 15:54:17.945660 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:17.945607    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5kkjk_08200567-2c28-4a77-83fb-c6f82cd9f42e/konnectivity-agent/0.log" limit=1024
Sep 08 15:54:20.804573 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:20.804533    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5b65f94d4c-4768n_f58af686-b28e-4106-a2ae-3a78b0d7cc50/manager/0.log"
Sep 08 15:54:22.729849 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:22.729818    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-8dlwl_c64984d1-f8b2-414b-8f55-0a4d68e867cf/cluster-monitoring-operator/0.log"
Sep 08 15:54:23.058493 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:23.058462    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gm4s9_e1f7427c-6a62-4e51-bac8-5241611604d2/node-exporter/0.log"
Sep 08 15:54:23.084785 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:23.084756    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gm4s9_e1f7427c-6a62-4e51-bac8-5241611604d2/kube-rbac-proxy/0.log"
Sep 08 15:54:23.107496 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:23.107468    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gm4s9_e1f7427c-6a62-4e51-bac8-5241611604d2/init-textfile/0.log"
Sep 08 15:54:25.605808 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:25.605777    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-czmnt/perf-node-gather-daemonset-76srx"]
Sep 08 15:54:25.606206 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:25.606153    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8fce02c-f8ae-43eb-a5d6-14c107eab70e" containerName="maas-api"
Sep 08 15:54:25.606206 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:25.606166    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8fce02c-f8ae-43eb-a5d6-14c107eab70e" containerName="maas-api"
Sep 08 15:54:25.606285 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:25.606232    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8fce02c-f8ae-43eb-a5d6-14c107eab70e" containerName="maas-api"
Sep 08 15:54:25.609229 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:25.609211    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-76srx"
Sep 08 15:54:25.611431 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:25.611410    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-czmnt\"/\"openshift-service-ca.crt\""
Sep 08 15:54:25.612122 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:25.612105    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-czmnt\"/\"kube-root-ca.crt\""
Sep 08 15:54:25.612222 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:25.612110    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-czmnt\"/\"default-dockercfg-fqwj2\""
Sep 08 15:54:25.615501 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:25.615481    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-czmnt/perf-node-gather-daemonset-76srx"]
Sep 08 15:54:25.679597 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:25.679566    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560-proc\") pod \"perf-node-gather-daemonset-76srx\" (UID: \"0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-76srx"
Sep 08 15:54:25.679751 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:25.679612    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8smfd\" (UniqueName: \"kubernetes.io/projected/0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560-kube-api-access-8smfd\") pod \"perf-node-gather-daemonset-76srx\" (UID: \"0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-76srx"
Sep 08 15:54:25.679751 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:25.679635    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560-podres\") pod \"perf-node-gather-daemonset-76srx\" (UID: \"0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-76srx"
Sep 08 15:54:25.679751 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:25.679659    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560-lib-modules\") pod \"perf-node-gather-daemonset-76srx\" (UID: \"0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-76srx"
Sep 08 15:54:25.679751 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:25.679681    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560-sys\") pod \"perf-node-gather-daemonset-76srx\" (UID: \"0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-76srx"
Sep 08 15:54:25.781142 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:25.781108    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560-proc\") pod \"perf-node-gather-daemonset-76srx\" (UID: \"0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-76srx"
Sep 08 15:54:25.781338 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:25.781159    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8smfd\" (UniqueName: \"kubernetes.io/projected/0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560-kube-api-access-8smfd\") pod \"perf-node-gather-daemonset-76srx\" (UID: \"0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-76srx"
Sep 08 15:54:25.781338 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:25.781181    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560-podres\") pod \"perf-node-gather-daemonset-76srx\" (UID: \"0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-76srx"
Sep 08 15:54:25.781338 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:25.781201    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560-lib-modules\") pod \"perf-node-gather-daemonset-76srx\" (UID: \"0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-76srx"
Sep 08 15:54:25.781338 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:25.781229    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560-sys\") pod \"perf-node-gather-daemonset-76srx\" (UID: \"0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-76srx"
Sep 08 15:54:25.781338 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:25.781258    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560-proc\") pod \"perf-node-gather-daemonset-76srx\" (UID: \"0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-76srx"
Sep 08 15:54:25.781338 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:25.781317    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560-podres\") pod \"perf-node-gather-daemonset-76srx\" (UID: \"0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-76srx"
Sep 08 15:54:25.781608 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:25.781383    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560-sys\") pod \"perf-node-gather-daemonset-76srx\" (UID: \"0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-76srx"
Sep 08 15:54:25.781608 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:25.781400    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560-lib-modules\") pod \"perf-node-gather-daemonset-76srx\" (UID: \"0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-76srx"
Sep 08 15:54:25.789142 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:25.789117    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8smfd\" (UniqueName: \"kubernetes.io/projected/0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560-kube-api-access-8smfd\") pod \"perf-node-gather-daemonset-76srx\" (UID: \"0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560\") " pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-76srx"
Sep 08 15:54:25.849311 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:25.849276    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vc5n9_17e30810-dfd0-4e6f-9204-731d1a6479a3/console-operator/2.log"
Sep 08 15:54:25.852860 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:25.852835    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vc5n9_17e30810-dfd0-4e6f-9204-731d1a6479a3/console-operator/3.log"
Sep 08 15:54:25.920575 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:25.920465    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-76srx"
Sep 08 15:54:26.194504 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:26.194478    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-czmnt/perf-node-gather-daemonset-76srx"]
Sep 08 15:54:26.197102 ip-10-0-137-43 kubenswrapper[2560]: W0908 15:54:26.197058    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0cfb2bb9_dc5d_4fca_9149_aaf7fcbaf560.slice/crio-e818a689b5f90b71c767d2b19e10d58fc4ffd71a6ceb0b777d935ffdc58eb7bf WatchSource:0}: Error finding container e818a689b5f90b71c767d2b19e10d58fc4ffd71a6ceb0b777d935ffdc58eb7bf: Status 404 returned error can't find the container with id e818a689b5f90b71c767d2b19e10d58fc4ffd71a6ceb0b777d935ffdc58eb7bf
Sep 08 15:54:26.731857 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:26.731813    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-76srx" event={"ID":"0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560","Type":"ContainerStarted","Data":"dbd47d0a41c3e14d5db2487364e1cd0a532484fcd839a50e164d543777fb38a3"}
Sep 08 15:54:26.732355 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:26.731951    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-76srx"
Sep 08 15:54:26.732355 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:26.731977    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-76srx" event={"ID":"0cfb2bb9-dc5d-4fca-9149-aaf7fcbaf560","Type":"ContainerStarted","Data":"e818a689b5f90b71c767d2b19e10d58fc4ffd71a6ceb0b777d935ffdc58eb7bf"}
Sep 08 15:54:26.751061 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:26.751011    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-76srx" podStartSLOduration=1.750996467 podStartE2EDuration="1.750996467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:54:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:54:26.749523455 +0000 UTC m=+2219.461465968" watchObservedRunningTime="2026-09-08 15:54:26.750996467 +0000 UTC m=+2219.462938958"
Sep 08 15:54:27.106700 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:27.106626    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-km8ph_7fc9b652-fc89-43ee-801e-440e67f2bad3/volume-data-source-validator/0.log"
Sep 08 15:54:28.199811 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:28.199781    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ct4vs_2df26fe4-655b-4248-a176-8028835f7379/dns/0.log"
Sep 08 15:54:28.218501 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:28.218470    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ct4vs_2df26fe4-655b-4248-a176-8028835f7379/kube-rbac-proxy/0.log"
Sep 08 15:54:28.301538 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:28.301514    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dvlfj_e26148ba-c65b-4e39-bd74-2f74c1ba38e6/dns-node-resolver/0.log"
Sep 08 15:54:28.877127 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:28.877100    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g7blt_762d10ee-64c5-40e1-896c-47e0cd55d613/node-ca/0.log"
Sep 08 15:54:30.191884 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:30.191849    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-kt4pd_a1a62f91-49cb-4c70-a3e0-92989a5ceb43/discovery/0.log"
Sep 08 15:54:30.248464 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:30.248437    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d44f46d57-r2s76_5bf49137-ceba-4cbc-8f05-25b821fcab99/kube-auth-proxy/0.log"
Sep 08 15:54:30.341855 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:30.341824    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-tscfp_231e053b-1392-4597-bc81-2437855e9687/payload-pre-processing/0.log"
Sep 08 15:54:30.378151 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:30.378119    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-zdsqr_77f92f27-262b-498d-a1fa-94ece7554033/payload-processing/0.log"
Sep 08 15:54:30.400021 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:30.399993    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-88497859b-8nkdj_5551e120-e4db-4d93-8d2a-9e3626db076e/router/0.log"
Sep 08 15:54:31.333017 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:31.332989    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z2r89_12943199-b315-4e6a-ae18-cae8dc762efe/serve-healthcheck-canary/0.log"
Sep 08 15:54:31.835056 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:31.835025    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-dkwzw_d4a10483-c29e-4a1c-9eb8-79d3197c1e35/insights-operator/0.log"
Sep 08 15:54:31.835380 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:31.835363    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-dkwzw_d4a10483-c29e-4a1c-9eb8-79d3197c1e35/insights-operator/1.log"
Sep 08 15:54:31.857627 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:31.857606    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pzl9w_dbacbcb2-59f9-4a4e-95a2-f42e2653eec4/kube-rbac-proxy/0.log"
Sep 08 15:54:31.878567 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:31.878544    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pzl9w_dbacbcb2-59f9-4a4e-95a2-f42e2653eec4/exporter/0.log"
Sep 08 15:54:31.901152 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:31.901132    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pzl9w_dbacbcb2-59f9-4a4e-95a2-f42e2653eec4/extractor/0.log"
Sep 08 15:54:32.747990 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:32.747961    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-czmnt/perf-node-gather-daemonset-76srx"
Sep 08 15:54:34.652995 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:34.652965    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-qlzp2_2c3b388f-7424-47df-8af6-a1c1dc717334/manager/0.log"
Sep 08 15:54:34.703026 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:34.702950    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7b994b57d8-fqs8r_caf1d3fd-2130-4426-b4b1-0d566242f364/manager/0.log"
Sep 08 15:54:34.808536 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:34.808509    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-kn4vl_5f1e5b0c-dddb-4859-8041-57996d5ec812/manager/0.log"
Sep 08 15:54:34.833768 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:34.833742    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-847d47f6c8-c7hnl_a72080d0-7eab-402c-a477-9d487435aab9/manager/0.log"
Sep 08 15:54:34.856340 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:34.856316    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-847d47f6c8-c7hnl_a72080d0-7eab-402c-a477-9d487435aab9/copy-manifests/0.log"
Sep 08 15:54:36.529435 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:36.529410    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-55vzg_9ddacf7a-3a11-4e36-a847-76f693bc9960/openshift-lws-operator/0.log"
Sep 08 15:54:36.985894 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:36.985862    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-kt4pd_a1a62f91-49cb-4c70-a3e0-92989a5ceb43/discovery/0.log"
Sep 08 15:54:37.016822 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:37.016796    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d44f46d57-r2s76_5bf49137-ceba-4cbc-8f05-25b821fcab99/kube-auth-proxy/0.log"
Sep 08 15:54:37.093470 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:37.093439    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-tscfp_231e053b-1392-4597-bc81-2437855e9687/payload-pre-processing/0.log"
Sep 08 15:54:37.120624 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:37.120594    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-zdsqr_77f92f27-262b-498d-a1fa-94ece7554033/payload-processing/0.log"
Sep 08 15:54:37.139645 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:37.139609    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-88497859b-8nkdj_5551e120-e4db-4d93-8d2a-9e3626db076e/router/0.log"
Sep 08 15:54:39.004823 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:39.004787    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-kt4pd_a1a62f91-49cb-4c70-a3e0-92989a5ceb43/discovery/0.log"
Sep 08 15:54:39.049297 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:39.049267    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d44f46d57-r2s76_5bf49137-ceba-4cbc-8f05-25b821fcab99/kube-auth-proxy/0.log"
Sep 08 15:54:39.130931 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:39.130902    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-tscfp_231e053b-1392-4597-bc81-2437855e9687/payload-pre-processing/0.log"
Sep 08 15:54:39.168212 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:39.168180    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-zdsqr_77f92f27-262b-498d-a1fa-94ece7554033/payload-processing/0.log"
Sep 08 15:54:39.185742 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:39.185716    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-88497859b-8nkdj_5551e120-e4db-4d93-8d2a-9e3626db076e/router/0.log"
Sep 08 15:54:42.581123 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:42.581091    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-857nk_21604064-e43d-4ec1-b6e9-9af32f6208ed/kube-storage-version-migrator-operator/1.log"
Sep 08 15:54:42.582200 ip-10-0-137-43 kubenswrapper[2560]: I0908 15:54:42.582180    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-857nk_21604064-e43d-4ec1-b6e9-9af32f6208ed/kube-storage-version-migrator-operator/0.log"